Data migration method and device, electronic equipment, storage medium and program product

Abstract

The invention provides a data migration method and device, electronic equipment, a storage medium and a program product, and relates to the technical field of IT and software development. In the embodiment of the invention, under the scene that the source cloud management data is migrated to the target cloud management, the influence of migration on the target cloud management can be reduced through a single-channel mode based on a data feature matching method, a data rechecking mechanism, idempotent operation and the like, and the consistency and integrity of the migrated data are ensured. In the embodiment of the invention, data migration is carried out by adopting the migration push channel which is independently deployed in a single channel, so that the decoupling of the data migration process and the target cloud management can be realized, and the influence of the data migration process on the normal business process of the target cloud management is reduced.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of IT and software development technology, and in particular to a data migration method, device, electronic device, storage medium and program product. Background Art

[0002] With the rapid development of information technology, cloud computing, as an efficient and flexible computing model, has been favored by more and more enterprises. Among them, private cloud and public cloud are two common cloud computing models. Private cloud is usually deployed within the enterprise and has a high degree of security and controllability; while public cloud provides elastic computing and storage resources, with the characteristics of low cost and fast expansion. With the rapid development of enterprise business, private cloud faces problems such as limited resources and high maintenance costs, while public cloud has become a more economical and efficient choice. Therefore, migrating private cloud data to public cloud has become a practical need for many enterprises. At present, cloud computing is mostly established in private cloud mode. In the actual operation process, its service model may not be able to meet the needs of users. More and more customers hope to migrate from private cloud to public cloud. The migration from private cloud to public cloud mainly involves the migration of cloud computing management platform (referred to as cloud management) data. Cloud management data refers to data related to cloud resource management, including configuration information, status information and operation logs of resources such as virtual machines, storage, and networks. The migration process faces many challenges:

[0003] 1. Challenges of data integrity and consistency in cross-platform migration: Since the source cloud management and the target cloud management use different data storage management methods, data format mismatch, data missing or different types may occur, which in turn leads to data inconsistency problems.

[0004] 2. Technical complexity and implementation difficulty challenges: Cross-platform data migration requires cross-platform integration and data management. The differences in the technology stacks of the two platforms will also increase technical complexity and implementation difficulty.

[0005] 3. Processing of customized attributes of source cloud management, mainly including non-standard specifications, customized image management methods, and differentiated capability strategies of source cloud management and target cloud management. Summary of the invention

[0006] Embodiments of the present invention provide a data migration method, device, electronic device, storage medium, and program product to at least partially solve the above-mentioned problems.

[0007] A first aspect of an embodiment of the present invention provides a data migration method, the method comprising:

[0008] Establishing a network channel between a target resource pool and a target cloud computing management platform so that the target cloud computing management platform can manage the target resource pool;

[0009] Receive user information of the source cloud computing management platform;

[0010] Establishing a user information mapping table based on the user information of the source cloud computing management platform and the user information of the target cloud computing management platform;

[0011] Determining users to be migrated based on the user information mapping table;

[0012] The cloud resource data related to the user to be migrated and the target resource pool is obtained through the data migration unit in the migration push channel.

[0013] Optionally, the method further comprises:

[0014] For users who have completed data migration and are waiting to be migrated, build order data to be pushed;

[0015] Adding a supplementary order identifier to the order data to be pushed, wherein the supplementary order identifier includes: a supplementary order identifier and a new order identifier;

[0016] The order data to be pushed carrying the order supplement mark is sent to the order push unit in the migration push channel.

[0017] Optionally, obtaining cloud resource data related to the target resource pool and the user to be migrated through a data migration unit in the migration push channel includes:

[0018] Reading the cloud resource data of the user to be migrated in the database of the source cloud computing management platform through the data migration unit;

[0019] Storing the cloud resource data in a cache;

[0020] Read the latest resource status data of the cloud resources in the target resource pool through the gateway of the target resource pool;

[0021] In the case where the cloud resource data in the cache is inconsistent with the latest resource status data, correcting the cloud resource data in the cache;

[0022] The corrected cloud resource data is saved to the database of the target cloud computing management platform.

[0023] Optionally, the to-be-migrated users include full-volume migration users, and reading the cloud resource data of the to-be-migrated users in the source cloud computing management platform database by the data migration unit includes:

[0024] Reading the full amount of cloud resource data of the user to be migrated in the source cloud computing management platform database through the data migration unit;

[0025] Complete the mapping comparison and field value conversion of the full cloud resource data to be migrated through the field mapping set;

[0026] The converted full cloud resource data is saved in the cache.

[0027] Optionally, the user to be migrated further includes an incrementally migrated user, and reading the cloud resource data of the user to be migrated in the source cloud computing management platform database by the data migration unit includes:

[0028] Read and cache the migrated resource information in the target cloud computing management platform database through the data migration unit;

[0029] Read resource information of users to be migrated in the source cloud computing management platform database through the data migration unit;

[0030] The data migration unit compares the migrated resource information in the cached target cloud computing management platform database with the resource information of the user to be migrated in the source cloud computing management platform database to determine the incremental cloud resource data information to be migrated;

[0031] Complete the mapping comparison and field value conversion of the incremental cloud resource data to be migrated through the field mapping set;

[0032] The converted incremental cloud resource data is saved in the cache.

[0033] Optionally, after the target cloud computing management platform manages the target resource pool, the method further includes:

[0034] Obtaining surplus resource information of the target resource pool;

[0035] Receiving a service request from a user terminal;

[0036] In response to the service request, and based on the surplus resource information, the surplus resources of the target resource pool are allocated to the user terminal.

[0037] A second aspect of an embodiment of the present invention provides a data migration device, the device comprising:

[0038] A management module, used to establish a network channel between a target resource pool and a target cloud computing management platform, so that the target cloud computing management platform can manage the target resource pool;

[0039] A receiving module, used for receiving user information of a source cloud computing management platform;

[0040] A mapping module, used to establish a user information mapping table based on the user information of the source cloud computing management platform and the user information of the target cloud computing management platform;

[0041] A determination module, used to determine the user to be migrated based on the user information mapping table;

[0042] The acquisition module is used to acquire the cloud resource data related to the user to be migrated and the target resource pool through the data migration unit in the migration push channel.

[0043] A third aspect of an embodiment of the present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the data migration method as described in the first aspect of the present invention when executed.

[0044] A fourth aspect of an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the data migration method as described in the first aspect of the present invention.

[0045] A fifth aspect of an embodiment of the present invention provides a computer program product, including a computer program / instruction, which is used by a processor to implement the steps in the data migration method described in the first aspect of the present invention.

[0046] In an embodiment of the present invention, in a scenario where data from a source cloud management system is migrated to a target cloud management system, the impact of the migration on the target cloud management system can be reduced through a single-channel mode, a data feature matching method, a data review mechanism, idempotent operations, and the like, thereby ensuring the consistency and integrity of the migrated data.

[0047] In the embodiment of the present invention, a single-channel independently deployed migration push channel is used to perform data migration, which can achieve decoupling of the data migration process and the target cloud management, and reduce the impact of the data migration process on the normal business process of the target cloud management.

[0048] In the embodiment of the present invention, during the data migration process, a feature adaptation method is used to implement the conversion of field values, solve the problem of matching data structure differences, reserve expansion space, and provide support for customized data migration.

[0049] In an embodiment of the present invention, the latest resource status data of the cloud resources in the target resource pool is read through the gateway of the target resource pool, and a secondary underlying data review is performed on the cloud resource data cached by the data migration unit based on the latest resource status data. This can solve the data consistency problems that may exist in the migration process and ensure the consistency of the migrated data.

[0050] In the embodiment of the present invention, a fallback mechanism for synchronizing and matching data at the granularity of single-user resources is used to cope with emergencies during the migration process, and the migration operation is idempotent and can be executed repeatedly.

[0051] In the embodiment of the present invention, a low-intrusive multiplexing order process with a supplementary order mark is used to realize supplementary delivery of an order, maximize the reuse of the original process, improve efficiency, and reduce costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0053] Figure 1 is a flowchart of the steps of the data migration method provided by an embodiment of the present invention;

[0054] Figure 2 is a schematic diagram of an implementation flow of a data migration method provided by an embodiment of the present invention;

[0055] Figure 3 is a flowchart of the steps of the order push process in the data migration method provided by an embodiment of the present invention;

[0056] Figure 4 It is a detailed flowchart of the data migration method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0057] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0058] A data migration method is disclosed in the related art, which includes: obtaining a data migration template of a source cloud platform; the data migration template includes configuration data of a first resource type; performing resource type conversion on the data migration template to obtain a target data migration template, the target data migration template includes configuration data of a second resource type required by the target cloud platform; creating a target host on the target cloud platform according to the configuration data of the second resource type in the target data migration template; and controlling the source cloud platform to send the source data to be migrated to the target host in the target cloud platform. Data migration across platforms and host resource types can be realized, and data migration efficiency can be improved.

[0059] It can be seen that this method mainly solves the problem of cross-platform data migration. It is necessary to create a host on the target platform. When both platforms are running, a new host cannot be created, that is, it cannot solve the problem of migrating existing data in scenarios where there is existing data.

[0060] A data migration method is also disclosed in the related technology, which includes: mounting a target storage block in a server in response to a mount instruction; wherein the target storage block stores target data of a source end; and then, migrating the data in the target storage block to a destination end in response to a migration instruction.

[0061] This method is mainly for achieving secure data migration, and cannot solve the data integrity and consistency issues in the data migration scenario.

[0062] In an embodiment of the present invention, an in-depth study is conducted on the above-mentioned problems, especially in the scenario of migrating data from a source cloud management system to a target cloud management system, how to perform data consistency verification, ensure data integrity, and reduce the impact of data migration on the target cloud management system.

[0063] Based on this, an embodiment of the present invention provides a data migration method, which can reduce the impact of migration on the target cloud management system and ensure the consistency and integrity of the migrated data through a single-channel mode, a data feature matching method, a data review mechanism, idempotent operations, etc. in the scenario where the source cloud management data is migrated to the target cloud management system.

[0064] Specifically, Figure 1 As shown, Figure 1 The data migration method provided by the embodiment of the present invention is shown in the flowchart of the steps. The data migration method includes the following steps:

[0065] S101, establishing a network channel between a target resource pool and a target cloud computing management platform, so that the target cloud computing management platform can manage the target resource pool.

[0066] S102: Receive user information from a source cloud computing management platform.

[0067] S103: Establish a user information mapping table based on the user information of the source cloud computing management platform and the user information of the target cloud computing management platform.

[0068] S104: Determine users to be migrated based on the user information mapping table.

[0069] S105: Obtain cloud resource data related to the user to be migrated and the target resource pool through the data migration unit in the migration push channel.

[0070] In the embodiment of the present invention, the above steps S101 to S105 may be executed by the target cloud computing management platform.

[0071] In the embodiment of the present invention, the cloud computing management platform refers to an interface or tool that provides users with centralized management of cloud computing resources.

[0072] Resource pool management is a standard process of cloud management. First, you need to ensure that the network between the resource pool and the cloud management is reachable. The cloud management can exchange data and issue instructions through the network and the gateway of the resource pool.

[0073] In the embodiment of the present invention, the target resource pool has completed the resource pool construction and has carried user services, so the resource pool management only requires the network connection between the target cloud management and the source resource pool to proceed to the next management operation.

[0074] During the resource pool management process, the resource pool information of the target resource pool of the source cloud management needs to be provided to the target cloud management for configuration. After the configuration is completed, the core function process is verified. Once the verification passes, the resource pool management operation is completed.

[0075] In an embodiment of the present invention, the resource pool information includes necessary information such as the resource pool name, resource pool type, resource pool unique identifier, resource pool gateway address, resource pool authentication token, resource pool VNC entry address, and information such as products and functions that the resource pool can support.

[0076] Among them, the resource pool type can include various types of resource pools, such as the first-generation CloudStack resource pool, the second-generation OpenStack resource pool, the third-generation self-developed resource pool, etc.

[0077] In the embodiment of the present invention, when the resource pool is successfully managed, a network channel is established between the target resource pool and the target cloud computing management platform to achieve data interaction and instruction issuance between the cloud management and the resource pool.

[0078] In the embodiment of the present invention, after the management is completed, the target cloud management can obtain the surplus information of the managed target resource pool, and can directly allocate resources for the surplus resources to undertake user services.

[0079] Specifically, in the embodiment of the present invention, after the target cloud computing management platform manages the target resource pool, the method further includes:

[0080] S1, obtaining surplus resource information of the target resource pool.

[0081] S2, receiving a service request from a user terminal.

[0082] S3: In response to the service request, allocating the surplus resources of the target resource pool to the user terminal based on the surplus resource information.

[0083] In the embodiment of the present invention, for the target resource pool that needs to be migrated, it is first treated as a new resource pool and managed by the target cloud management, so that the target cloud management can directly operate and allocate the idle resources of the target resource pool.

[0084] For example, in the scenario of migrating data from a private cloud to a public cloud, data migration between cloud management is required. For example, if a private cloud manages a resource pool, during the migration process, the resource pool managed by the private cloud is first managed on the public cloud. The cloud management of the public cloud cannot directly obtain the resources that have been allocated in the resource pool, but after the management is completed, it can operate and allocate the idle resources in the resource pool.

[0085] For the cloud resource data of the allocated resources, data migration is required. The private cloud needs to synchronize the relationship between these resources and users (that is, the cloud resource data related to the users to be migrated and the resource pool) to the public cloud. The public cloud records these allocated resources and completes the migration of private cloud data to the public cloud.

[0086] In an embodiment of the present invention, data migration can be performed while the target cloud management is processing business requests normally. Specifically, in an embodiment of the present invention, a migration push channel is deployed between the source cloud management and the target cloud management in a single-channel independent deployment manner, and data migration and order push are performed based on the migration push channel. Specifically, the migration push channel includes: a data migration unit and an order push unit. Among them, the data migration unit is used to complete data migration, and the order push unit is used to complete order push.

[0087] For ease of understanding, the following combination Figure 2 The data migration method provided by the embodiment of the present invention is explained. Figure 2 FIG. 1 is a schematic diagram showing an implementation flow of a data migration method provided by an embodiment of the present invention. Figure 2 As shown, in an embodiment of the present invention, the source cloud management user is first pushed to the unified user authentication unit of the target cloud management, and then the migration of the cloud resource data of the source cloud management to the cloud resource data of the target cloud management is completed through the data migration unit, and finally the order supplement operation is completed through the order push unit to establish the association relationship between the order and the migrated cloud resources.

[0088] Specifically, in an embodiment of the present invention, user authentication of the source cloud management and the current cloud management is completed through a unified user authentication unit, so that in subsequent steps a user information mapping table can be established based on the user information of the source cloud computing management platform and the user information of the target cloud computing management platform.

[0089] In the embodiment of the present invention, the source cloud management pushes its own user information to the unified user authentication unit, and the target cloud management then obtains the user information of the source cloud management based on the unified user authentication unit.

[0090] For example, assuming there is a user A on the source cloud management system and a user a on the target cloud management system, and both user A and user a have been authenticated by a unified user authentication unit, then after the user information of the source cloud management system is pushed to the target cloud management system, the target cloud management system can establish a mapping relationship between the two, for example, it can be determined that user A and user a are the same user.

[0091] In the embodiment of the present invention, the user information on the source cloud management cannot be automatically synchronized to the target cloud management. After the resource pool is managed to the target cloud management, the source cloud management can push the user information to the target cloud management. There are many ways to push. The most common one is to push through the API interface, export the user information of the source cloud management, assemble it according to the requirements of the target cloud management interface parameters, and push the user information through batch scripts. After the push is successful, the target cloud management will create new user information. You can also enter or import user information through the Dashboard. After the entry or import is successful, the target cloud management will create new user information.

[0092] In the embodiment of the present invention, the user information push of the source cloud management can specifically support three methods. Method 1 is API push, which organizes the user information of the source cloud management through batch scripts. The following is a possible example:

[0093] API:POST / api / v1 / accounts

[0094] Request body (JSON format):

[0095] {

[0096] "username":"newuser1",

[0097] "email":"newuser1@example.com",

[0098] "mobile":"15300000000",

[0099] "source":"channel",

[0100] "payType":"Payment type"

[0101] }

[0102] Successful response (JSON format):

[0103]

[0104] The second method is to operate through the Dashboard, enter user information, and submit the form to complete the user information push.

[0105] The third method is to import template information through the Dashboard, usually a CSV or Excel template, organize the source cloud management user information according to the template content, and complete the import operation.

[0106] In an embodiment of the present invention, after the target cloud management obtains the user information of the source cloud management, a unique identifier can be generated in the target cloud management. The unique identifier can be a user ID. Specifically, the user's email address + mobile phone number can be used as the user identifier to associate the source cloud management user and the target cloud management user. The source cloud management user's ID and the target cloud management user ID are associated to establish a user information mapping table.

[0107] The following is an example of a possible mapping table structure design:

[0108]

[0109]

[0110] Among them, mapping_id is the unique identifier of each mapping record, source_user_id and target_user_id represent the user identifiers of the source cloud management and the target cloud management respectively, email and mobile are redundant fields used to confirm user information, resource_type and resource_id describe the resource type and its unique identifier, status indicates the status of the migration, created_at and updated_at record the creation and last update time of the record respectively, and migr_flag is the migration flag, 0 represents migration, and non-zero represents the number of migrations.

[0111] In the embodiment of the present invention, the user to be migrated can be determined through the user information mapping table, and whether the user to be migrated is a full migration user or an incremental migration user can be determined through the migration flag.

[0112] In an embodiment of the present invention, the data migration unit requires access to the database of the source cloud management and the resource pool gateway of the source cloud management. The order push unit requires access to the database of the target cloud management and is in the same network environment as the target cloud management. The unified order unit can be accessed to complete the order push operation. After the order is pushed successfully, the order message will be received by the target cloud management and notify the order push unit to complete the order information association operation.

[0113] In the embodiment of the present invention, the order push time is selected by analyzing the historical data of monitoring, selecting the minimum load period of the target cloud management, and migrating and verifying each user one by one.

[0114] In the embodiment of the present invention, migration supports single-user sequential migration and multi-user synchronous migration, and can be flexibly selected according to the load of the target cloud management and the number of users to be migrated.

[0115] In an embodiment of the present invention, in order to achieve the decoupling of the data migration process and the normal business process of the target cloud management and reduce the impact of data migration on the target cloud management, the order push unit and the data migration unit are independently deployed in a single channel, and the minimum load period of the target cloud management is obtained by monitoring data, and migration and verification are performed one by one according to users during the minimum load period.

[0116] Specifically, in the data migration method provided in the embodiment of the present invention, the method further includes:

[0117] S106, constructing order data to be pushed for the users to be migrated who have completed data migration;

[0118] S107, adding a supplementary order identifier to the order data to be pushed, wherein the supplementary order identifier includes: a supplementary order identifier and a new order identifier;

[0119] S108, sending the order data to be pushed carrying the order replacement identifier to the order pushing unit in the migration push channel.

[0120] In an embodiment of the present invention, based on the above steps S101 to S105, the cloud resource data of the source cloud management database can be synchronized to the database of the target cloud management. Since the billing modes of the source cloud management and the target cloud management may be different, the order supplementary delivery process must be carried out according to different situations. In situation one, the billing modes of the source cloud management and the target cloud management are the same, and the original billing model and orders can be continued, then the order supplementary process can be directly entered. In situation two, the billing modes of the source cloud management and the target cloud management are different, and the order supplementary delivery needs to be carried out according to the billing mode of the target cloud management.

[0121] In the embodiment of the present invention, based on the principle of minimum cost and reusability, the method of constructing order data and creating new order data is consistent, and the original order placement interface can be reused to perform order replenishment.

[0122] In an embodiment of the present invention, the order data includes attributes related to price, such as ACCOUNT_ID representing the unique identifier of the ordering user, SERVICE_TYPE representing the product type, RESOURCE_TYPE representing the resource type, and the resource price can be determined by the product type and the resource type. RES_ITEM_CONFIG can be a string in JSON format, including resource specifications, such as CPU, memory and other necessary resource attributes. The final order data is presented in JSON or other forms supported by the order interface.

[0123] In the embodiment of the present invention, a replenishment order mark REVERSE_SYNC_FLAG can be added to the constructed order data, and its value is 0 and 1, 0 represents a new order, and 1 represents a replenishment order. The replenishment order mark is used to identify whether it is a new order or a replenishment order. For a new order, cloud resources need to be actually allocated, while a replenishment order only needs to establish an association with existing resources, and there is no need to actually allocate cloud resources. At the same time, OLD_RESOURCE_ID is added to the existing resource ID, and its value is the cloud resource ID migrated to the target cloud management, which is used to associate with the newly created replenishment order.

[0124] In an embodiment of the present invention, the order push unit receives an order message provided by the order placement interface, and parses the order supplement identifier from the message body. If the order supplement identifier REVERSE_SYNC_FLAG exists and its value is 1, the actual step of allocating cloud resources is skipped, and the order and cloud resource association link is directly entered. The cloud resource OLD_RESOURCE_ID information contained in the order is parsed, and the migrated cloud resource information is queried through the value. The queried record is associated with the ORDER_ID of the newly created order, that is, the main order ID, to complete the association operation of the order supplement resource and the new order.

[0125] In the embodiment of the present invention, for ease of understanding, the following Figure 3 The order push process in the embodiment of the present invention is explained. In the embodiment of the present invention, after completing the cloud resource data migration from the source cloud management to the target cloud management, the user's order data is missing, and it is necessary to perform an order replenishment operation according to the billing mode of the source cloud management and the target cloud management. If the billing mode of the source cloud management and the target cloud management is the same, directly enter the order replenishment process. If the billing mode of the source cloud management and the target cloud management is different, it is necessary to perform the order replenishment according to the billing mode of the target cloud management.

[0126] Specifically, in the embodiment of the present invention, first, the order push unit reads the N user information of the order to be pushed, and constructs the order data based on the user information, which can be implemented through the order placement interface. In the embodiment of the present invention, the order placement interface usually supports parameter organization methods such as JSON and XML, including the necessary user unique identifier, product type, resource type, resource specification and other information. The following is a partial example of order data:

[0127]

[0128]

[0129] Where order_id is the unique identifier of the order, account_id is the account ID of the ordering user, order_config is the order-related configuration information, pay_pattern is the billing mode, M represents monthly billing, creation_time is the time when the order was created, and status indicates the current status of the order.

[0130] If the billing model of the source cloud management system and the target cloud management system is the same, the order-related information can be obtained through the order data or resource data of the source cloud management system. If the billing model of the source cloud management system and the target cloud management system is different, it is necessary to match the corresponding information from the source cloud management system according to the billing model of the target cloud management system. If the match fails, it is necessary to consider using the default value or specified value to fill the gap.

[0131] Furthermore, the order push unit adds a replenishment order identifier to the constructed order data.

[0132] Specifically, in the constructed order data, an example of adding a replenishment order identifier and an existing resource ID is as follows:

[0133]

[0134]

[0135] Among them, reverse_sync_flag is the flag for supplementary order, and its value is 0 and 1, 0 represents a new order, and 1 represents a supplementary order. Old_resource_id is the cloud resource ID migrated to the target cloud management, which is used to associate the newly created supplementary order with the migrated cloud resources.

[0136] In the embodiment of the present invention, the order push unit pushes the order data to the unified order center, and the target cloud management (eg, public cloud) further receives the resource tasks issued by the order center to complete the association between the replenishment resources and the new order.

[0137] In an embodiment of the present invention, the order push unit receives an order message provided by the order placement interface, and parses the reverse_sync_flag from the message body. If the reverse_sync_flag is 0, it means that it is a new order and cloud resources need to be actually allocated. If the reverse_sync_flag is 1, the step of actually allocating cloud resources is skipped, and the information of the cloud resources that have been migrated is queried through the old_resource_id. The parsed order_id is updated to the resource and order association table to complete the association between the order-making resources and the new order.

[0138] In the embodiment of the present invention, the order push unit is independently deployed using a single channel. Therefore, when a target resource pool needs to be migrated, the migration push channel can be opened. This channel is decoupled from the target cloud management, and the order push process has no impact on the normal service of the target cloud management. Furthermore, in the scenario where order push is required, the migration push channel can be opened to push the order to the unified order unit. The target cloud management can obtain the new orders generated by the regular business process and the orders constructed by the order push unit from the unified order unit to complete the associated operations of the replenishment resources and the new orders.

[0139] In the embodiment of the present invention, the order pushing unit may push orders one by one or in batches.

[0140] In the embodiment of the present invention, all steps support single-user resource granularity data rollback, each migration step is reversible and recoverable, and the operation needs to be idempotent and can be continued or rolled back from any interruption position. The migration log contains basic information, resource information, change details, operable information, rollback status and additional information. The basic information is characterized by mainly including: log ID, operation time, operation type, operation status, operator / system.

[0141] Resource information mainly includes: resource ID, resource type, source location, and target location. Change details include: state before change, state after change, and change description. Reversible operation information includes: rollback operation, rollback status, and additional information includes: related log ID (if the operation is related to other log records, record the IDs of these logs), notes (any additional information, such as the context of the operation, special instructions, etc.)

[0142] In an embodiment of the present invention, a migration log can be recorded for each step of data migration. The log is recorded to the resource granularity and records detailed change information of the resources. Each migration step is reversible and recoverable, ensuring that the operation is idempotent and can be continued or rolled back from any interruption location, providing flexible operation options.

[0143] In the embodiment of the present invention, based on the above embodiment, step S105 includes:

[0144] S1051, reading the cloud resource data of the user to be migrated from the database of the source cloud computing management platform through a data migration unit, and storing the cloud resource data in a cache.

[0145] S1052, reading the latest resource status data of the cloud resources in the target resource pool through the gateway of the target resource pool;

[0146] When the cloud resource data in the cache is inconsistent with the latest resource status data, the cloud resource data in the cache is corrected.

[0147] S1053, saving the corrected cloud resource data to the database of the target cloud computing management platform.

[0148] In the embodiment of the present invention, when migrating data from the source cloud management system to the target cloud management system, it is first necessary to determine the information of the users to be migrated, minimize the number of target users to be migrated, and do not need to migrate test users or users who have withdrawn from the network. Secondly, it is necessary to ensure that the source cloud management database and the target cloud management database are network-accessible to the data migration unit, that is, the data migration unit can access the source cloud management database through the network, minimize permission access, that is, the data migration unit can only access the source cloud management database. At the same time, it is necessary to ensure that the data migration unit can access the target cloud management database through the network, minimize permission access, that is, it can access and operate the data-related tables of the source cloud management database. After the network connectivity check of the source database and the target database is passed, and the permission check is passed, the data migration step can be entered.

[0149] In an embodiment of the present invention, after the data migration unit obtains the cloud resource data to be migrated from the database of the source cloud computing management platform, the data is saved in the cache area to perform a second review of the data. Specifically, the cached data is used as the data to be corrected, and the target resource pool is accessed to obtain the latest resource status data of the cloud resources as the correction data. The data to be corrected is corrected according to the correction data. The specific correction content includes but is not limited to the elastic cloud host status, cloud hard disk binding or unbinding status, elastic IP binding or unbinding status, etc. The corrected data is saved to the target cloud management database through the data migration unit to complete the migration of the cloud resource data.

[0150] In the embodiment of the present invention, the elastic cloud host is a scalable computing service that allows users to adjust computing resources at any time according to actual needs. Elastic IP is a network service that allows users to assign a fixed public IP address to a cloud server. Cloud hard disk is a storage service that provides the ability to store data persistently.

[0151] Taking the cloud host status as an example, if the status of the synchronized data obtained from the cache is RUNNING, and the status obtained from the resource pool bottom layer is STOPPED, the status is inconsistent. The status is corrected using the resource pool bottom layer status as the correction value.

[0152] After all cached data is corrected, it is saved to the target cloud management database through the data migration unit to complete the migration of cloud resource data.

[0153] In the embodiment of the present invention, during the data migration process, the data migration unit may migrate the target resource pools one by one.

[0154] In the embodiment of the present invention, when the user to be migrated is a full migration user, step S1051 includes: reading the cloud resource data of the user to be migrated in the source cloud computing management platform database by the data migration unit, including:

[0155] S11, reading, through a data migration unit, all cloud resource data of the users to be migrated in a source cloud computing management platform database.

[0156] S12, completing the mapping comparison and field value conversion of the full cloud resource data to be migrated through the field mapping set.

[0157] S13, saving the converted full cloud resource data in the cache.

[0158] In an embodiment of the present invention, firstly, the data migration unit reads one or more unique identifiers of users to be migrated, and first queries the migration flag MIGR_FlAG from the target cloud management database, whose default value is 0. The default value is set when a new user is created. If the migration flag is 0, it means that it has not been migrated yet, and the user information needs to be fully migrated. The data migration unit reads the full amount of data of the user's cloud resources to be migrated from the source cloud management database, including but not limited to elastic cloud host information, cloud hard disk information, elastic IP, VPC, subnet, mirror information, etc., and maps the fields of the information to be migrated according to the migration field mapping table of the source cloud management database and the target cloud management database. After the mapping conversion is completed, a secondary review of the underlying migration data is performed. If the migration flag is 1, it means that it has been migrated and incremental migration is required.

[0159] In the embodiment of the present invention, during the feature-based field mapping process, two sets are first set:

[0160] Source field set: S = {s1, s2, ..., s n};

[0161] Target field set: T = {t1, t2, ..., t m};

[0162] Set the field mapping to a mapping function f:S→T, where for each source field s i ∈S, there is a target field t j ∈T such that f(s i ) = t j In some cases, not all source fields may find corresponding target fields, or there may be multiple source fields mapped to the same target field. In this case, feature conversion rules are needed to solve the problem. The feature conversion rule is set as function g, which accepts the value of the source field as input and returns the converted value.

[0163] For each cloud resource record s in the source cloud management database, apply the field mapping function f and the conversion function g to obtain the value of the target field and construct the target record vector t, where t j =g(s i )(If f(s i ) = t j and there exists a conversion rule), otherwise t j =s i (If directly mapped) Insert the target record vector t into the target database after passing the underlying secondary review.

[0164] Here is a possible example:

[0165]

[0166] The feature conversion function is used to convert field values ​​to adapt to the structure or format requirements of the target cloud management database. The following is an example of a feature conversion function:

[0167] Disk size conversion function - diskSizeConverter

[0168] The source cloud management and the target cloud management may have different units for disk size. For example, the source cloud management may use GB as the unit, while the target cloud management may use MB. Based on the above characteristics, the following conversion function is set to handle this difference:

[0169] def diskSizeConverter(source_disk_size):

[0170] #Conversion logic: 1GB = 1024MB

[0171] target_disk_size=source_disk_size*1024

[0172] return target_disk_size

[0173] The migration operation needs to traverse the field mapping collection. For directly mapped fields, the values ​​are directly copied. For fields that need to be converted, the corresponding feature conversion function is called to perform field conversion.

[0174] Specifically, when the user to be migrated is an incremental migration user, step S1051 includes:

[0175] S21, reading and caching the migrated resource information in the target cloud computing management platform database through the data migration unit.

[0176] S22: Read resource information of the user to be migrated in the source cloud computing management platform database through the data migration unit.

[0177] S23, the data migration unit compares the migrated resource information in the cached target cloud computing management platform database with the resource information of the users to be migrated in the source cloud computing management platform database to determine the incremental cloud resource data to be migrated.

[0178] S24, completing the mapping comparison and field value conversion of the incremental cloud resource data to be migrated through the field mapping set.

[0179] S25, saving the converted incremental cloud resource data in the cache.

[0180] In an embodiment of the present invention, for migrated users, the data migration unit may first read the resource information that has been migrated from the target cloud management database for caching, and then read the resource information from the source cloud management database for comparison, filter out the cloud resource information that needs to be incrementally migrated, and perform field mapping of the information to be migrated according to the migration field mapping tables of the source cloud management database and the target cloud management database. After completing the mapping conversion, the migration data is reviewed for the second time.

[0181] In the embodiment of the present invention, the specific steps of mapping conversion are similar to the above step S12 and will not be repeated here.

[0182] For ease of understanding, the following combination Figure 4 The data migration method provided by the embodiment of the present invention is explained. Figure 4 A detailed flow chart of a data migration method provided by an embodiment of the present invention is shown.

[0183] Specifically, in the embodiment of the present invention, the preconditions of the data migration unit may be verified before data migration, which requires the following steps:

[0184] S31, check whether the data migration unit is reachable to the source cloud management database network.

[0185] S32, verify the read-only account permission of the source cloud management database, and can query the source cloud management database data and return it.

[0186] S33, check whether the data migration unit is reachable to the target cloud management database network.

[0187] S34, verify the read and write account permissions of the target cloud management database, and can query, insert, and update resource data.

[0188] S35, check whether the network from the data migration unit to the resource pool of the source cloud management is reachable.

[0189] S36, checking whether the data migration unit is accessible to the gateway of the target resource pool, and whether the provided authorization key can access the interface of the resource pool gateway and obtain a correct response.

[0190] In the embodiment of the present invention, if any of the above verification steps fails, a migration failure log needs to be recorded and the data migration is terminated.

[0191] When the verification is passed, the user to be migrated of the source cloud computing management platform is read through the data migration unit.

[0192] It is further checked whether the migration flag is 0, that is, whether the user to be migrated is a full migration user or an incremental migration user.

[0193] If the value is not 0, that is, the user to be migrated is an incremental migration user, perform the following steps:

[0194] The data migration unit reads the migrated resource information of the target cloud management database; the data migration unit caches the migrated resource information of the target cloud management database; the data migration unit reads the resource information of the users to be migrated in the source cloud management database; and calculates the incremental cloud resource information to be migrated.

[0195] When the value is 0, that is, the user to be migrated is a full migration user, the following steps are performed: the data migration unit reads the full cloud resource information of the user to be migrated from the cloud management database.

[0196] After obtaining the full cloud resource information or the incremental cloud resource information, the data migration unit completes the mapping and comparison of the cloud resource information to be migrated through the field mapping set.

[0197] Subsequently, the data migration unit stores the converted cloud resource information into a cache and conducts a second review of the migrated data. After the review is completed, the data migration unit saves the corrected data to the target cloud management database to complete the data migration.

[0198] Based on the same inventive concept, an embodiment of the present invention further provides a data migration device, the device comprising:

[0199] A management module, used to establish a network channel between a target resource pool and a target cloud computing management platform, so that the target cloud computing management platform can manage the target resource pool;

[0200] A receiving module, used for receiving user information of a source cloud computing management platform;

[0201] A mapping module, used to establish a user information mapping table based on the user information of the source cloud computing management platform and the user information of the target cloud computing management platform;

[0202] A determination module, used to determine the user to be migrated based on the user information mapping table;

[0203] The acquisition module is used to acquire the cloud resource data related to the user to be migrated and the target resource pool through the data migration unit in the migration push channel.

[0204] Optionally, the device further comprises:

[0205] A construction module is used to construct order data to be pushed for users to be migrated who have completed data migration;

[0206] An adding module is used to add a supplementary order identifier to the order data to be pushed, wherein the supplementary order identifier includes: a supplementary order identifier and a new order identifier;

[0207] The sending module is used to send the order data to be pushed carrying the order supplement mark to the order push unit in the migration push channel.

[0208] Optionally, the acquisition module includes:

[0209] A cache submodule, configured to read the cloud resource data of the user to be migrated from the database of the source cloud computing management platform through the data migration unit, and store the cloud resource data in the cache;

[0210] The reading submodule is used to read the latest resource status data of the cloud resources in the target resource pool through the gateway of the target resource pool;

[0211] A correction submodule, configured to correct the cloud resource data in the cache when the cloud resource data in the cache is inconsistent with the latest resource status data;

[0212] The saving submodule is used to save the corrected cloud resource data to the database of the target cloud computing management platform.

[0213] Optionally, the users to be migrated include full migration users, and the cache submodule is specifically used to:

[0214] Reading the full amount of cloud resource data of the user to be migrated in the source cloud computing management platform database through the data migration unit;

[0215] Complete the mapping comparison and field value conversion of the full cloud resource data to be migrated through the field mapping set;

[0216] The converted full cloud resource data is saved in the cache.

[0217] Optionally, the user to be migrated includes an incrementally migrated user, and the cache submodule is specifically used to:

[0218] Read and cache the migrated resource information in the target cloud computing management platform database through the data migration unit;

[0219] Read resource information of users to be migrated in the source cloud computing management platform database through the data migration unit;

[0220] The data migration unit compares the migrated resource information in the cached target cloud computing management platform database with the resource information of the user to be migrated in the source cloud computing management platform database to determine the incremental cloud resource data information to be migrated;

[0221] Complete the mapping comparison and field value conversion of the incremental cloud resource data to be migrated through the field mapping set;

[0222] The converted incremental cloud resource data is saved in the cache.

[0223] Optionally, the device also includes: an allocation module, used to obtain the remaining resource information of the target resource pool; receive a service request from a user terminal; and in response to the service request, allocate the remaining resources of the target resource pool to the user terminal based on the remaining resource information.

[0224] Based on the same inventive concept, an embodiment of the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes, the steps in the data migration method described in any of the above embodiments are implemented.

[0225] Based on the same inventive concept, an embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps in the data migration method described in any of the above embodiments are implemented.

[0226] Based on the same inventive concept, an embodiment of the present invention provides a computer program product, including a computer program / instruction, which implements the steps of the data migration method described in any of the above embodiments when executed by a processor.

[0227] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0228] It will be appreciated by those skilled in the art that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0229] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (apparatus), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0230] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0231] These computer program instructions can also be loaded onto a computer or other programmable terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0232] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0233] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0234] The above is a detailed introduction to a data migration method provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A data migration method, characterized in that: The method comprises: Establishing a network channel between a target resource pool and a target cloud computing management platform so that the target cloud computing management platform can manage the target resource pool; Receive user information of the source cloud computing management platform; Establishing a user information mapping table based on the user information of the source cloud computing management platform and the user information of the target cloud computing management platform; Determining users to be migrated based on the user information mapping table; The cloud resource data related to the user to be migrated and the target resource pool is obtained through the data migration unit in the migration push channel.

2. The data migration method according to claim 1, characterized in that: The method further comprises: For users who have completed data migration and are waiting to be migrated, build order data to be pushed; Adding a supplementary order identifier to the order data to be pushed, wherein the supplementary order identifier includes: a supplementary order identifier and a new order identifier; The order data to be pushed carrying the order supplement mark is sent to the order push unit in the migration push channel.

3. The data migration method according to claim 1, characterized in that: Obtaining cloud resource data related to the target resource pool and the user to be migrated through the data migration unit in the migration push channel includes: Reading the cloud resource data of the user to be migrated in the database of the source cloud computing management platform through the data migration unit, and storing the cloud resource data in the cache; Read the latest resource status data of the cloud resources in the target resource pool through the gateway of the target resource pool; In the case where the cloud resource data in the cache is inconsistent with the latest resource status data, correcting the cloud resource data in the cache; The corrected cloud resource data is saved to the database of the target cloud computing management platform.

4. The data migration method according to claim 3, characterized in that: The users to be migrated include full migration users, and reading the cloud resource data of the users to be migrated in the source cloud computing management platform database by the data migration unit, and storing the cloud resource data in the cache includes: Reading the full amount of cloud resource data of the user to be migrated in the source cloud computing management platform database through the data migration unit; Complete the mapping comparison and field value conversion of the full cloud resource data to be migrated through the field mapping set; The converted full cloud resource data is saved in the cache.

5. The data migration method according to claim 4, characterized in that: The users to be migrated also include incremental migration users. The cloud resource data of the users to be migrated in the source cloud computing management platform database is read by a data migration unit, and the cloud resource data is stored in a cache, including: Read and cache the migrated resource information in the target cloud computing management platform database through the data migration unit; Read resource information of users to be migrated in the source cloud computing management platform database through the data migration unit; The data migration unit compares the migrated resource information in the cached target cloud computing management platform database with the resource information of the user to be migrated in the source cloud computing management platform database to determine the incremental cloud resource data information to be migrated; Complete the mapping comparison and field value conversion of the incremental cloud resource data to be migrated through the field mapping set; The converted incremental cloud resource data is saved in the cache.

6. The data migration method according to any one of claims 1 to 5, characterized in that: After the target cloud computing management platform manages the target resource pool, the method further includes: Obtaining surplus resource information of the target resource pool; Receiving a service request from a user terminal; In response to the service request, and based on the surplus resource information, the surplus resources of the target resource pool are allocated to the user terminal.

7. A data migration device, characterized in that: The device comprises: A management module, used to establish a network channel between a target resource pool and a target cloud computing management platform, so that the target cloud computing management platform can manage the target resource pool; A receiving module, used for receiving user information of a source cloud computing management platform; A mapping module, used to establish a user information mapping table based on the user information of the source cloud computing management platform and the user information of the target cloud computing management platform; A determination module, used to determine the user to be migrated based on the user information mapping table; The acquisition module is used to acquire the cloud resource data related to the user to be migrated and the target resource pool through the data migration unit in the migration push channel.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the data migration method described in any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the data migration method described in any one of claims 1 to 6 is implemented.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the steps in the data migration method described in any one of claims 1-6 are implemented.

Images