A database fast migration method, system, device and storage medium

Abstract

The application relates to a database fast migration method, system and device and a storage medium, wherein the method comprises the following steps: obtaining the SAN disk condition of a production area division needing to be migrated, and recording the corresponding storage volume; obtaining a storage volume ID, creating a snapshot according to the storage volume ID, obtaining a snapshot name, and then obtaining a snapshot ID; cloning the snapshot, obtaining a new snapshot, and recording a new snapshot name; querying a host list through the storage volume ID, and establishing a mapping of the new snapshot to the host; after the mapping is created, querying a WWN according to the storage volume name, and querying a corresponding WWID based on the new snapshot name; the host scans the storage volume to perform mapping checking, and whether the storage volume is mapped is confirmed through the WWID; and a new DB cluster is created according to the WWID. Compared with the prior art, the application has the advantages of simple process, effective guarantee of data integrity and consistency, and saving of database configurations such as user permissions in a source database.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of database, in particular to a database fast migration method, system, device and storage medium. BACKGROUND

[0002] PolarDB is a new generation of relational cloud-native database independently developed by Ali Cloud, which has the low-cost advantage of distributed design and the ease of use of centralized design. PolarDB adopts storage computing separation and soft and hard integrated design to meet the needs of large-scale application scenarios. In the design of distributed systems, there are many heterogeneous / migration needs between data, and the use scenarios of database migration are very wide. In the traditional database migration mode, there are generally offline migration and online migration, and the data of the source storage warehouse is migrated through the data transmission service (DTS), to realize full or incremental migration. However, this migration mode is time-consuming and labor-intensive, and the preparation time before migration is long. In the new library of migration, the user permissions and other database initialization configurations need to be re-set, the environment requirements for data migration are high, the process is complex, and the data migration efficiency is low. SUMMARY

[0003] The purpose of the present application is to overcome the defects of the prior art and provide a database fast migration method, system, device and storage medium. The present application is based on PolarDB database and proposes a storage replication migration mode. The production divided SAN disk is stored by storage replication, and the production instance is mounted after replication, so as to realize the fast migration of the database. The operation process of the present application mode is simple, and the storage replication method can effectively ensure the integrity and consistency of the data. At the same time, the data disk saved in the source library saves the user permissions and other database configurations. Based on the distributed storage architecture, the database can be quickly migrated through the mode when the polarstack cluster is abnormal, so as to quickly enable the database service.

[0004] The purpose of the present application can be realized by the following technical solutions:

[0005] A database fast migration method based on PolarDB database, comprising the following steps:

[0006] S1, obtaining the SAN disk condition of the production area divided by the database to be migrated, and recording the corresponding storage volume;

[0007] S2, obtaining the storage volume ID, creating a snapshot according to the storage volume ID, obtaining the snapshot name, and then obtaining the snapshot ID;

[0008] S3, cloning the snapshot to obtain a new snapshot, and recording the new snapshot name;

[0009] S4, mapping the new snapshot to the host list is established by querying the host list through the storage volume ID;

[0010] S5, after the mapping is created, the WWN is queried according to the storage volume name, and the corresponding WWID is obtained based on the new snapshot name;

[0011] S6, the host scans the storage volume for mapping inspection, and confirms whether the storage volume is mapped through the WWID;

[0012] S7, a new DB cluster is created according to the WWID.

[0013] Further, in step S3, the mapping of the new snapshot to the host is specifically established by establishing the mapping of the obtained storage volume ID to the corresponding host list.

[0014] Further, in step S3, when the business continuity requirement is high, the immediate available strategy is selected when cloning the snapshot, so that the data in the cluster is maintained in a readable and writable state.

[0015] Further, in step S4, the mapping of the new snapshot is performed for all hosts in the cluster in the host list.

[0016] Further, in step S7, when creating a new cluster, the data disk is not formatted.

[0017] Further, the method further comprises step S8: connecting the database to verify whether the new DB cluster data exists and is complete.

[0018] A database migration system, comprising a storage volume query module, a snapshot creation module, a snapshot cloning module, a host list query module, a mapping module and a new cluster creation module;

[0019] The storage volume query module is used to obtain the SAN disk condition of the production area division where the database to be migrated is located, and record the corresponding storage volume.

[0020] The snapshot creation module is used to obtain the storage volume ID, create a snapshot according to the storage volume ID, obtain the snapshot name, and then obtain the snapshot ID;

[0021] The snapshot cloning module is used to clone the snapshot, obtain a new snapshot, and record the new snapshot name;

[0022] The host list query module is used to query the host list through the storage volume ID;

[0023] The mapping module is used to establish the mapping of the new snapshot to the host, and to confirm whether the storage volume is mapped;

[0024] The new cluster creation module is used to create a new DB cluster.

[0025] Furthermore, the system also includes a verification module; the verification module is used to verify the existence and integrity of the new DB cluster data.

[0026] A database rapid migration apparatus includes a memory and a processor. The memory stores a computer program, and the processor invokes the program instructions to execute a database rapid migration method as described above.

[0027] A computer-readable storage medium includes a computer program that can be executed by a processor to implement a database rapid migration method as described above.

[0028] Compared with the prior art, the present invention has the following beneficial effects:

[0029] 1. This invention, based on the PolarDB database, proposes a storage replication migration mode. By replicating the SAN disks partitioned for production use and then mounting the replicated disks to the production instance, rapid database migration is achieved. This invention features a simple process, reducing the burden on users and minimizing the risk of errors caused by complex procedures.

[0030] 2. The storage replication and rapid migration method proposed in this invention can effectively ensure data integrity and consistency while preserving database configurations such as user permissions in the source database. Based on a distributed storage architecture, it can achieve rapid database migration in the event of an anomaly in the PolarStack cluster, thereby quickly enabling database services.

[0031] 3. The database fast migration method proposed in this invention supports local or cross-regional data migration and can also be used in the multi-location, multi-center database storage mode, which has strong practical application value. Attached Figure Description

[0032] Figure 1 This is a flowchart illustrating the implementation of the present invention. Detailed Implementation

[0033] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0034] Example 1

[0035] like Figure 1 The diagram shows a flowchart of a fast database migration method proposed in this invention. Based on the cluster architecture of the PolarDB distributed database, data is stored in different logical volumes, and the method supports storage volume recovery. The method includes the following steps:

[0036] 1) Obtain the SAN (Storage Area Network) disk condition of the production area division where the database to be migrated is located, and record the corresponding storage volume;

[0037] 2) Obtain the storage volume ID, create a snapshot according to the storage volume ID, obtain the snapshot name, and then obtain the snapshot ID;

[0038] 3) Clone the snapshot to obtain a new snapshot, and record the new snapshot name;

[0039] 4) Query the host list through the storage volume ID, and establish a mapping of the new snapshot to the host;

[0040] 5) After creating the mapping, query the WWN (world wide number) according to the storage volume name, and obtain the corresponding WWID (storage disk identifier) based on the new snapshot name;

[0041] 6) The host scans the storage volume for mapping checking, and confirms whether the storage volume is mapped through the WWID;

[0042] 7) Create a new DB cluster according to the WWID.

[0043] 8) Connect the database to verify whether the new DB cluster data exists and is complete.

[0044] In step 3), the mapping of the new snapshot to the host is specifically: establishing a mapping from the obtained storage volume ID to the corresponding host list. When the business continuity requirement is high, the immediate available strategy is selected when cloning the snapshot, so that the data in the cluster is maintained in a readable and writable state.

[0045] In step 4), the mapping of the new snapshot is performed for all hosts in the cluster in the host list.

[0046] In step 7), when creating a new cluster, the data disk is not formatted.

[0047] In specific implementation, the database migration steps are as follows through the storage replication mode:

[0048] S1, log in to the storage device of the production area where the database to be migrated is located, query the corresponding volume ID according to the storage volume name, and record it as lun_id;

[0049] S2, create a snapshot according to the volume lun_id in step S1, obtain the snapshot name, and record it as snapshot_name;

[0050] S3, obtain the snapshot ID according to the snapshot name snapshot_name, and record it as source_id;

[0051] S4, clone snapshot, input snapshot name snapshot_name and snapshot ID, i.e., source_id, obtain new snapshot name snapshot_name2, further, the volume cloning progress can be viewed by the show clone general name=snapshot_name2 command, and the volume cloning efficiency is calculated.

[0052] In particular, different replication strategies can be selected according to business needs when performing volume cloning. For cases with high business continuity requirements, an immediately available strategy is selected during cloning, which can support minute-level data startup and realize fast recovery of continuous operation of business. In this process, the data in the cluster is always readable and writable. The cloning snapshot speed is affected by the data size. Generally, the storage replication speed is 100M / S.

[0053] S5, query the host list by the volume ID, recorded as host_id_list;

[0054] S6, create a mapping of the storage volume to the host, in particular, all hosts in the host list host_id_list in S5 need to be mapped in the cluster;

[0055] S7, find the WWN, query the corresponding WWID according to the new snapshot name snapshot_name2 obtained in step S3;

[0056] S8, the host re-scans the volume for mapping inspection, and confirms whether the volume is mapped by executing the multipath command through the WWID;

[0057] S9, log in to the console and create a new cluster through the WWID, in particular, the data disk is not formatted when creating a new cluster;

[0058] S10, verify the cloned DB cluster data, find the Pod according to the cluster ID in step S9, select the required node found, enter the Pod, connect the database after success, and verify whether the source database, table, user, etc. exist.

[0059] Through the above operation process, the migration of the database can be completed conveniently and quickly. Generally, after verifying that the database migration is completed, the database needs to be analyzed, statistical information is collected, and the migration efficiency is verified, thereby providing an effective reference for database migration in the same scenario.

[0060] The application realizes storage-level database replication cloning, simple operation process, different replication strategies for different business scenarios, rapid database migration, rapid database service activation, and immediate business availability. After database migration, the database is analyzed, statistical information is collected, and effective data reference is obtained for the rapid database migration mode.

[0061] Embodiment 2

[0062] The embodiment provides a database rapid migration system for implementing the database rapid migration method in embodiment 1, comprising a storage volume query module, a snapshot creation module, a snapshot cloning module, a host list query module, a mapping module, a new cluster creation module, and a verification module.

[0063] The storage volume query module is used to obtain the SAN disk condition of the production area division where the database to be migrated is located, and record the corresponding storage volume.

[0064] The snapshot creation module is used to obtain the storage volume ID, create a snapshot according to the storage volume ID, obtain the snapshot name, and then obtain the snapshot ID.

[0065] The snapshot cloning module is used to clone the snapshot, obtain a new snapshot, and record the new snapshot name.

[0066] The host list query module is used to query the host list through the storage volume ID.

[0067] The mapping module is used to establish the mapping of the new snapshot to the host, and confirm whether the storage volume is mapped.

[0068] The new cluster creation module is used to create a new DB cluster.

[0069] The verification module is used to verify whether the data of the new DB cluster exists and is complete.

[0070] The embodiment also provides a database rapid migration device and a computer readable storage medium, wherein the device comprises a memory and a processor, the memory stores a computer program, and the processor can execute the program instructions to implement the database rapid migration method in the above embodiment. In addition, the computer readable storage medium comprises a computer program, and the computer program can be executed by the processor to implement the database rapid migration method in the above embodiment.

[0071] The preferred embodiments of the present application have been described above in detail. It should be understood that modifications and variations to the present application can be affected by those skilled in the art without departing from the scope of the application. Accordingly, it is intended that all possible modifications and alterations be included within the scope of the present application as defined by the following claims.

Claims

1. A method for fast migration of a database, characterized by, The database fast migration method based on a PolarDB database implementation includes the following steps: S1, obtaining the SAN disk condition of a production area in which a database to be migrated is located, and recording the corresponding storage volume; S2, obtaining the storage volume ID, creating a snapshot according to the storage volume ID, obtaining the snapshot name, and then obtaining the snapshot ID; S3, cloning the snapshot to obtain a new snapshot, and recording the new snapshot name; S4, querying the host list through the storage volume ID, and establishing a mapping of the new snapshot to the host; S5, after the mapping is created, querying the WWN according to the storage volume name, and querying the corresponding WWID based on the new snapshot name; S6, the host scans the storage volume for mapping checking, and confirms whether the storage volume is mapped through the WWID; S7, creating a new DB cluster according to the WWID.

2. The method of claim 1, wherein, In step S3, the mapping of the new snapshot to the host is specifically established by establishing a mapping of the obtained storage volume ID to the corresponding host list.

3. The method of claim 1, wherein, In step S3, when the business continuity requirement is high, an immediately available strategy is selected when cloning the snapshot, so that the data in the cluster is maintained in a readable and writable state.

4. The method of claim 1, wherein, In step S4, the mapping of the new snapshot is performed for all hosts in the cluster in the host list.

5. The method of claim 1, wherein, In step S7, the data disk is not formatted when creating a new cluster.

6. The method of claim 1, wherein, Step S8 is also included: connecting the database to verify whether the data of the new DB cluster exists and is complete.

7. A database fast migration system for implementing a database fast migration method according to any one of claims 1 to 6, characterized in that, The storage volume query module, the snapshot creation module, the snapshot cloning module, the host list query module, the mapping module, and the new cluster creation module are included. The storage volume query module is used to obtain the SAN disk condition of a production area in which a database to be migrated is located, and record the corresponding storage volume. The snapshot creation module is used to obtain the storage volume ID, create a snapshot according to the storage volume ID, obtain the snapshot name, and then obtain the snapshot ID. The snapshot cloning module is used to clone the snapshot to obtain a new snapshot, and record the new snapshot name. The host list query module is used to query the host list through the storage volume ID. The mapping module is used to establish a mapping of the new snapshot to the host, and confirm whether the storage volume is mapped. The new cluster creation module is used to create a new DB cluster.

8. The system of claim 7, wherein, The verification module is also included, which is used to verify whether the data of the new DB cluster exists and is complete.

9. A database fast migration apparatus, characterized by comprising: The memory and the processor are included, the memory stores a computer program, and the processor invokes the program instruction to be able to execute the database fast migration method according to any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that, The computer program can be executed by the processor to implement the database fast migration method according to any one of claims 1-6.

Images