Data migration method and system

Abstract

The invention provides a data migration method and system.The method comprises the steps that a migration task is analyzed to obtain plan information in the migration task, the plan information comprises rule instances, source environment information and target environment information, and the rule instances comprise conversion rules and extraction rules corresponding to different model types; selecting a corresponding extraction adapter according to the source environment information, and selecting a corresponding conversion adapter according to the target environment information; extracting corresponding to-be-migrated data from a source database corresponding to the source environment information through an extraction adapter according to an extraction rule; and converting the to-be-migrated data into an intermediate model file through the conversion adapter according to the conversion rule, and migrating the intermediate model file to a target database corresponding to the target environment information, and loading and using the intermediate model file. The method supports automatic backup synchronization, and has a perfect exception compensation mechanism.

Claims

1. A data migration method, characterized in that: include: Parsing the migration task to obtain the plan information therein, the plan information includes rule instances, source environment information, and target environment information, the rule instances include conversion rules and extraction rules corresponding to different model types; Selecting a corresponding extraction adapter according to the source environment information, and selecting a corresponding conversion adapter according to the target environment information; Extracting corresponding data to be migrated from a source database corresponding to the source environment information according to the extraction rule by the extraction adapter; The data to be migrated is converted into an intermediate model file according to the conversion rule by the conversion adapter, and the intermediate model file is migrated to the target database corresponding to the target environment information and loaded for use.

2. The data migration method according to claim 1, characterized in that: Before obtaining the rule instance in the parsing migration task, the following steps are included: Creating a migration plan based on a set of the rule instances; the migration plan determines source environment information and target environment information, and sets corresponding actual value ranges for all parameters associated with the rule instances; After receiving the task request, the corresponding migration task is generated according to the migration plan.

3. The data migration method according to claim 1, characterized in that: The extracting corresponding data to be migrated from the source database corresponding to the source environment information according to the extraction rule by the extraction adapter comprises: inputting the extraction rules and variable information into the extraction adapter, and connecting the extraction adapter to the source database; According to the extraction rules and variable information, the specified data in the source database is converted into an intermediate model file through the extraction adapter to obtain the data to be migrated.

4. The data migration method according to claim 3, characterized in that: The extraction rule includes a structural model extraction rule, the extraction adapter includes a relational database adapter, the type of the source database is a structural model database, and according to the extraction rule and variable information, converting the specified data in the source database into an intermediate model file through the extraction adapter to obtain the data to be migrated includes: Controlling the relational database adapter to extract data table structure information corresponding to the specified data from the structure model database according to the structure model extraction rule, wherein the data table structure information includes basic information, attribute information, and index information of the data table structure; The relational database adapter is controlled to convert the basic information, the attribute information and the index information corresponding to the data table structure into the intermediate model file.

5. The data migration method according to claim 3, characterized in that: The extraction rule includes a data model extraction rule, the extraction adapter includes a relational database adapter, the type of the source database is a data model database, and according to the extraction rule and variable information, converting the specified data in the source database into an intermediate model file through the extraction adapter to obtain the data to be migrated includes: Controlling the relational database adapter to generate a corresponding structured query statement according to the data model extraction rule and the variable information; Controlling the relational database adapter to obtain the specified data corresponding to the structured query statement from the data model database; The relational database adapter is controlled to convert the specified data into the intermediate model file.

6. The data migration method according to claim 3, characterized in that: The extraction rule includes a file model extraction rule, the extraction adapter includes a file model adapter, the type of the source database is cloud storage and file database, and according to the extraction rule and variable information, converting the specified data in the source database into an intermediate model file through the extraction adapter to obtain the data to be migrated includes: Controlling the file model adapter to obtain a file list corresponding to the specified data from the cloud storage or the file database according to the variable information and the file model extraction rule; The file model adapter is controlled to download all target files according to the file list, and convert all the target files into the intermediate model files.

7. The data migration method according to claim 6, characterized in that: The controlling the file model adapter to obtain the file list corresponding to the specified data from the cloud storage or the file database according to the variable information and the file model extraction rule comprises: Assembling and generating a file directory address according to the variable information and the storage information corresponding to the source database, and calling an application interface provided by a third party to which the source database belongs to obtain a file list corresponding to the file directory address; The step of controlling the file model adapter to download all target files according to the file list and converting all the target files into the intermediate model files comprises: The application interface is called to cyclically download all the target files to the local computer, the path information corresponding to the local file directory and all the target files is recorded, and the intermediate model file is generated in a summarized manner.

8. The data migration method according to claim 1, characterized in that: Also includes: The conversion adapter extracts the corresponding data to be backed up from the target database corresponding to the target environment information according to the extraction rule, converts the data to be backed up into an intermediate model file according to the conversion rule, and backs up and stores it in a local file directory.

9. The data migration method according to claim 1, characterized in that: The conversion rule includes a structural model conversion rule, the conversion adapter includes a relational database adapter, and the conversion of the to-be-migrated data into an intermediate model file according to the conversion rule by the conversion adapter, and migrating the intermediate model file to the target database includes: Controlling the relational database adapter to extract the target environment information and the intermediate model file corresponding to the structural model conversion rule; Control the relational database adapter to parse the relational database information in the target environment information to obtain a target database type; the target database type is a structural model database; The relational database adapter is controlled to generate a first structured query script adapted to the structural model database as the intermediate model file according to the intermediate model file and the target database type and in accordance with the structural model conversion rule.

10. The data migration method according to claim 1, characterized in that: The conversion rule includes a data model conversion rule, the conversion adapter includes a relational database adapter, and the conversion of the to-be-migrated data into an intermediate model file according to the conversion rule by the conversion adapter, and migrating the intermediate model file to the target database includes: Controlling the relational database adapter to extract the target environment information and the intermediate model file corresponding to the data model conversion rule; Controlling the relational database adapter to obtain the intermediate model backup file corresponding to the data model conversion rule; Control the relational database adapter to parse the relational database information in the target environment information to obtain a target database type; the target database type is a data model database; The relational database adapter is controlled to generate a second structured query script adapted to the data model database as the intermediate model file according to the intermediate model file, the intermediate model backup file and the target database type and in accordance with the data model conversion rule.

11. The data migration method according to claim 9, characterized in that: The step of migrating the intermediate model file to the target database corresponding to the target environment information and loading and using the target database includes: The first structured query script as the intermediate model file is stored in a structured model database, and the first structured query script is executed in the structured model database.

12. The data migration method according to claim 10, characterized in that: The step of migrating the intermediate model file to the target database corresponding to the target environment information and loading and using the target database includes: The second structured query script as the intermediate model file is stored in a data model database, and the second structured query script is executed in the data model database.

13. A data migration system, characterized in that: The data migration system comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the data migration method according to any one of claims 1 to 12.

Images