Heterogeneous data migration method and related device

Abstract

The invention discloses a heterogeneous data migration method and a related device, and relates to the technical field of data migration, and the method comprises the following steps: firstly, based on obtained data structure information of an original system and a target system, constructing a field corresponding relation between the original system and the target system; extracting the data structure information of each field in the original system, the data structure information of each field in the corresponding target system, the data volume of each field in the original system and the data storage space of each field in the corresponding target system; based on the extracted information, whether the data of the original system can be migrated to the target system or not is judged, factors of data structure difference and data size are considered, migration feasibility is ensured, format conversion is conducted on the data of the original system to meet the data structure requirement of the target system, the data obtained after format conversion is imported into the target system, and the data migration efficiency is improved. And heterogeneous data migration is realized.

Description

Technical Field

[0001] The present application relates to the field of data migration, and in particular to a heterogeneous data migration method and related devices. Background Art

[0002] With the increasing demand for financial product innovation and the rapid development of computer application technology, the demand for data migration is increasing. Data migration mainly refers to extracting, converting and loading data from the original system into the target system to achieve data reuse and versatility.

[0003] Currently, data migration between homogeneous systems is generally performed through the migration tools provided by the database or the migration functions provided by the database management software.

[0004] However, data migration between heterogeneous systems involves not only simple data transfer, but also needs to consider the differences in data structures between the original system and the target system. As a result, the migration tools provided by the database or the migration functions provided by the database management software cannot meet the requirements of data migration between systems with different data structures. Summary of the invention

[0005] In view of the above problems, the present application provides a heterogeneous data migration method and related devices. In order to meet the needs of heterogeneous data migration, the specific solutions are as follows:

[0006] The first aspect of the present application provides a method for migrating heterogeneous data, comprising:

[0007] Acquiring data structure information of an original system and data structure information of a target system, wherein the structure of the data of the original system is different from the structure of the data of the target system;

[0008] Based on the data structure information of the original system and the data structure information of the target system, construct a field correspondence relationship between the original system and the target system;

[0009] Based on the field correspondence between the original system and the target system, extracting data structure information of each field in the original system, data structure information of each field in the target system, data volume of each field in the original system, and data storage space of each field in the target system, wherein each field in the original system has a field correspondence with each field in the target system;

[0010] Based on the data structure information of each field in the original system, the data structure information of each field in the target system, the data volume of each field in the original system, and the data storage space of each field in the target system, determining whether the data of the original system can be migrated to the target system;

[0011] If the data of the original system can be migrated to the target system, the format of the data of the original system is converted, and the data after the format conversion is imported into the target system.

[0012] In a possible implementation, judging whether the data of the original system can be migrated to the target system based on the data structure information of each field in the original system, the data structure information of each field in the target system, the data amount of each field in the original system, and the data storage space of each field in the target system includes:

[0013] Determine whether the data structure information of each field in the original system can be mapped to the data structure information of each field in the target system, and whether the amount of data of each field in the original system is greater than the data storage space of each field in the target system;

[0014] If the data structure information of each field in the original system can be mapped to the data structure information of each field in the target system, and the amount of data of each field in the original system is not greater than the data storage space of each field in the target system, it is determined that the data of the original system can be migrated to the target system.

[0015] In a possible implementation, converting the format of the data of the original system includes:

[0016] Determining a format mapping relationship based on data structure information of each field in the original system and data structure information of each field in the target system;

[0017] Based on the format mapping relationship, the data of the original system is format converted to obtain format-converted data.

[0018] In a possible implementation, before converting the format of the data of the original system, the method further includes:

[0019] If the data of the original system can be migrated to the target system, a stop operation instruction is sent to the original system, and the data of the original system is backed up.

[0020] In a possible implementation, the method further includes:

[0021] Determining whether the data of the target system is operating normally;

[0022] If the data of the target system is not running normally, the stop operation instruction is sent to the target system, and the original system is restored according to the backup data.

[0023] A second aspect of the present application provides a heterogeneous data migration device, including:

[0024] An acquisition unit, used for acquiring data structure information of an original system and data structure information of a target system, wherein the structure of the data of the original system is different from the structure of the data of the target system;

[0025] A construction unit, configured to construct a field correspondence relationship between the original system and the target system based on the data structure information of the original system and the data structure information of the target system;

[0026] an extraction unit, configured to extract data structure information of each field in the original system, data structure information of each field in the target system, data volume of each field in the original system, and data storage space of each field in the target system based on a field correspondence relationship between the original system and the target system, wherein each field in the original system has a field correspondence relationship with each field in the target system;

[0027] a judging unit, configured to judge whether the data of the original system can be migrated to the target system based on the data structure information of each field in the original system, the data structure information of each field in the target system, the data amount of each field in the original system, and the data storage space of each field in the target system;

[0028] The conversion unit is used to convert the format of the data in the original system if the data in the original system can be migrated to the target system, and import the format-converted data into the target system.

[0029] In a possible implementation, the determining unit includes:

[0030] A judging subunit, used to judge whether the data structure information of each field in the original system can be mapped to the data structure information of each field in the target system, and whether the amount of data of each field in the original system is greater than the data storage space of each field in the target system;

[0031] A determination subunit is used to determine that the data of the original system can be migrated to the target system if the data structure information of each field in the original system can be mapped to the data structure information of each field in the target system, and the amount of data of each field in the original system is not greater than the data storage space of each field in the target system.

[0032] A third aspect of the present application provides a computer program product, comprising computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements the heterogeneous data migration method of the first aspect or any implementation of the first aspect.

[0033] A fourth aspect of the present application provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein:

[0034] The memory is used to store computer programs;

[0035] The processor is used to execute the computer program so that the electronic device can implement the heterogeneous data migration method of the above-mentioned first aspect or any implementation manner of the first aspect.

[0036] A fifth aspect of the present application provides a computer storage medium, which carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement the method for migrating heterogeneous data of the above-mentioned first aspect or any implementation of the first aspect.

[0037] By means of the above technical solution, the present application provides a method for migrating heterogeneous data and a related device. The method first constructs a field correspondence relationship between the original system and the target system based on the acquired data structure information. Then, the data structure information of each field in the original system, the corresponding data structure information of each field in the target system, the data volume of each field in the original system, and the corresponding data storage space of each field in the target system are extracted. Based on the above extracted information, it is determined whether the data of the original system can be migrated to the target system, taking into account the factors of data structure differences and data volume to ensure the feasibility of migration, converting the format of the data of the original system to meet the data structure requirements of the target system, and importing the format-converted data into the target system, thereby realizing the migration of heterogeneous data. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0039] Figure 1 A schematic diagram of a process for migrating heterogeneous data provided in an embodiment of the present application;

[0040] Figure 2 A schematic diagram of the structure of a heterogeneous data migration device provided in an embodiment of the present application;

[0041] Figure 3 A schematic diagram of the hardware structure of a heterogeneous data migration device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0042] The following describes the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation method section of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0043] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0044] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0045] In order to improve the migration efficiency and accuracy of heterogeneous data, the present application provides a heterogeneous data migration method. The heterogeneous data migration method provided by the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0046] Please refer to the attached Figure 1 , Figure 1 A schematic diagram of a process flow of a heterogeneous data migration method provided in an embodiment of the present application. The method may include the following steps:

[0047] Step S101: acquiring data structure information of an original system and data structure information of a target system, wherein the data structure of the original system is different from the data structure of the target system.

[0048] In this application, you can access the database of the original system and use database management tools (such as MySQL Workbench, SQL Server Management Studio, etc.) to view the data structure information, which may include all related database tables, field definitions of each table, field relationships, field types, etc. Among them, the target system may introduce new fields, modify field types or lengths, and may delete fields.

[0049] To make it easier to understand, let's take an example: Assume that the original system is an order management system, which has a table called orders, containing the following fields:

[0050] order_id (order ID, integer, primary key);

[0051] customer_name (customer name, character type, not empty);

[0052] order_date (order date, date type, not null);

[0053] total_amount (total amount, float, not null).

[0054] The target system is an upgraded order management system, which has a table called order_details with different fields:

[0055] detail_id (detail ID, integer, primary key);

[0056] customer_full_name (customer full name, character type, not empty, longer);

[0057] order_timestamp (order timestamp, timestamp type, not empty);

[0058] total_price (total price, float, not null, higher precision);

[0059] status (order status, character type, empty allowed).

[0060] Step S102: Based on the data structure information of the original system and the data structure information of the target system, a field correspondence relationship between the original system and the target system is constructed.

[0061] In this application, fields with the same or similar business meanings in the original system and the target system can be identified, and for the matching fields, their correspondence, i.e., field correspondence, is recorded. If the target system has fields that the original system does not have, it is decided whether to fill in default values, use calculated fields, or ignore these fields. If the original system has fields that the target system does not have, it is decided whether to retain the data of these fields (if possible) or discard them.

[0062] Specifically, there are many ways to identify fields with the same or similar business meanings in the original system and the target system. Models based on text matching include string similarity algorithms and word embedding models. Among them, string similarity algorithms can use string similarity algorithms such as cosine similarity, Jaccard similarity, and Levenshtein distance to compare the similarity of field names and definitions. When the similarity exceeds a certain threshold, it is considered that the fields may have the same business meaning. Word embedding models can use word embedding models such as Word2Vec and BERT to convert field names and descriptions into high-dimensional vectors. By calculating cosine similarity or other distance metrics between vectors, fields with similar business meanings can be identified.

[0063] Classification models based on machine learning can first extract features from information such as field names, descriptions, data types, and lengths. These features can include lexical features (such as words in field names), semantic features (such as vector representations obtained using word embedding models), statistical features (such as the distribution of field values), etc. Then use a dataset with known field correspondences as training data. Select a suitable machine learning algorithm (such as logistic regression, support vector machine, decision tree, random forest, neural network, etc.) for training. The goal is to classify field pairs as "same business meaning" or "different business meaning". Then use cross-validation, confusion matrix, accuracy, recall, and other indicators to evaluate model performance. Adjust feature selection, model parameters, etc. based on the evaluation results to improve model performance. Finally, apply the trained model to fields in the original system and the target system to identify fields with the same or similar business meanings.

[0064] Semantic matching models based on deep learning include Siamese networks and pre-trained language models such as BERT. The Siamese network consists of two neural networks with shared parameters, which are used to process the input of field names and descriptions. The network outputs the similarity score between the two fields. By training the Siamese network, it can accurately determine whether the two fields have the same business meaning. Pre-trained language models such as BERT are fine-tuned using BERT or other pre-trained language models to adapt to the task of field semantic matching. The field names and descriptions are input, their semantic representations are obtained through the model, and the similarity between the semantic representations is calculated to identify fields with the same or similar business meanings.

[0065] For ease of understanding, an example is given below: Take the orders table of the original system and the order_details table of the target system as an example. The order_id corresponds to the detail_id field; the customer_name corresponds to the customer_full_name field; the order_date corresponds to the order_timestamp field; and the total_amount corresponds to the total_price field. The status field does not exist in the original system and can be filled with a default value (such as "pending") for the target system after migration. Step S103: Based on the field correspondence between the original system and the target system, extract the data structure information of each field in the original system, the data structure information of each field in the target system, the data volume of each field in the original system, and the data storage space of each field in the target system. Each field in the original system has a field correspondence with each field in the target system.

[0066] In this application, based on the field correspondence between the original system and the target system, the data structure information of each field in the original system can be extracted: the field name, data type (such as integer, floating point number, string, date, etc.), length (for string type) or precision (for numeric type), etc. The data structure information of each field in the target system: similarly, the data type, length or precision of the corresponding field in the target system. This information helps to understand the organization and storage methods of data in the original system and the target system.

[0067] The data volume of each field in the original system refers to the number or size of data stored in each field. The data storage space of each field in the target system refers to the size of the storage space allocated for each field in the target system. This information helps to understand and evaluate whether there is corresponding storage space in the target system to accommodate the data of the original system.

[0068] For ease of understanding, an example is given below: Take the orders table of the original system and the order_details table of the target system as examples.

[0069] Original system (orders table)

[0070] Field data structure information: order_id: Field name: Order ID; Data type: Integer; Is it primary key: Yes; Remarks: None; customer_name: Field name: Customer name; Data type: Character type Whether it can be empty: No; Remarks: None (assuming that the length is not specified, it is determined according to the system default or actual usage); order_date: Field name: Order date; Data type: Date type; Whether it can be empty: No; Remarks: None; total_amount: Field name: Total amount; Data type: Floating point type; Whether it can be empty: No; Remarks: None (assuming that the precision is not specified, it is determined according to the system default or actual usage).

[0071] Field data volume: Assume that there are N records in the orders table, where N is a specific number representing the total number of orders. For each field, its data volume can be considered as N (because each record contains these fields).

[0072] Target system (order_details table)

[0073] Field data structure information: detail_id: Field name: Detail ID; Data type: Integer; Is it the primary key: Yes; Remarks: Usually automatically generated by the system to uniquely identify each record; customer_full_name: Field name: Customer full name; Data type: Character; Is it allowed to be empty: No; Remarks: The length is longer than the customer_name of the original system to accommodate a more complete customer name; order_timestamp: Field name: Order timestamp; Data type: Timestamp; Is it allowed to be empty: No; Remarks: Contains more information (such as time) than the order_date of the original system; total_price: Field name: Total price; Data type: Floating point; Is it allowed to be empty: No; Remarks: The precision is higher than the total_amount of the original system to support more accurate price representation; status: Field name: Order status; Data type: Character; Is it allowed to be empty: Yes; Remarks: It does not exist in the original system and can be filled with default values ​​after migration.

[0074] Field data storage space: detail_id: Since it is an integer and usually automatically increments, its storage space is relatively small and is the same for each record; customer_full_name: The storage space is determined based on the character length and encoding method. Due to its longer length, more storage space may be required; order_timestamp: The timestamp type usually occupies a fixed amount of storage space; total_price: The storage space of the floating-point type is determined based on the precision and representation method. Due to its higher precision, more storage space may be required; status: The storage space of the character field depends on the character length and encoding method, but since it is allowed to be empty and may have a default value, its actual storage space may be small; Step S104: Based on the data structure information of each field in the original system, the data structure information of each field in the target system, the data volume of each field in the original system, and the data storage space of each field in the target system, determine whether the data of the original system can be migrated to the target system.

[0075] In the present application, it is determined whether the data structure information of each field in the original system can be mapped to the data structure information of each field in the target system, and whether the data volume of each field in the original system is greater than the data storage space of each field in the target system. If the data structure information of each field in the original system can be mapped to the data structure information of each field in the target system, and the data volume of each field in the original system is not greater than the data storage space of each field in the target system, it is determined that the data of the original system can be migrated to the target system.

[0076] Specifically, the mapping of data structure information can check whether the data type of each field in the original system can find a corresponding or compatible data type in the target system. Verify whether the constraints of the field (such as non-null, uniqueness, etc.) can be maintained in the target system. The evaluation of data volume can compare whether the data volume matches the data storage space of the corresponding field in the target system to ensure that the data migration does not exceed the storage capacity of the target system.

[0077] For ease of understanding, an example is given below: Take the orders table of the original system and the order_details table of the target system as examples. Mapping of data structure information: order_id (integer, primary key) corresponds to detail_id (integer, primary key) in the target system. The data types and primary key constraints are consistent and can be mapped. customer_name (character, non-null) corresponds to customer_full_name (character, non-null, longer length) in the target system. Although the names and lengths are different, the data types are the same, and the target system has a longer length that can accommodate the data of the original system, so they can be mapped. order_date (date type, non-null) corresponds to order_timestamp (timestamp type, non-null) in the target system. It should be noted here that the data types are different, but the timestamp usually contains date information, so mapping can be achieved through conversion. total_amount (floating point type, non-null) corresponds to total_price (floating point type, non-null, higher precision) in the target system. The data types are the same, and the precision of the target system is higher, and the data of the original system can be accommodated, so they can be mapped.

[0078] Data volume assessment: Assume that there are 1 million records in the orders table of the original system, and the field data volume in each record does not exceed the data storage space limit of the corresponding field in the target system (for example, the length of customer_name does not exceed the length of customer_full_name, and the numerical range of total_amount is within the precision range of total_price).

[0079] Through evaluation, confirm that the data volume of the original system will not exceed the storage capacity of the target system.

[0080] If the data of the original system can be migrated to the target system, step S105 is executed.

[0081] Step S105: convert the format of the data in the original system, and import the converted data into the target system.

[0082] Before converting the format of the data in the original system, the method further includes: if the data in the original system can be migrated to the target system, sending a stop operation instruction to the original system and backing up the data in the original system.

[0083] Before data format conversion, in order to ensure data consistency and integrity, a stop command is usually sent to the original system to avoid data conflicts or losses during data migration. At the same time, a comprehensive backup of the data in the original system is performed to ensure that it can be quickly restored to the original state if problems occur during or after data migration.

[0084] Before converting the format of the data in the original system, in order to ensure the quality and accuracy of the data, the old data system can also be cleaned. In the original system, there may be various problems, such as the narrative information not matching the data situation, the same or similar data being repeated, etc. These problems need to be solved through a series of operations, including but not limited to finding and deleting dirty data, invalid data, etc. Specifically, database statements (such as SQL) can be used to find these dirty data. For example, the SELECT statement can be used in combination with conditional judgment to filter out data records that do not meet the requirements, and then the DELETE statement can be used to delete them. In addition, the database indexes and constraints can be used to prevent the generation of dirty data, such as setting unique constraints to avoid the insertion of duplicate data.

[0085] Data validation is another operation to ensure data quality and accuracy. It mainly includes six types of validation: data length validation, data format validation, null value and default value validation, interval range validation, consistency validation, and integrity validation. These validation types cover all aspects of data validation and can ensure data accuracy, consistency, and integrity. Among them, data length validation: check whether the data meets specific length requirements. Data format validation: check whether the data meets specific format requirements, such as date format, phone number format, etc. Null value and default value validation: check whether the data is empty or whether the default value is used, and ensure that these null values ​​and default values ​​are logically reasonable. Interval range validation: check whether the data is within a specific range, such as whether the age is between 0-100. Consistency validation: check whether the data is logically consistent, such as whether the order amount in the order table is consistent with the payment amount in the payment table. Integrity validation: check whether the data is complete, such as whether necessary fields or records are missing. At the same time, you can monitor whether the environment is running smoothly. For example, you can use system monitoring tools (such as top, vmstat, etc.) to check the CPU and memory usage of the target system to ensure that there are enough resources to support the migration process. If you find that the CPU or memory usage is too high, you need to optimize the migration plan or increase hardware resources. Use disk monitoring tools (such as iostat, df, etc.) to check the disk's write speed and remaining space. If you find that the write speed is too slow or the remaining space is insufficient, you need to adjust the migration strategy or increase the disk capacity.

[0086] In the present application, a format mapping relationship is determined based on the data structure information of each field in the original system and the data structure information of each field in the target system. Based on the format mapping relationship, the data of the original system is format-converted to obtain format-converted data.

[0087] Specifically, based on the data structure information of each field in the original system and the data structure of each field in the target system, in addition to establishing the correspondence between the fields, the mapping relationship between the data format of each field in the original system and the data format of the corresponding field in the target system can also be determined. This is because even if two fields have the same business meaning, their data formats (such as data type, length, precision, date format, etc.) may be different. According to the determined format mapping relationship, the data in the original system is format converted. This may include operations such as data type conversion (such as converting date type to timestamp type), data length adjustment (such as truncation or padding characters), and data precision adjustment (such as rounding or truncating decimal places). During the conversion process, it is necessary to ensure the integrity and accuracy of the data to avoid data loss or deformation.

[0088] For ease of understanding, an example is given below: Take the orders table of the original system and the order_details table of the target system as examples.

[0089] Original field: customer_name (character type, maximum length 50), target field: customer_full_name (character type, maximum length 100), although customer_name and customer_full_name represent the same meaning in business (namely, the customer's name), their data formats are different. Both are character type, so the data types are compatible and no conversion is required. The maximum length of the original field is 50, while the maximum length of the target field is 100. The target field customer_full_name has a longer length and can accommodate longer customer names, so the character length can be extended from the original system limit to the target system limit. Based on the mapping relationship between order_id and detail_id, you can create a mapping table or directly use order_id as detail_id; based on the mapping relationship between order_date and order_timestamp, you can convert the date data of the order_date field into timestamp data; based on the mapping relationship between total_amount and total_price, you can adjust the floating point data of the total_amount field to the precision required by the target system.

[0090] Specifically, background functions (such as spool command and exp command) can be used to import format-converted data into the target system. This usually involves operations such as data loading, checking and verification. During the import process, it is necessary to monitor the loading progress and status of the data to ensure that the data can be correctly and completely imported into the target system. At the same time, it is also necessary to check and verify the imported data to ensure the accuracy and consistency of the data. For example, check whether detail_id is unique, whether customer_full_name is complete, whether order_timestamp is correctly converted, etc. Run the same business scenarios and test cases as the original system in the target system, and compare whether the results are consistent. After the data migration and conversion are completed, the creat command can also be used to build tables and add indexes and business constraints, and the imp command can be used to put the data back into the newly created database. In summary, a method for migrating heterogeneous data provided by an embodiment of the present application first constructs a field correspondence relationship between the original system and the target system based on the data structure information obtained. Then, the data structure information of each field in the original system, the data structure information of each field in the corresponding target system, the data volume of each field in the original system, and the data storage space of each field in the corresponding target system are extracted therefrom. Based on the information extracted above, we determine whether the data in the original system can be migrated to the target system. We consider the differences in data structure and the amount of data to ensure the feasibility of migration. We convert the data in the original system into a format that meets the data structure requirements of the target system and import the converted data into the target system, thus realizing the migration of heterogeneous data.

[0091] Further, based on the above embodiment, the method may further include the following steps: determining whether the data of the target system is running normally. If the data of the target system is not running normally, a stop operation instruction is sent to the target system, and the original system is restored according to the backup data.

[0092] It should be noted that the data in the target system may not operate normally due to a variety of reasons, such as errors in the data migration process, configuration problems in the new system environment, adjustments to business logic, etc.

[0093] In this application, when it is found that the data of the target system is not running normally, a stop operation instruction needs to be sent to the target system immediately to prevent the problem from further expanding or causing greater losses. At the same time, the original system needs to be restored according to the backup data to ensure business continuity and data integrity. During the recovery process, special attention should be paid to the integrity and consistency of the data, and it should be verified whether the restored data can normally participate in the processing and display of business logic. If there are still problems with the restored data, it is necessary to further investigate the cause and fix the problem.

[0094] If a failure occurs during the data migration process, stop the data migration immediately, find out the cause of the failure, and try to resume the migration.

[0095] In summary, by implementing the described data migration monitoring and recovery process, this application can quickly curb the spread of problems, restore normal operation in the shortest possible time, significantly reduce the risks during and after the data migration process, and improve business continuity and security.

[0096] A heterogeneous data migration method provided in an embodiment of the present application is introduced above, and a device for executing the heterogeneous data migration method will be introduced below.

[0097] See also Figure 2 , Figure 2 A schematic diagram of a heterogeneous data migration device provided in an embodiment of the present application. Figure 2 As shown, the heterogeneous data migration device includes:

[0098] The acquisition unit 11 is used to acquire data structure information of an original system and data structure information of a target system, wherein the data structure of the original system is different from the data structure of the target system.

[0099] The construction unit 12 is used to construct a field correspondence relationship between the original system and the target system based on the data structure information of the original system and the data structure information of the target system.

[0100] The extraction unit 13 is used to extract data structure information of each field in the original system, data structure information of each field in the target system, the data volume of each field in the original system and the data storage space of each field in the target system based on the field correspondence between the original system and the target system, and each field in the original system has a field correspondence with each field in the target system.

[0101] The judgment unit 14 is used to judge whether the data of the original system can be migrated to the target system based on the data structure information of each field in the original system, the data structure information of each field in the target system, the data volume of each field in the original system and the data storage space of each field in the target system.

[0102] The conversion unit 15 is used for converting the data of the original system into a format if the data of the original system can be migrated to the target system, and importing the format-converted data into the target system.

[0103] In a possible implementation, the determining unit 14 includes:

[0104] The judgment subunit is used to judge whether the data structure information of each field in the original system can be mapped to the data structure information of each field in the target system, and whether the data volume of each field in the original system is greater than the data storage space of each field in the target system.

[0105] A determination subunit is used to determine that the data of the original system can be migrated to the target system if the data structure information of each field in the original system can be mapped to the data structure information of each field in the target system, and the amount of data of each field in the original system is not greater than the data storage space of each field in the target system.

[0106] In a possible implementation, the conversion unit 15 includes:

[0107] The relationship determination subunit is used to determine the format mapping relationship based on the data structure information of each field in the original system and the data structure information of each field in the target system.

[0108] The conversion subunit is used to perform format conversion on the data of the original system based on the format mapping relationship to obtain data after format conversion.

[0109] In a possible implementation, the device further includes:

[0110] The backup unit is used to send a stop operation instruction to the original system if the data of the original system can be migrated to the target system, and to back up the data of the original system.

[0111] In a possible implementation, the device further includes:

[0112] The operation determination unit is used to determine whether the data of the target system is running normally.

[0113] The recovery unit is used to send the stop operation instruction to the target system if the data of the target system does not operate normally, and recover the original system according to the backup data.

[0114] The present application also provides an electronic device in an embodiment. Figure 3 As shown, it shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiment of the present application. The electronic device in the embodiment of the present application may include but is not limited to fixed terminals such as mobile phones, laptops, PDAs (personal digital assistants), PADs (tablet computers), desktop computers, etc. Figure 3 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0115] like Figure 3As shown, the electronic device may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 to a random access memory (RAM) 303. When the electronic device is powered on, various programs and data required for the operation of the electronic device are also stored in the RAM 303. The processing device 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0116] Typically, the following devices may be connected to the I / O interface 305: an input device 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 308 including, for example, a memory card, a hard disk, etc.; and a communication device 309. The communication device 309 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although Figure 3 An electronic device having various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0117] An embodiment of the present application also provides a computer program product including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements any heterogeneous data migration method provided in the embodiment of the present application.

[0118] A computer-readable storage medium is also provided in an embodiment of the present application. The storage medium carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any heterogeneous data migration method provided in an embodiment of the present application.

[0119] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0120] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0121] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0122] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

Claims

1. A method for migrating heterogeneous data, characterized in that: include: Acquiring data structure information of an original system and data structure information of a target system, wherein the structure of the data of the original system is different from the structure of the data of the target system; Based on the data structure information of the original system and the data structure information of the target system, construct a field correspondence relationship between the original system and the target system; Based on the field correspondence between the original system and the target system, extracting data structure information of each field in the original system, data structure information of each field in the target system, data volume of each field in the original system, and data storage space of each field in the target system, wherein each field in the original system has a field correspondence with each field in the target system; Based on the data structure information of each field in the original system, the data structure information of each field in the target system, the data volume of each field in the original system, and the data storage space of each field in the target system, determining whether the data of the original system can be migrated to the target system; If the data of the original system can be migrated to the target system, the format of the data of the original system is converted, and the data after the format conversion is imported into the target system.

2. The heterogeneous data migration method according to claim 1, characterized in that: The determining whether the data of the original system can be migrated to the target system based on the data structure information of each field in the original system, the data structure information of each field in the target system, the data volume of each field in the original system, and the data storage space of each field in the target system includes: Determine whether the data structure information of each field in the original system can be mapped to the data structure information of each field in the target system, and whether the amount of data of each field in the original system is greater than the data storage space of each field in the target system; If the data structure information of each field in the original system can be mapped to the data structure information of each field in the target system, and the amount of data of each field in the original system is not greater than the data storage space of each field in the target system, it is determined that the data of the original system can be migrated to the target system.

3. The heterogeneous data migration method according to claim 1, characterized in that: The format conversion of the data of the original system includes: Determining a format mapping relationship based on data structure information of each field in the original system and data structure information of each field in the target system; Based on the format mapping relationship, the data of the original system is format converted to obtain format-converted data.

4. The heterogeneous data migration method according to claim 1, characterized in that: Before converting the format of the data of the original system, the method further includes: If the data of the original system can be migrated to the target system, a stop operation instruction is sent to the original system, and the data of the original system is backed up.

5. The heterogeneous data migration method according to claim 4, characterized in that: The method further comprises: Determining whether the data of the target system is operating normally; If the data of the target system is not running normally, the stop operation instruction is sent to the target system, and the original system is restored according to the backup data.

6. A heterogeneous data migration device, characterized in that: include: An acquisition unit, used for acquiring data structure information of an original system and data structure information of a target system, wherein the structure of the data of the original system is different from the structure of the data of the target system; A construction unit, configured to construct a field correspondence relationship between the original system and the target system based on the data structure information of the original system and the data structure information of the target system; an extraction unit, configured to extract data structure information of each field in the original system, data structure information of each field in the target system, data volume of each field in the original system, and data storage space of each field in the target system based on a field correspondence relationship between the original system and the target system, wherein each field in the original system has a field correspondence relationship with each field in the target system; a judging unit, configured to judge whether the data of the original system can be migrated to the target system based on the data structure information of each field in the original system, the data structure information of each field in the target system, the data amount of each field in the original system, and the data storage space of each field in the target system; The conversion unit is used to convert the format of the data in the original system if the data in the original system can be migrated to the target system, and import the format-converted data into the target system.

7. The heterogeneous data migration device according to claim 6, characterized in that: The judging unit comprises: A judging subunit, used to judge whether the data structure information of each field in the original system can be mapped to the data structure information of each field in the target system, and whether the amount of data of each field in the original system is greater than the data storage space of each field in the target system; A determination subunit is used to determine that the data of the original system can be migrated to the target system if the data structure information of each field in the original system can be mapped to the data structure information of each field in the target system, and the amount of data of each field in the original system is not greater than the data storage space of each field in the target system.

8. A computer program product, characterized in that It comprises computer-readable instructions, and when the computer-readable instructions are executed on an electronic device, the electronic device implements the heterogeneous data migration method as claimed in any one of claims 1 to 5.

9. An electronic device, characterized in that: The method comprises at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program so that the electronic device can implement the heterogeneous data migration method as described in any one of claims 1 to 5.

10. A computer storage medium, characterized in that: The storage medium carries one or more computer programs, and when the one or more computer programs are executed by an electronic device, the electronic device can implement the heterogeneous data migration method as described in any one of claims 1 to 5.

Images