Heterogeneous database data migration automatic correction method and device

Abstract

The present invention provides a method and device for automatic correction of data migration in heterogeneous databases, the method comprising: acquiring source data of a source database and target data of a target database from which the source data is migrated; performing data comparison between the source data and the corresponding target data to determine whether there is an abnormality in the target data obtained by migrating the source data to the target database; if there is an abnormality, correcting the target data. The present invention can automatically acquire and correct data in the migration process, improve the accuracy of data migration, and effectively reduce the program development cost during data migration correction.

Description

Technical Field

[0001] The present invention relates to the field of database technology, and in particular to a method and device for automatic correction of heterogeneous database data migration. Background Art

[0002] Currently, when migrating data between heterogeneous databases, for example, migrating data from Oracle product applications to relational databases like MySQL, after migrating the data from the source database to the target database, the data needs to be corrected to determine if the heterogeneous database migration was successful. This data also typically needs to be converted to meet the storage requirements of the target database or the user. Conventional technology requires developing a program to perform data correction and conversion for each data migration. Summary of the Invention

[0003] One object of the present invention is to provide a method for automatically correcting data migration in heterogeneous databases, which automatically acquires and corrects data during the migration process, improving the accuracy of data migration and effectively reducing program development costs for data migration correction. Another object of the present invention is to provide an apparatus for automatically correcting data migration in heterogeneous databases. Another object of the present invention is to provide a computer device. Yet another object of the present invention is to provide a readable medium.

[0004] In order to achieve the above objectives, the present invention discloses, on one hand, a method for automatically correcting data migration in a heterogeneous database, comprising:

[0005] Acquire source data from a source database and migrate target data from the source data to a target database;

[0006] Comparing the source data with the corresponding target data to determine whether there are any anomalies in the target data obtained by migrating the source data to the target database;

[0007] If there is an abnormality, the target data is corrected.

[0008] Preferably, the step of obtaining source data from a source database specifically includes:

[0009] Sending a source data acquisition request to the source database according to a preset source data range and table structure information of the source database;

[0010] The source data returned by the source database according to the source data acquisition request is received.

[0011] Preferably, obtaining target data from the source data to be migrated to the target database specifically includes:

[0012] Sending a target data acquisition request to the target database according to the target data range corresponding to the preset source data range and the table structure information of the target database;

[0013] Receive the target data returned by the target database according to the target data acquisition request.

[0014] Preferably, the comparing the source data with the corresponding target data to determine whether the target data obtained by migrating the source data to the target database has any anomalies specifically includes:

[0015] The source data and the target data are compared text by text to determine whether the target data has any anomalies such as missing primary keys, missing indexes, and non-empty data.

[0016] Preferably, further comprising:

[0017] If an exception exists, at least one of the operations of establishing a primary key, establishing an index, and setting a field non-null attribute is performed on the target data according to the exception detection result of the target data.

[0018] Preferably, further comprising:

[0019] If there is no abnormality, the target data is converted according to the preset conversion mapping relationship.

[0020] Preferably, the converting the target data content according to the preset conversion mapping relationship specifically includes:

[0021] Determine, from a preset conversion mapping relationship table, a preset conversion mapping relationship of the target data of the field from the source database to the target database according to the field of the target data, the source database, and the target database;

[0022] Perform data content conversion on the target data of the field according to the conversion mapping relationship.

[0023] The present invention also discloses a device for automatically correcting heterogeneous database data migration, comprising:

[0024] A data acquisition module is used to acquire source data from a source database and migrate target data from the source data to a target database;

[0025] A data verification module is used to compare the source data with the corresponding target data to determine whether there are any anomalies in the target data obtained by migrating the source data to the target database;

[0026] The data correction module is used to correct the target data if an anomaly exists.

[0027] The present invention also discloses a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor.

[0028] When the processor executes the program, the method described above is implemented.

[0029] The present invention also discloses a computer readable medium having a computer program stored thereon.

[0030] When the program is executed by a processor, the above-mentioned method is implemented.

[0031] The automatic correction method for heterogeneous database data migration of the present invention automatically obtains source data of a source database and target data stored in a target database after the source data is migrated to the target database, performs data comparison on the source data and the target data to determine whether the target data stored in the target database formed after the source data is migrated to the target database has any anomalies, and corrects the target data with anomalies, thereby realizing automatic correction of data migration between heterogeneous databases. Therefore, by adopting the automatic correction method for heterogeneous database data migration of the present invention, developers can realize automatic verification and correction of the data migration process through simple information configuration, ensure the accuracy of data migration between heterogeneous databases, unify the processing logic of homogeneity and universality in the data migration process, and avoid the problem of waste of R&D resources and low data migration efficiency caused by the need to develop data verification and correction programs for each data migration. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0033] Figure 1 A flowchart showing a specific embodiment of the method for automatic correction of heterogeneous database data migration according to the present invention;

[0034] Figure 2 A flowchart illustrating a specific embodiment of the method for automatically correcting heterogeneous database data migration according to the present invention for obtaining source data from a source database is shown;

[0035] Figure 3 A flowchart illustrating a specific embodiment of the method for automatically correcting heterogeneous database data migration according to the present invention for obtaining target data migrated from the source data to the target database;

[0036] Figure 4A flowchart showing a specific embodiment S200 of the method for automatic correction of heterogeneous database data migration according to the present invention;

[0037] Figure 5 A flowchart showing a specific embodiment S300 of the method for automatic correction of heterogeneous database data migration according to the present invention is shown;

[0038] Figure 6 A flowchart showing a specific embodiment of the method for automatic correction of heterogeneous database data migration of the present invention, including data content conversion;

[0039] Figure 7 A flowchart showing a specific embodiment S400 of the method for automatic correction of heterogeneous database data migration according to the present invention is shown;

[0040] Figure 8 A flowchart showing a specific embodiment of the device for automatically correcting heterogeneous database data migration according to the present invention;

[0041] Figure 9 A flowchart showing a specific embodiment of the device for automatic correction of heterogeneous database data migration according to the present invention includes a data conversion module;

[0042] Figure 10 A schematic diagram showing the structure of a computer device suitable for implementing an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0043] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0044] It should be noted that the method and device for automatic correction of heterogeneous database data migration disclosed in the present application can be used in the field of artificial intelligence technology, and can also be used in any field other than the field of artificial intelligence technology. The application field of the method and device for automatic correction of heterogeneous database data migration disclosed in the present application is not limited.

[0045] In order to facilitate the understanding of the technical solution provided by the present application, the relevant contents of the technical solution of the present application are first described below. The automatic correction method for heterogeneous database data migration provided by the embodiment of the present invention is aimed at the problem that a set of programs need to be developed for each data migration process to complete the correction and conversion of data during the existing data migration between heterogeneous databases. The method automatically obtains the source data of the source database and the target data stored in the target database after the source data is migrated to the target database, compares the source data and the target data to determine whether there are any abnormalities in the target data stored in the target database after the source data is migrated to the target database, and corrects the target data with abnormalities, thereby realizing the automatic correction of data migration between heterogeneous databases. Therefore, by adopting the automatic correction method for heterogeneous database data migration of the present invention, developers can realize the automatic verification and correction of the data migration process through simple information configuration, ensure the accuracy of data migration between heterogeneous databases, and unify the processing logic of homogeneity and universality in the data migration process, thereby avoiding the problem of waste of R&D resources and low data migration efficiency caused by the need to develop data verification and correction programs for each data migration.

[0046] An embodiment of the present invention provides a heterogeneous database data migration automatic correction system, which may include a source database, a target database, and a heterogeneous database data migration automatic correction device.

[0047] The source data stored in the source database can be migrated to the target database using the database's own data migration tool or a self-developed data migration tool, resulting in target data corresponding to the source data stored in the target database. For example, the source data can be exported from the source database using the source database's own data pump, which can then be imported into the target database using the target database's dump tool to obtain the target data. Migrating source data from the source database to the target database means extracting the source data from the source database and storing it in corresponding locations in the target database, thereby obtaining the target data in the target database. On the one hand, data migration errors may occur during the data migration process, resulting in missing data after the data is migrated from the source database to the target database. On the other hand, due to the different database structures of the source and target databases, the target data migrated to the target database may contain errors due to these differences. Therefore, the target data migrated to the target database needs to be verified and corrected to ensure the accuracy of the target data obtained after the source data from the source database is migrated to the target database.

[0048] Based on this, the heterogeneous database data migration automatic correction device of the heterogeneous database data migration automatic correction system provided in an embodiment of the present invention is used to obtain source data of a source database and target data of a target database after the source data is migrated, and compare the source data with the corresponding target data to determine whether there is any abnormality in the target data obtained by migrating the source data to the target database. If there is any abnormality, the target data is corrected.

[0049] The automatic correction device for heterogeneous database data migration can automatically obtain the source data of the source database and the target data stored in the target database after the source data is migrated to the target database, perform data comparison on the source data and the target data to determine whether there are any abnormalities in the target data stored in the target database after the source data is migrated to the target database, and correct the target data with abnormalities, thereby realizing automatic correction of data migration between heterogeneous databases. Therefore, by adopting the automatic correction method for heterogeneous database data migration of the present invention, developers can realize automatic verification and correction of the data migration process through simple information configuration, ensure the accuracy of data migration between heterogeneous databases, unify the processing logic of homogeneity and universality in the data migration process, and avoid the problem of waste of R&D resources and low data migration efficiency caused by the need to develop data verification and correction programs for each data migration.

[0050] The following uses a heterogeneous database data migration automatic correction device as an example to illustrate the implementation process of the heterogeneous database data migration automatic correction method provided by the embodiment of the present invention. It is understood that the execution subject of the heterogeneous database data migration automatic correction method provided by the embodiment of the present invention includes but is not limited to the heterogeneous database data migration automatic correction device.

[0051] According to one aspect of the present invention, this embodiment discloses a method for automatically correcting heterogeneous database data migration. Figure 1 As shown, in this embodiment, the method includes:

[0052] S100: Acquire source data from a source database and target data from a target database to which the source data is migrated.

[0053] S200: performing a data comparison between the source data and the corresponding target data to determine whether there is any anomaly in the target data obtained by migrating the source data to the target database.

[0054] S300: If there is an abnormality, the target data is corrected.

[0055] The automatic correction method for heterogeneous database data migration of the present invention automatically obtains source data of a source database and target data stored in a target database after the source data is migrated to the target database, performs data comparison on the source data and the target data to determine whether the target data stored in the target database formed after the source data is migrated to the target database has any anomalies, and corrects the target data with anomalies, thereby realizing automatic correction of data migration between heterogeneous databases. Therefore, by adopting the automatic correction method for heterogeneous database data migration of the present invention, developers can realize automatic verification and correction of the data migration process through simple information configuration, ensure the accuracy of data migration between heterogeneous databases, unify the processing logic of homogeneity and universality in the data migration process, and avoid the problem of waste of R&D resources and low data migration efficiency caused by the need to develop data verification and correction programs for each data migration.

[0056] In a preferred embodiment, Figure 2 As shown, the step S100 of obtaining source data from a source database may specifically include:

[0057] S111: Sending a source data acquisition request to the source database according to a preset source data range and table structure information of the source database.

[0058] S112: Receive the source data returned by the source database according to the source data acquisition request.

[0059] Specifically, in this preferred embodiment, a source data acquisition request can be formed based on the developer's preset source data range to be corrected and the source database's table structure information. This source data acquisition request is then transmitted to the source database to retrieve the corresponding source data from the source database. The source data range can include, for example, one or more data tables involved in the data migration and the number of data entries required to be migrated. The database's table structure information can include parameters such as the primary key, index, and non-null field information of the data tables in the database. In a preferred embodiment, a source database user capable of reading data stored in the source database can be created. By simulating the source database user's access to the source database, a source data acquisition request is sent to the source database, causing the source database to return the source data corresponding to the source data acquisition request. The source data can include multiple data entries, each of which includes fields such as a primary key, non-null fields, and index information. The acquired source data can be stored in a source text file for easy access and data comparison during subsequent correction. For example, Table 1 shows the table structure information acquired from the source and target databases.

[0060] Table 1

[0061]

[0062] In a preferred embodiment, Figure 3 As shown, S100 acquiring target data migrated from the source data to the target database may specifically include:

[0063] S121: Sending a target data acquisition request to the target database according to the target data range corresponding to the preset source data range and the table structure information of the target database.

[0064] S122: Receive the target data returned by the target database according to the target data acquisition request.

[0065] Specifically, in this preferred embodiment, the source data in the preset source data range is the data to be corrected, and the target data of the target database to which the source data in the source data range is migrated corresponds to the target data range. Thus, a target data acquisition request can be formed based on the target data range corresponding to the source data range to be corrected preset by the developer and the table structure information of the target database, and the target data acquisition request can be transmitted to the target database to obtain the corresponding target data from the target database. In a preferred embodiment, a target database user that can read the data stored in the target database can be created, and by simulating the process of the target database user accessing the target database, that is, sending a target data acquisition request to the target database, the target database returns the target data corresponding to the target data acquisition request. The target data may include multiple data, each of which includes fields such as a primary key, a non-empty field, and index information. The acquired target data can be stored in a target text file for easy call in the subsequent correction process and data comparison.

[0066] In a preferred embodiment, Figure 4 As shown, the step S200 of comparing the source data with the corresponding target data to determine whether the target data obtained by migrating the source data to the target database has any anomalies specifically includes:

[0067] S210: Performing a text comparison between the source data and the target data line by line to determine whether the target data has anomalies such as missing primary keys, missing indexes, and non-empty data.

[0068] Specifically, in the above example, the source data may include multiple data items, each of which includes fields such as a primary key, non-null fields, and index information. The acquired source data can be stored in a source text file for easy access during the subsequent correction process. The target data may include multiple data items, each of which includes fields such as a primary key, non-null fields, and index information. The acquired target data can be stored in a target text file for easy access during the subsequent correction process. Thus, the source text file and the target text file can be called, and the multiple source data items and the multiple target data items stored in the source text file and the target text file can be compared one by one. That is, the primary keys, non-null fields, and index information of the corresponding two source data items and the target data items are compared to see if they are the same. If they are not the same, the target data obtained after the source data is migrated to the target database may contain anomalies. By comparing the source data and the target data item by item, it can be determined whether the target data has anomalies such as missing primary keys, missing indexes, and non-null data, thereby achieving automatic verification after data migration. Table 2 shows the results of performing a text comparison of the text files of the source database and the text files of the target database respectively through Shell script calls.

[0069] Table 2

[0070]

[0071] In a preferred embodiment, Figure 5 As shown, the correction of the target data in S300 specifically includes:

[0072] S310: performing at least one of the operations of establishing a primary key, establishing an index, and setting a field non-null attribute on the target data according to the abnormality detection result of the target data.

[0073] Specifically, by performing data comparison on the source data and the target data, it can be determined whether the target data obtained by migrating the source data to the target database has anomalies such as missing primary keys, missing indexes, and non-empty data. When the target data is automatically verified to determine that the target data has anomalies, in this preferred embodiment, the target data with the anomalies can be targeted and corrected. For example, for anomalies such as missing primary keys, missing indexes, and non-empty data, the target data can be corrected by corresponding operations such as establishing primary keys, establishing indexes, and setting field non-empty attributes based on the type of anomaly in the target data according to the source data, so as to achieve the purpose of automatic correction of the abnormal target data. In a specific example, a Shell script can be developed to call the source text file and the target text file for data comparison, and then the verification results of the comparison anomaly are stored in the result text file. According to the verification inconsistency information in the result text file, the table structure information of the target database after migration configured by the developer is called to perform correction operations such as establishing primary keys, indexes, setting field non-empty attributes, and assigning default values ​​to empty values.

[0074] In a preferred embodiment, Figure 6 As shown, the method further comprises:

[0075] S400: If there is no abnormality, convert the data content of the target data according to a preset conversion mapping relationship.

[0076] Specifically, when there is no abnormality in the data comparison result, the target data can be automatically converted according to the conversion mapping relationship pre-configured by the developer, so as to achieve the purpose of automatically converting data based on the developer's settings.

[0077] In a preferred embodiment, Figure 7 As shown, the step S400 of converting the target data content according to the preset conversion mapping relationship may further include:

[0078] S410: Determine a preset conversion mapping relationship of the target data of the field from the source database to the target database from a preset conversion mapping relationship table according to the field of the target data, the source database and the target database.

[0079] S420: Perform data content conversion on the target data of the field according to the conversion mapping relationship.

[0080] Specifically, developers can preset a conversion mapping relationship table to set the data content conversion required after the source data is migrated to the target database, and obtain the target data with the data content required by the developer. The preset conversion mapping relationship may include multiple fields and the preset conversion mapping relationship corresponding to each field in the source database and the target database. When it is determined that there is no abnormality in the target data, the preset conversion mapping relationship corresponding to each field in the source data can be determined by searching the preset conversion mapping relationship table, and the data content of the target data can be converted according to the conversion mapping relationship. For example, Table 3 shows a preset conversion mapping relationship table in a specific example, which defines the parameters and precision of the two fields of occupation code and unit net value in the source database and the target database respectively.

[0081] Table 3

[0082]

[0083] In summary, the present invention provides a method for automatic verification, correction and conversion of data migration in heterogeneous databases. By automatically checking and correcting the data during the data migration process, and performing a series of processing actions such as data content conversion on the data that has passed the verification, it finally realizes the homogeneous verification, verification and conversion of data migration between heterogeneous data sources, effectively solving the risk of requiring re-development and testing for each data migration, reducing the R&D cost and system risk of data migration, and improving the intensive level and automation level of data migration.

[0084] Based on the same principle, this embodiment also discloses a heterogeneous database data migration automatic correction device. Figure 8 As shown, in this embodiment, the device includes a data acquisition module 11 , a data verification module 12 and a data correction module 13 .

[0085] The data acquisition module 11 is used to acquire source data from a source database and target data from a target database after migrating the source data.

[0086] The data verification module 12 is used to compare the source data with the corresponding target data to determine whether there is any anomaly in the target data obtained by migrating the source data to the target database.

[0087] The data correction module 13 is used to correct the target data if an abnormality exists.

[0088] The automatic correction method for heterogeneous database data migration of the present invention automatically obtains source data of a source database and target data stored in a target database after the source data is migrated to the target database, performs data comparison on the source data and the target data to determine whether the target data stored in the target database formed after the source data is migrated to the target database has any anomalies, and corrects the target data with anomalies, thereby realizing automatic correction of data migration between heterogeneous databases. Therefore, by adopting the automatic correction method for heterogeneous database data migration of the present invention, developers can realize automatic verification and correction of the data migration process through simple information configuration, ensure the accuracy of data migration between heterogeneous databases, unify the processing logic of homogeneity and universality in the data migration process, and avoid the problem of waste of R&D resources and low data migration efficiency caused by the need to develop data verification and correction programs for each data migration.

[0089] In a preferred embodiment, the data acquisition module 11 is specifically configured to send a source data acquisition request to the source database according to a preset source data range and table structure information of the source database, and receive the source data returned by the source database according to the source data acquisition request.

[0090] Specifically, in this preferred embodiment, a source data acquisition request can be formed based on the source data range to be corrected and the table structure information of the source database preset by the developer. The source data acquisition request is transmitted to the source database to obtain the corresponding source data from the source database. In a preferred embodiment, a source database user that can read the data stored in the source database can be created. By simulating the process of the source database user accessing the source database, a source data acquisition request is sent to the source database, so that the source database returns the source data corresponding to the source data acquisition request. The source data may include multiple data items, each of which includes fields such as a primary key, a non-empty field, and index information. The acquired source data can be stored in a source text file to facilitate subsequent call and data comparison during the correction process.

[0091] In a preferred embodiment, the data acquisition module 11 is specifically configured to send a target data acquisition request to the target database according to the target data range corresponding to the preset source data range and the table structure information of the target database, and receive the target data returned by the target database according to the target data acquisition request.

[0092] Specifically, in this preferred embodiment, the source data in the preset source data range is the data to be corrected, and the target data of the target database to which the source data in the source data range is migrated corresponds to the target data range. Thus, a target data acquisition request can be formed based on the target data range corresponding to the source data range to be corrected preset by the developer and the table structure information of the target database, and the target data acquisition request can be transmitted to the target database to obtain the corresponding target data from the target database. In a preferred embodiment, a target database user that can read the data stored in the target database can be created, and by simulating the process of the target database user accessing the target database, that is, sending a target data acquisition request to the target database, the target database returns the target data corresponding to the target data acquisition request. The target data may include multiple data, each of which includes fields such as a primary key, a non-empty field, and index information. The acquired target data can be stored in a target text file for easy call in the subsequent correction process and data comparison.

[0093] In a preferred embodiment, the data verification module 12 is specifically used to perform a text comparison between the source data and the target data to determine whether the target data has any anomalies such as missing primary keys, missing indexes, and non-empty data.

[0094] Specifically, in the above example, the source data may include multiple data items, each of which includes fields such as a primary key, non-empty fields, and index information. The obtained source data can be stored in a source text file for easy access during a subsequent correction process. The target data may include multiple data items, each of which includes fields such as a primary key, non-empty fields, and index information. The obtained target data can be stored in a target text file for easy access during a subsequent correction process. Thus, the source text file and the target text file can be called, and the multiple source data items and multiple target data items stored in the source text file and the target text file can be compared one by one, that is, the primary keys, non-empty fields, and index information in the corresponding two source data items and the target data items are compared to see if they are the same. If they are not the same, the target data obtained after the source data is migrated to the target database may contain anomalies. By comparing the source data and the target data one by one, it can be determined whether the target data has anomalies such as missing primary keys, missing indexes, and non-empty data, thereby achieving automatic verification after data migration.

[0095] In a preferred embodiment, the data correction module 13 is specifically configured to perform at least one of the operations of establishing a primary key, establishing an index, and setting a field non-null attribute on the target data according to the abnormality detection result of the target data.

[0096] Specifically, by performing data comparison on the source data and the target data, it can be determined whether the target data obtained by migrating the source data to the target database has anomalies such as missing primary keys, missing indexes, and non-empty data. When the target data is automatically verified to determine that the target data has anomalies, in this preferred embodiment, the target data with the anomalies can be targeted and corrected. For example, for anomalies such as missing primary keys, missing indexes, and non-empty data, the target data can be corrected by corresponding operations such as establishing primary keys, establishing indexes, and setting field non-empty attributes based on the type of anomaly in the target data according to the source data, so as to achieve the purpose of automatic correction of the abnormal target data. In a specific example, a Shell script can be developed to call the source text file and the target text file for data comparison, and then the verification results of the comparison anomaly are stored in the result text file. According to the verification inconsistency information in the result text file, the table structure information of the target database after migration configured by the developer is called to perform correction operations such as establishing primary keys, indexes, setting field non-empty attributes, and assigning default values ​​to empty values.

[0097] In a preferred embodiment, Figure 9 As shown, the device further includes a data conversion module 14. The data conversion module 14 is configured to convert the data content of the target data according to a preset conversion mapping relationship if no abnormality exists.

[0098] Specifically, when there is no abnormality in the data comparison result, the target data can be automatically converted according to the conversion mapping relationship pre-configured by the developer, so as to achieve the purpose of automatically converting data based on the developer's settings.

[0099] In a preferred embodiment, the data conversion module 14 is specifically configured to determine, from a preset conversion mapping table, a preset conversion mapping relationship for the target data in the field from the source database to the target database based on the field of the target data, the source database, and the target database, and perform data content conversion on the target data in the field according to the conversion mapping relationship.

[0100] Specifically, developers can preset a conversion mapping relationship table to set the data content conversion required after the source data is migrated to the target database, and obtain the target data with the data content required by the developer. The preset conversion mapping relationship may include multiple fields and the preset conversion mapping relationship corresponding to each field in the source database and the target database. When it is determined that there is no abnormality in the target data, the preset conversion mapping relationship corresponding to each field in the source data can be determined by searching the preset conversion mapping relationship table, and the data content of the target data can be converted according to the conversion mapping relationship. For example, Table 1 shows a preset conversion mapping relationship table in a specific example, which defines the parameters and precision of the two fields of occupation code and unit net value in the source database and the target database respectively.

[0101] In summary, the present invention provides an automatic verification, correction and conversion device for data migration in heterogeneous databases. By automatically checking and correcting the data during the data migration process, and performing a series of processing actions such as data content conversion on the data that has passed the verification, it finally realizes the homogeneous verification, verification and conversion of data migration between heterogeneous data sources, effectively solving the risk of requiring re-development and testing for each data migration, reducing the R&D cost and system risk of data migration, and improving the intensive level and automation level of data migration.

[0102] Since the principle of solving the problem by this device is similar to that of the above method, the implementation of this device can refer to the implementation of the method and will not be repeated here.

[0103] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer device. Specifically, the computer device may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0104] In a typical example, a computer device specifically includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the method executed by the client as described above is implemented, or when the processor executes the program, the method executed by the server as described above is implemented.

[0105] Reference below Figure 10 , which shows a structural diagram of a computer device 600 suitable for implementing an embodiment of the present application.

[0106] like Figure 10 As shown, computer device 600 includes a central processing unit (CPU) 601, which can perform various appropriate tasks and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage portion 608 into a random access memory (RAM) 603. Various programs and data required for the operation of system 600 are also stored in RAM 603. CPU 601, ROM 602, and RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to bus 604.

[0107] The following components are connected to the I / O interface 605: an input section 606 including a keyboard, a mouse, and the like; an output section 607 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage section 608 including devices such as a hard disk; and a communication section 609 including a network interface card such as a LAN card or a modem. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. Removable media 611, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 610 as needed, so that computer programs read therefrom can be installed in the storage section 608 as needed.

[0108] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program tangibly embodied on a machine-readable medium, the computer program including program code for executing the methods illustrated in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication portion 609 and / or installed from removable media 611.

[0109] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0110] For the convenience of description, the above devices are described as being divided into various units according to their functions. Of course, when implementing this application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0111] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0112] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0113] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0114] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0115] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0116] The present application may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communications network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0117] The various embodiments in this specification are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the system embodiments are generally similar to the method embodiments, so the description is relatively simple. For relevant parts, refer to the description of the method embodiments.

[0118] The foregoing is merely an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A method for automatic correction of heterogeneous database data migration, characterized in that: include: Acquire source data from a source database and migrate target data from the source data to a target database; Comparing the source data with the corresponding target data to determine whether there is any anomaly in the target data obtained by migrating the source data to the target database; If there is an abnormality, the target data is corrected; The data comparison between the source data and the corresponding target data to determine whether the target data obtained by migrating the source data to the target database has an abnormality specifically includes: Performing a text comparison on the source data and the target data line by line to determine whether the target data has anomalies such as missing primary keys, missing indexes, and non-empty data; The obtained source data is stored in a source text file, the target data includes multiple data, each of which includes a primary key, a non-empty field and index information, and the obtained target data is stored in a target text file, thereby calling the source text file and the target text file, and comparing the multiple source data and the multiple target data stored in the source text file and the target text file one by one, that is, comparing whether the primary keys, non-empty fields and index information in the corresponding two source data and target data are the same to determine whether there is an abnormality in the target data obtained by migrating the source data to the target database.

2. The method for automatic correction of heterogeneous database data migration according to claim 1, characterized in that: The obtaining of source data from a source database specifically includes: Sending a source data acquisition request to the source database according to a preset source data range and table structure information of the source database; The source data returned by the source database according to the source data acquisition request is received.

3. The method for automatic correction of heterogeneous database data migration according to claim 2, characterized in that: Acquiring the target data migrated from the source data to the target database specifically includes: Sending a target data acquisition request to the target database according to the target data range corresponding to the preset source data range and the table structure information of the target database; The target data returned by the target database according to the target data acquisition request is received.

4. The method for automatic correction of heterogeneous database data migration according to claim 1, characterized in that: Further including: If an exception exists, at least one of the operations of establishing a primary key, establishing an index, and setting a field non-null attribute is performed on the target data according to the exception detection result of the target data.

5. The method for automatic correction of heterogeneous database data migration according to claim 1, characterized in that: Further including: If there is no abnormality, the target data is converted in data content according to a preset conversion mapping relationship.

6. The method for automatic correction of heterogeneous database data migration according to claim 5, characterized in that: The data content conversion of the target data according to the preset conversion mapping relationship specifically includes: Determine a preset conversion mapping relationship of the target data of the field from the source database to the target database from a preset conversion mapping relationship table according to the field of the target data, the source database and the target database; The target data of the field is converted in data content according to the conversion mapping relationship.

7. An automatic correction device for heterogeneous database data migration, characterized in that: include: A data acquisition module, used to acquire source data from a source database and target data from a target database after the source data is migrated; A data verification module, used to compare the source data with the corresponding target data to determine whether there is any anomaly in the target data obtained by migrating the source data to the target database; A data correction module, used to correct the target data if an abnormality exists; The data verification module is specifically used to: compare the source data and the target data text by text to determine whether the target data has anomalies such as missing primary key, missing index, and non-empty data; The obtained source data is stored in a source text file, the target data includes multiple data, each of which includes a primary key, a non-empty field and index information, and the obtained target data is stored in a target text file, thereby calling the source text file and the target text file, and comparing the multiple source data and the multiple target data stored in the source text file and the target text file one by one, that is, comparing whether the primary keys, non-empty fields and index information in the corresponding two source data and target data are the same to determine whether there is an abnormality in the target data obtained by migrating the source data to the target database.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 6 is implemented.

9. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images