Database migration verification method, device, equipment and medium

Abstract

The embodiment of the present invention discloses a database migration verification method, device, equipment and medium, which relates to the field of big data technology. The method includes: determining a data source and a reference program group, and determining a program group to be verified; obtaining migration verification data from a data source according to a preset data source time, and determining a reference data table and a reference running time based on the migration verification data, and determining a data table to be verified and a running time to be verified; generating a candidate verification combination; comparing the reference running time corresponding to the candidate verification combination with the running time to be verified to determine the program running performance, and comparing the reference data table corresponding to the candidate verification combination with the data table to be verified to determine the program code status; determining the program migration verification result according to the program running performance and / or program code status corresponding to each candidate verification combination, and performing migration verification on the new database according to the program migration verification result. The efficiency and accuracy of database migration verification are improved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of big data technology, and in particular to a database migration verification method, apparatus, device, and medium. Background Art

[0002] With the continuous development of the big data industry, database migrations involving massive amounts of data will face increasing challenges in future database transformations. Database transformation for big data applications (such as data warehouses) is a massive undertaking. Due to differences in structure and syntax across database types, the database transformation process involves modifying thousands of program operations, creating a massive workload. Therefore, improving the accuracy and efficiency of database migration verification is crucial. Summary of the Invention

[0003] The present invention provides a database migration verification method, device, equipment and medium to improve the accuracy and efficiency of database migration verification.

[0004] According to one aspect of the present invention, a database migration verification method is provided, comprising:

[0005] Obtaining a preset data source time, determining the data source and a reference program group in the old database, and determining a program group to be verified in the new database based on the reference program group; wherein the reference program group is determined based on a reference benchmark program and a reference association configuration table determined from a candidate area layer of the old database;

[0006] Obtaining migration verification data from the data source according to the preset data source time, determining a reference data table and a reference running time corresponding to each reference program in the reference program group based on the migration verification data, and determining a data table to be verified and a running time to be verified corresponding to each program to be verified in the program to be verified group based on the migration verification data;

[0007] generating a candidate verification combination including a reference program and a program to be verified having the same program identifier;

[0008] Comparing the reference running time corresponding to the candidate verification combination with the running time to be verified to determine the program running performance of the program to be verified in the candidate verification combination, and comparing the reference data table corresponding to the candidate verification combination with the data table to be verified to determine the program code status of the program to be verified in the candidate verification combination;

[0009] According to the program running performance and / or program code status corresponding to each candidate verification combination, the program migration verification result of the program to be verified in each candidate verification combination is determined, and according to the program migration verification result, the new database is migrated and verified.

[0010] According to another aspect of the present invention, there is provided a database migration verification device, comprising:

[0011] a program group determination module, configured to obtain a preset data source time, determine the data source and a reference program group in the old database, and determine a program group to be verified in the new database based on the reference program group; wherein the reference program group is determined based on a reference benchmark program determined from a candidate region layer in the old database and a reference association configuration table;

[0012] a verification data determination module, configured to obtain migration verification data from the data source according to the preset data source time, determine a reference data table and a reference running time corresponding to each reference program in the reference program group based on the migration verification data, and determine a to-be-verified data table and a to-be-verified running time corresponding to each to-be-verified program in the to-be-verified program group based on the migration verification data;

[0013] A verification combination generating module, configured to generate a candidate verification combination including a reference program and a program to be verified having the same program identifier;

[0014] a verification data comparison module, configured to compare the reference runtime corresponding to the candidate verification combination with the runtime to be verified to determine the program running performance of the program to be verified in the candidate verification combination, and to compare the reference data table corresponding to the candidate verification combination with the data table to be verified to determine the program code status of the program to be verified in the candidate verification combination;

[0015] The migration verification module is used to determine the program migration verification results of the programs to be verified in each candidate verification combination according to the program running performance and / or program code status corresponding to each candidate verification combination, and perform migration verification on the new database according to the program migration verification results.

[0016] According to another aspect of the present invention, there is provided an electronic device, comprising:

[0017] one or more processors;

[0018] a memory for storing one or more programs;

[0019] When one or more programs are executed by one or more processors, the one or more processors can execute any one of the database migration verification methods provided by the embodiments of the present invention.

[0020] According to another aspect of the present invention, a computer-readable storage medium is provided. The computer-readable storage medium stores computer instructions, which are used to enable a processor to implement any database migration verification method provided by an embodiment of the present invention when executed.

[0021] An embodiment of the present invention provides a database migration verification solution, which determines a reference program group in a data source and an old database by obtaining a preset data source time, and determines a program group to be verified in a new database based on the reference program group; wherein the reference program group is determined based on a reference benchmark program and a reference associated configuration table determined from a candidate area layer of the old database; migration verification data is obtained from the data source according to the preset data source time, and based on the migration verification data, a reference data table and a reference running time corresponding to each reference program in the reference program group are determined, and based on the migration verification data, a data table and a reference running time corresponding to each program to be verified in the program group to be verified are determined. Running time; generating a candidate verification combination including a reference program and a program to be verified with the same program identifier; comparing the reference running time corresponding to the candidate verification combination with the running time to be verified, determining the program running performance of the program to be verified in the candidate verification combination, and comparing the reference data table corresponding to the candidate verification combination with the data table to be verified, determining the program code status of the program to be verified in the candidate verification combination; determining the program migration verification result of the program to be verified in each candidate verification combination according to the program running performance and / or program code status corresponding to each candidate verification combination, and performing migration verification on the new database according to the program migration verification result. The above scheme realizes the automation of database migration verification and improves the efficiency of database migration verification by determining the program migration verification result of the candidate verification combination according to the program running performance and program code status of the candidate verification combination, and then performing migration verification on the new database; at the same time, the embodiment of the present invention performs migration verification on the new database from two aspects of program running performance and program code status, thereby improving the accuracy of database migration verification.

[0022] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 This is a flowchart of a database migration verification method provided by the first embodiment of the present invention;

[0025] Figure 2 This is a flowchart of a database migration verification method provided by the second embodiment of the present invention;

[0026] Figure 3A This is a schematic diagram of a hierarchical framework for overall data loading in a data warehouse provided by a third embodiment of the present invention;

[0027] Figure 3B This is a schematic diagram of a data flow link provided by the third embodiment of the present invention;

[0028] Figure 3C This is a schematic diagram of the dependency relationship between reference programs of different candidate area layers provided by the third embodiment of the present invention;

[0029] Figure 3D This is an operation flow chart of a data flow application module provided in the third embodiment of the present invention;

[0030] Figure 3E This is an operation flow chart of a data flow acquisition module provided in the third embodiment of the present invention;

[0031] Figure 3F This is an operation flow chart of a data flow comparison module provided in the third embodiment of the present invention;

[0032] Figure 3G This is an operation flow chart of a full-process data flow module provided by the third embodiment of the present invention;

[0033] Figure 4 This is a schematic diagram of the structure of a database migration verification device provided by the fourth embodiment of the present invention;

[0034] Figure 5 This is a schematic diagram of the structure of an electronic device for implementing a database migration verification method provided in Example 5 of the present invention. DETAILED DESCRIPTION

[0035] The present invention will be further described in detail below with reference to the accompanying drawings and examples. It will be understood that the specific embodiments described herein are intended only to illustrate the present invention and are not intended to limit the present invention. It should also be noted that, for ease of description, the accompanying drawings only illustrate portions relevant to the present invention, not all structures.

[0036] Example 1

[0037] Figure 1 This is a flowchart of a database migration verification method provided in Example 1 of the present invention. This embodiment is applicable to situations where a migrated database is to be verified. The method can be executed by a database migration verification device, which can be implemented in software and / or hardware and can be configured in an electronic device that carries the database migration verification function.

[0038] See also Figure 1 The database migration verification method shown includes:

[0039] S110 , obtaining a preset data source time, determining the data source and a reference program group in the old database, and determining a program group to be verified in the new database based on the reference program group.

[0040] The preset data source time refers to the time at which data is retrieved from the data source, i.e., data corresponding to the preset data source time is retrieved from the data source. The data source can be used to provide migration verification data. Exemplarily, the data source can be different business systems. Migration verification data refers to data retrieved from the data source and used to verify the database migration.

[0041] The old database refers to the original database. For example, the old database can be an old data warehouse. The reference program group refers to a combination of reference programs involved in database migration verification. The reference program refers to the program in the old database involved in migration verification.

[0042] The new database refers to the database obtained by migrating the old database. For example, the new database can be a new data warehouse. The group of programs to be verified refers to a combination of programs to be verified that participate in database migration verification. Programs to be verified refer to programs in the new database that participate in migration verification.

[0043] Exemplarily, the reference program group is determined based on a reference benchmark program determined from a candidate region layer in the legacy database and a reference association configuration table. The candidate region layer refers to a structural region layer in the legacy database. The reference benchmark program can be used to determine other reference programs in the reference program group other than the reference benchmark program. The reference association configuration table, also known as a dependency configuration table, can be used to record the dependency relationships between various programs in the legacy database.

[0044] It should be noted that the reference programs in the reference program group correspond one-to-one to the programs to be verified in the program group to be verified.

[0045] S120. Obtain migration verification data from the data source according to the preset data source time, determine the reference data table and reference running time corresponding to each reference program in the reference program group based on the migration verification data, and determine the data table to be verified and running time to be verified corresponding to each program to be verified in the program group to be verified based on the migration verification data.

[0046] The reference data table refers to the running results of the reference program. The reference runtime refers to the runtime of the reference program. The data table to be verified refers to the running results of the program to be verified. The runtime to be verified refers to the runtime of the program to be verified.

[0047] S130: Generate a candidate verification combination including a reference program and a program to be verified having the same program identifier.

[0048] The program identifier can be used to uniquely identify the program. A candidate verification combination refers to a set of reference programs and programs to be verified that have the same program identifier. It should be noted that the corresponding reference programs and programs to be verified all have the same program identifier.

[0049] S140. Compare the reference running time corresponding to the candidate verification combination with the running time to be verified to determine the program running performance of the program to be verified in the candidate verification combination, and compare the reference data table corresponding to the candidate verification combination with the data table to be verified to determine the program code status of the program to be verified in the candidate verification combination.

[0050] The program performance can represent the performance of the program to be verified. For example, the program performance can include normal program performance and abnormal program performance. Normal program performance means that the program to be verified has no abnormalities. Abnormal program performance means that the program to be verified has abnormalities.

[0051] The program code status can represent the coding status of the program to be verified. For example, the program code status can include normal program code and abnormal program code. Normal program code means that the code in the program to be verified is accurately written. Abnormal program code means that there are errors in the coding of the program to be verified.

[0052] In an optional embodiment, the reference running time corresponding to the candidate verification combination and the running time to be verified are compared to determine the program running performance of the program to be verified in the candidate verification combination, including: comparing the reference running time corresponding to the candidate verification combination and the running time to be verified to obtain a running time deviation value; determining the program running performance of the program to be verified in the candidate verification combination based on the running time deviation value and a preset deviation threshold.

[0053] The runtime deviation value can represent the difference between the reference runtime and the runtime to be verified. For example, the difference between the reference runtime corresponding to the reference program in any candidate verification combination and the runtime to be verified corresponding to the program to be verified in the candidate verification combination is used as the runtime deviation value of the candidate verification combination.

[0054] The embodiment of the present invention does not impose any limitation on the size of the preset deviation threshold, which can be set by technicians based on experience or needs, or determined repeatedly through a large number of experiments.

[0055] It can be understood that by introducing the running time deviation value and the preset deviation threshold to determine the program running performance of the program to be verified, the accuracy and reliability of the determined program running performance are improved.

[0056] In an optional embodiment, the program running performance of the program to be verified in the candidate verification combination is determined based on the running time deviation value and the preset deviation threshold, including: if the running time deviation value is greater than the preset deviation threshold, then the program running performance of the program to be verified in the candidate verification combination is determined to be abnormal program performance; if the running time deviation value is less than or equal to the preset deviation threshold, then the program running performance of the program to be verified in the candidate verification combination is determined to be normal program performance.

[0057] Exemplarily, for any candidate verification combination, if the running time deviation value of the candidate verification combination is greater than the preset deviation threshold, then the program running performance of the program to be verified in the candidate verification combination is determined to be abnormal program performance; if the running time deviation value of the candidate verification combination is less than or equal to the preset deviation threshold, then the program running performance of the program to be verified in the candidate verification combination is determined to be normal program performance.

[0058] It can be understood that by specifically limiting the program running performance of the program to be verified in the candidate verification combination according to the preset deviation threshold and the running time deviation value, the accuracy of the determined program running performance is improved.

[0059] Exemplarily, for any candidate verification combination, if the program running performance of the program to be verified in the candidate verification combination is abnormal, the program to be verified in the candidate verification combination is optimized.

[0060] In an optional embodiment, the reference data table corresponding to the candidate verification combination and the data table to be verified are compared to determine the program code status of the program to be verified in the candidate verification combination, including: determining the reference data volume and reference data details in the reference data table corresponding to the candidate verification combination, and determining the reference data volume and data details to be verified in the data table to be verified corresponding to the candidate verification combination; comparing the reference data volume and the data volume to be verified to obtain a data volume comparison result, and comparing the reference data details and the data details to be verified to obtain a data detail comparison result; determining the program code status of the program to be verified in the candidate verification combination based on the data volume comparison result and the data detail comparison result corresponding to the candidate verification combination.

[0061] The reference data volume refers to the number of data items in the reference data table. The reference data details refer to the field contents in the reference data table. The pending data volume refers to the number of data items in the pending data table. The pending data details refer to the field contents in the pending data table.

[0062] The data volume comparison result refers to the result obtained by comparing the reference data volume corresponding to a candidate verification combination with the data volume to be verified. For example, the data volume comparison result may include a normal data volume comparison and an abnormal data volume comparison. A normal data volume comparison means that the reference data volume corresponding to a candidate verification combination is the same as the data volume to be verified. An abnormal data volume comparison means that the reference data volume corresponding to a candidate verification combination is different from the data volume to be verified.

[0063] The data detail comparison result refers to the result obtained by comparing the reference data details corresponding to a candidate verification combination with the data details to be verified. For example, the data detail comparison result may include a normal data detail comparison and an abnormal data detail comparison. A normal data detail comparison means that the reference data details corresponding to a candidate verification combination are identical to the data details to be verified. A abnormal data detail comparison means that the reference data details corresponding to a candidate verification combination are different from the data details to be verified.

[0064] Exemplarily, the program code status of the program to be verified in the candidate verification combination is determined based on the data volume comparison result and / or data detail comparison result corresponding to the candidate verification combination, including: for any candidate verification combination, if the data volume comparison result of the candidate verification combination is that the data volume comparison is abnormal, then the program code status of the program to be verified in the candidate verification combination is determined to be program code abnormal; if the data detail comparison result of the candidate verification combination is that the data detail comparison is abnormal, then the program code status of the program to be verified in the candidate verification combination is determined to be program code abnormal; if the data volume comparison result of the candidate verification program combination is that the data volume comparison is normal, and the data detail comparison result of the candidate verification program combination is that the data detail comparison is normal, then the program code status of the program to be verified in the candidate verification combination is determined to be program code normal.

[0065] It can be understood that by determining the program code status of the program to be verified in the candidate verification combination based on the quantity comparison results and the data detail comparison results, the program code status is determined from multiple angles, thereby improving the accuracy of the determined program code status.

[0066] In an optional embodiment, the method also includes: if the program code status of the program to be verified in the candidate verification combination is a program code exception, then determining the operation exception data in the data table to be verified corresponding to the candidate verification combination based on the data volume comparison results and / or data detail comparison results corresponding to the candidate verification combination; and updating the program to be verified in the candidate verification combination based on the operation exception data.

[0067] The abnormal operation data refers to abnormal data in the data table to be verified of the program to be verified.

[0068] Exemplarily, if the data volume comparison result of the candidate verification combination is a data volume comparison anomaly, the data corresponding to the data volume comparison anomaly is determined from the data table to be verified as at least part of the operation abnormality data; if the data detail comparison result of the candidate verification combination is a data detail comparison anomaly, the data corresponding to the data detail comparison anomaly is determined from the data table to be verified as at least part of the operation abnormality data.

[0069] For example, when comparing the data volume and details of the reference data table corresponding to the candidate verification combination with the data table to be verified, the comparison is performed automatically within the device. The device only displays the abnormal operation data. If sensitive data is included in the abnormal operation data, the sensitive data in the abnormal operation data is first deformed before being displayed. The benefit is that the data security is improved.

[0070] Exemplarily, for any candidate verification combination, if the program code status of the program to be verified in the candidate verification combination is a program code exception, then based on the data volume comparison results and / or data detail comparison results corresponding to the candidate verification combination, the operation exception data is determined from the data table to be verified corresponding to the program to be verified in the candidate verification combination; based on the operation exception data, the program to be verified in the candidate verification combination is updated.

[0071] The abnormal operation data in the embodiment of the present invention can be understood as data in which there are differences between the reference data table corresponding to a candidate verification combination and the data table to be verified.

[0072] It can be understood that by updating the program to be verified based on the abnormal operation data, the accuracy of the program update is improved.

[0073] Specifically, for any candidate verification combination, the reference running time corresponding to the reference program in the candidate verification combination is compared with the running time to be verified corresponding to the program to be verified in the candidate verification combination to determine the program running performance of the program to be verified in the candidate verification combination; the reference data table corresponding to the reference program in the candidate verification combination is compared with the data table to be verified corresponding to the program to be verified in the candidate verification combination to determine the program code status of the program to be verified in the candidate verification combination.

[0074] S150 , determining the program migration verification results of the programs to be verified in each candidate verification combination according to the program running performance and / or program code status corresponding to each candidate verification combination, and performing migration verification on the new database according to the program migration verification results.

[0075] The program migration verification result refers to the result of performing migration verification on the program to be verified in the new database. For example, the program migration verification result may include a successful program migration verification and a failed program migration verification. A successful program migration verification indicates that the program to be verified has no anomalies, indicating that the migration of the program to be verified in the new database has no anomalies. A failed program migration verification indicates that the program to be verified has an anomaly, indicating that the migration of the program to be verified in the new database has an anomaly.

[0076] Exemplarily, for any candidate verification combination, if the program running performance corresponding to the candidate verification combination is normal program performance, and the program code status corresponding to the candidate verification combination is normal program code, then the program migration verification result of the program to be verified in the candidate verification combination is determined to be a successful program migration verification; if not, then the program migration verification result of the program to be verified in the candidate verification combination is determined to be a failed program migration verification.

[0077] Furthermore, if the program migration verification results corresponding to each candidate verification combination are all successful, then the new database migration verification is determined to be passed; if there is at least one program migration verification failure among the program migration verification results corresponding to each candidate verification combination, then the new database migration verification is determined to be unsuccessful.

[0078] Specifically, based on the program running performance and / or program code status corresponding to each candidate verification combination, the program migration verification result of the program to be verified in each candidate verification combination is determined, and the new database is migrated and verified based on the program migration verification result of each candidate verification combination.

[0079] An embodiment of the present invention provides a database migration verification solution, which determines a reference program group in a data source and an old database by obtaining a preset data source time, and determines a program group to be verified in a new database based on the reference program group; wherein the reference program group is determined based on a reference benchmark program and a reference associated configuration table determined from a candidate area layer of the old database; migration verification data is obtained from the data source according to the preset data source time, and based on the migration verification data, a reference data table and a reference running time corresponding to each reference program in the reference program group are determined, and based on the migration verification data, a data table and a reference running time corresponding to each program to be verified in the program group to be verified are determined. Running time; generating a candidate verification combination including a reference program and a program to be verified with the same program identifier; comparing the reference running time corresponding to the candidate verification combination with the running time to be verified, determining the program running performance of the program to be verified in the candidate verification combination, and comparing the reference data table corresponding to the candidate verification combination with the data table to be verified, determining the program code status of the program to be verified in the candidate verification combination; determining the program migration verification result of the program to be verified in each candidate verification combination according to the program running performance and / or program code status corresponding to each candidate verification combination, and performing migration verification on the new database according to the program migration verification result. The above scheme realizes the automation of database migration verification and improves the efficiency of database migration verification by determining the program migration verification result of the candidate verification combination according to the program running performance and program code status of the candidate verification combination, and then performing migration verification on the new database; at the same time, the embodiment of the present invention performs migration verification on the new database from two aspects of program running performance and program code status, thereby improving the accuracy of database migration verification.

[0080] Example 2

[0081] Figure 2 This is a flowchart of a database migration verification method provided by the second embodiment of the present invention. Based on the above embodiments, this embodiment further refines the operation of "determining the data source and the reference program group in the old database" into "selecting a reference benchmark program from the candidate area layer of the old database, parsing the execution status of the reference benchmark program, and determining whether there is a predecessor data table from the parsing result; wherein, the candidate area layer includes a temporary area, a basic area, and a summary area; if so, determining the reference predecessor program corresponding to the predecessor data table; determining whether there is a reference successor program for the reference benchmark program according to the reference association configuration table corresponding to the reference benchmark program; if so, generating a reference program group including a reference benchmark program, a reference predecessor program, and a reference successor program; determining the data source according to the reference program in the temporary area in the reference program group" to improve the determination mechanism of the data source and the reference program group in the old database. It should be noted that for the parts not described in detail in the embodiments of the present invention, reference can be made to the descriptions of other embodiments.

[0082] See also Figure 2 The database migration verification method shown includes:

[0083] S200: Obtain preset data source time.

[0084] S210 , selecting a reference benchmark program from the candidate area layer of the old database, analyzing the execution status of the reference benchmark program, and determining whether a preceding data table exists from the analysis result.

[0085] The preceding data table may be understood as a reference data table corresponding to a reference program preceding the reference benchmark program.

[0086] Exemplarily, the candidate area layer includes a temporary area, a basic area, and a summary area. Reference programs in the temporary area can obtain a reference data table corresponding to the reference programs in the temporary area based on migration verification data obtained from a data source. Reference programs in the basic area can obtain a reference data table corresponding to the reference programs in the basic area based on the reference data table in the temporary area. Reference programs in the summary area can obtain a reference data table corresponding to the reference programs in the summary area based on the reference data table in the basic area.

[0087] Exemplarily, a reference area layer is determined from candidate area layers of an old database; a reference benchmark program is selected from the reference area layer; the execution of the reference benchmark program is analyzed, and whether a preceding data table is required during the execution of the reference benchmark program is determined from the analysis result.

[0088] The reference layer refers to a layer that can provide a reference benchmark program. For example, the reference layer can be a basic layer or a summary layer.

[0089] For example, if a reference benchmark program is selected from the basic area, the corresponding preceding data table includes the reference data table corresponding to the reference program in the temporary area; if a reference benchmark program is selected from the summary area, the corresponding preceding data table includes the reference data table corresponding to the reference program in the temporary area and the reference data table corresponding to the reference program in the summary area.

[0090] S220: If it exists, determine the reference predecessor program corresponding to the predecessor data table.

[0091] The reference predecessor program refers to a program that is associated with the reference base program and is executed before the reference base program.

[0092] Specifically, if a previous data table exists in the analysis result, it indicates that the execution of the reference benchmark program requires the participation of the reference data table of the previous candidate layer, and the corresponding reference previous program is determined based on the previous data table. It should be noted that there is a one-to-one correspondence between the program and the data table.

[0093] S230 : Determine whether the reference benchmark program has a reference subsequent program according to the reference association configuration table corresponding to the reference benchmark program.

[0094] The reference subsequent program refers to a reference program that is associated with the reference base program and is executed after the reference base program.

[0095] Exemplarily, other programs associated with the reference benchmark program are searched in the reference association configuration table to determine whether there is a program executed after the reference benchmark program in the other programs, that is, whether there is a reference subsequent program.

[0096] Exemplarily, if the reference benchmark program is a reference program in the basic area, the reference subsequent program may be a reference program in the summary area; if the reference benchmark program is a reference program in the summary area, there is no reference subsequent program.

[0097] Specifically, it is determined whether the reference base program has a reference subsequent program according to the reference association configuration table of the old database. The reference association configuration table is a pre-set table used to record the dependency relationship between various programs in the old database.

[0098] S240 , if it exists, generating a reference program group including a reference base program, a reference predecessor program, and a reference successor program.

[0099] Exemplarily, if the reference benchmark program is a reference program in the basic area, there is a reference subsequent program, and a reference program group including the reference benchmark program, the reference preceding program and the reference subsequent program is generated; if the reference benchmark program is a reference program in the summary area, there is no reference subsequent program, and a reference program group including the reference benchmark program and the reference preceding program is generated.

[0100] S250 : Determine a data source based on the reference programs in the temporary area of ​​the reference program group, and determine a program group to be verified in the new database based on the reference program group.

[0101] Illustratively, there is a one-to-one correspondence between the reference program in the temporary area and the data source, and the corresponding data source can be determined based on the reference program in the temporary area.

[0102] S260. Obtain migration verification data from the data source according to the preset data source time, determine the reference data table and reference running time corresponding to each reference program in the reference program group based on the migration verification data, and determine the data table to be verified and running time to be verified corresponding to each program to be verified in the program group to be verified based on the migration verification data.

[0103] S270: Generate a candidate verification combination including a reference program and a program to be verified having the same program identifier.

[0104] S280. Compare the reference running time corresponding to the candidate verification combination with the running time to be verified to determine the program running performance of the program to be verified in the candidate verification combination, and compare the reference data table corresponding to the candidate verification combination with the data table to be verified to determine the program code status of the program to be verified in the candidate verification combination.

[0105] S290. Determine the program migration verification result of the program to be verified in each candidate verification combination according to the program running performance and / or program code status corresponding to each candidate verification combination, and perform migration verification on the new database according to the program migration verification result.

[0106] An embodiment of the present invention provides a database migration verification solution, which refines the operation of determining the data source and the reference program group in the old database into selecting a reference benchmark program from the candidate area layer of the old database, parsing the execution status of the reference benchmark program, and determining whether a predecessor data table exists from the parsing result; wherein the candidate area layer includes a temporary area, a basic area, and a summary area; if so, determining the reference predecessor program corresponding to the predecessor data table; determining whether the reference benchmark program has a reference successor program based on the reference association configuration table corresponding to the reference benchmark program; if so, generating a reference program group including the reference benchmark program, a reference predecessor program, and a reference successor program; determining the data source based on the reference program in the temporary area in the reference program group, thereby improving the determination mechanism of the data source and the reference program group in the old database. The above scheme improves the accuracy of the reference program group by determining the reference predecessor program based on the analysis results of the reference benchmark program, and determining the reference successor program based on the reference association configuration table, thereby generating a reference program group including the reference benchmark program, the reference predecessor program and the reference successor program. At the same time, by analyzing the execution status of the reference benchmark program and determining from the analysis results whether there is a predecessor data table, if so, determining the reference predecessor program corresponding to the predecessor data table, it is achieved that the reference predecessor program is determined from the perspective of the actual operation status of the reference benchmark program, thereby improving the accuracy of the determined reference predecessor program.

[0107] Based on the above technical solution, an embodiment of the present invention further provides a method for updating reference programs in a reference program group. In an optional embodiment, the method further includes: traversing each reference program in the reference program group and updating the reference program in the reference program group based on the traversal result.

[0108] The traversal result refers to the result obtained by traversing the reference programs in the reference program group.

[0109] Exemplarily, each reference program in the reference program group is executed sequentially to obtain a traversal result; if the traversal result is normal, there is no need to update the reference programs in the reference program group; if the traversal result is abnormal, the program abnormality data of the traversal abnormality is determined, and the reference programs in the reference program group are updated according to the program abnormality data.

[0110] A normal traversal can indicate that the reference program code in the reference program group is accurate and the program execution chain is complete. An abnormal traversal can indicate that the reference program code in the reference program group contains errors and / or that the program execution chain is incomplete. An incomplete program execution chain means that at least one reference program is missing from the reference program group. A complete program execution chain means that all reference programs are missing from the reference program group.

[0111] The program exception data may represent the cause of the traversal exception. For example, the program exception data may include a reference program in the reference program group having a certain exception, or a reference program missing from the reference program group.

[0112] In the embodiment of the present invention, since the content recorded in the reference association configuration table may be incomplete, and the reference subsequent program determined according to the reference association configuration table may also be incomplete, it is necessary to traverse the reference programs in the reference program group and update the reference programs in the reference program group in a timely manner, thereby improving the comprehensiveness and accuracy of the reference programs in the reference program group.

[0113] Example 3

[0114] The embodiment of the present invention provides an optional example based on the above embodiment. It should be noted that for parts not described in detail in the embodiment of the present invention, reference can be made to the descriptions of other embodiments.

[0115] The development of the information and innovation industry has become the key to achieving digital transformation and promoting the development of the industrial chain. The existing technology has a process of migrating from TeraData databases to various distributed databases (including MppDB and Hadoop databases). In the future, the migration of various databases will also become a common project. For the post-testing and verification of database migration, it is necessary to confirm the result data after running each batch program to ensure the high consistency of the data. However, verification through manual number generation obviously cannot meet the test requirements, and production data cannot be used as test data.

[0116] An embodiment of the present invention proposes a framework for a database migration verification system that involves a large number of batch program job modifications during database migration. The database migration verification system is a closed production data flow system. On the old database in the production environment, while obtaining a series of table data generated during the batch program operation, it will use the same production data source to run and load each modified new program job on the new database in the test environment to obtain a series of table data on the new database, and then compare the table data content on the old and new databases to determine the correctness of the migration and modification of each program job (wherein, due to the confidentiality of production data, only when inconsistencies are found can the details of the partially transformed data be displayed). Among them, the database migration verification system can be used to verify database migration.

[0117] This paper takes a common big data application, data warehouse, as an example to specifically illustrate the database migration verification method.

[0118] Figure 3A It shows the overall data loading layered framework of the data warehouse: at the temporary area layer, each temporary area program job corresponds to a data source, and the corresponding temporary area data table is obtained after loading and running; at the basic area layer, each basic area program job will use one or more temporary area data tables or other basic area program jobs in the same layer to run the loaded data table, and then get the corresponding basic area data table after running the load; at the summary area layer, each summary area program job will use one or more basic area data tables or other summary area program jobs in the same layer to run the loaded data table, and then get the corresponding summary area data table after running the load.

[0119] The database migration verification system in the embodiment of the present invention is mainly divided into the following five modules: user management module, data flow application module, data flow acquisition module, data flow comparison module and full-process data flow module.

[0120] Among them, the user management module can realize the management of various system users, permissions, resources and other modules, and provide a unified login entrance, so that users can maintain tasks at each stage. The data flow application module can be used to determine a data flow link information (i.e., determine the data source and reference program group) and the running date (i.e., the preset data source time), and record the corresponding data table set (i.e., the reference data table set) of the program job involved during the running process. The data flow acquisition module can obtain the migration verification data under the running date, and run the corresponding program in the old database of the production environment and the new database of the test environment respectively. The data flow comparison module can perform comparison based on the table data content obtained in the data flow acquisition module and the running time of the entire program to obtain the program running performance and program code status. The full-process data flow module can complete the full-process test verification of all programs under the entire batch framework in an automated manner through the acquisition and comparison of data flow.

[0121] The user management module provides a portal homepage for the unified login database migration verification system. Users can add, modify, discard, retrieve, and approve related tasks (including the data traffic application module, data traffic acquisition module, data traffic comparison module, and full-process data traffic module mentioned later), making it easier for users to maintain tasks at each stage.

[0122] The data flow application module is mainly used to determine a data flow link information and record the contents of each reference data table during the program operation. Figure 3B For example: through the temporary area program (i.e. the reference program of the temporary area) Lin1, Lin2, Lin3, Lin4 and Lin5, the migration verification data obtained from the corresponding data source is loaded into the temporary area, and the temporary area data tables (i.e. the reference data tables of the temporary area) Lin-DATA-1, Lin-DATA-2, Lin-DATA-3, Lin-DATA-4 and Lin-DATA-5 are obtained respectively; through the basic area program (i.e. the reference program of the basic area) Ji1, the data tables Lin-DATA-1, Lin-DATA-2, Lin-DATA-3, Lin-DATA-4 and Lin-DATA-5 are loaded into the temporary area. DATA-2, Lin-DATA-3, get the basic area data table (i.e. the reference data table of the basic area) Ji1-DATA-1; through the basic area program Ji2, run and load the temporary area data tables Lin-DATA-4 and Lin-DATA-5, get the basic area data table Ji2-DATA-1; through the summary area program (i.e. the reference program of the summary area) Hui1, load the basic area data tables Ji1-DATA-1 and Ji2-DATA-1, get the summary area data table (i.e. the reference data table of the summary area) Hui1-DATA-1. For details, see Figure 3CSchematic diagram of the dependency relationship between reference programs at different candidate layer levels.

[0123] For example, see Figure 3D The operation flow chart of the data traffic application module shown. Specifically, step one, create a data traffic application task. Step two, select the reference benchmark program that needs to be tested and verified (usually a program in the basic area or a program in the summary area). Step three, automatically parse the job content of the reference benchmark program to determine whether there is a pre-order data table that needs to be used (the reference program in the general temporary area does not have a pre-order program job); if so, record the reference pre-order program and the corresponding pre-order data table. Step four, based on the dependency configuration table (that is, a common application dependency parameter table), determine whether the reference benchmark program has a reference post-order program; if so, record the reference post-order program and the corresponding post-order data table. Step five, the traversal of the selected program job is completed. Step six, confirm whether all reference programs involved in the traversal process have been traversed; if not, repeat steps three to five for the reference programs that have not yet been traversed. For example, for Figure 3C A set of reference programs. If the reference benchmark program Ji1 is initially selected, after finding the reference predecessor programs Lin1, Lin2, and Lin3, continue to find the reference successor program Hui1. For the reference successor program Hui1, continue to traverse to find another reference predecessor program Ji2, and then find the reference predecessor program Lin4 and program Lin5. Step 7. After traversing all reference programs with previous and next dependencies, confirm the entire reference data flow framework, including the reference programs at each layer in the reference data flow framework and the corresponding reference data tables. Step 8. Enter the preset data source time, that is, the specific date for running the program on production. It can be a historical date before the current date or a future date after the current date. It should be noted that the historical date cannot be earlier than the cleanup date, and future dates need to wait. Step 9. This task is created.

[0124] For example, see Figure 3E The data flow acquisition module operation flow chart shown in the figure obtains the migration verification data of the preset data source time, and runs the relevant reference program for loading the old database in the production environment and the relevant verification program for the corresponding new database in the test environment respectively.

[0125] Specifically, step one, create a data traffic acquisition task. Step two, in response to the task selection instruction, select a data traffic application task and obtain the required preset data source time. If the preset data source time is a future date, it will automatically start after waiting for the corresponding date; if the preset data source time is a historical date, it will enter the traffic acquisition process. Step three, Figure 3CTaking the set of reference programs shown as an example, first obtain the migration verification data under the preset data source time: run and load the temporary area programs Lin1-Lin5 on the old production environment database to obtain the temporary area data tables Lin-DATA-1, Lin-DATA-2, Lin-DATA-3, Lin-DATA-4 and Lin-DATA-5, which are temporarily stored in the traffic library; based on the temporary area data tables Lin-DATA-1, Lin-DATA-2 and Lin-DATA-3, load the basic area program Ji1 to obtain the basic area data table Ji-DATA-1 corresponding to the basic area program; based on the temporary area data tables Lin-DATA-4 and Lin-DATA-5, load the basic area program Ji2 to obtain the basic area data table Ji-DATA-2 corresponding to the basic area program, and temporarily store it in the traffic library; based on the basic area data tables Ji-DATA-1 and Ji-DATA-2, load the summary area program Hui1 to obtain the summary area data table Hui-DATA-1; at the same time, record the reference running time of each reference program on the old production environment database.Step 4. On the other hand, use the migration verification data of the same preset data source time: run and load the new temporary area program (i.e. the program to be verified in the temporary area of ​​the new database) XLin1-XLin5 (which is applicable to the new database after migration or transformation and corresponds to the temporary area program Lin1-Lin5 of the old database) on the new database in the test environment, and obtain the new temporary area data tables (i.e. the data tables to be verified corresponding to the temporary area in the new database) XLin-DATA-1, XLin-DATA-2, XLin-DATA-3, XLin-DATA-4 and XLin-DATA-5, which are temporarily stored in the traffic library; based on the new temporary area data tables XLin-DATA-1, XLin-DATA-2 and XLin-DATA-3, run the new basic area program (i.e. the program to be verified in the basic area of ​​the new database) XJi1 (which is applicable to the new database after migration or transformation and corresponds to the old database) on the new database in the test environment. Based on the new temporary area data tables XLin-DATA-4 and XLin-DATA-5, the new basic area program XJi2 is run on the new database of the test environment to obtain the data table XJi-DATA-2 corresponding to the new basic area program, and temporarily store it in the traffic library; based on the data tables XJi-DATA-1 and XJi-DATA-2 to be verified in the basic area of ​​the new database, the new summary area program (i.e. the program to be verified in the summary area of ​​the new database) XHui1 (which is migrated or transformed and is suitable for the new database and corresponds to the old database summary area program Hui1) is run on the new database of the test environment to obtain the data table XHui-DATA-1 to be verified in the new summary area, and temporarily store it in the traffic library; at the same time, the running time of each program to be verified on the new database of the test environment is recorded. Step 5: Set the status of this data traffic acquisition task to completed.

[0126] For example, the data flow comparison module primarily compares the table data obtained by the aforementioned data flow acquisition module to draw data comparison conclusions. Furthermore, during program execution, runtime is also recorded for performance comparison to identify any performance issues. The task selection instruction can be used to select the current data flow application task from existing data flow application tasks.

[0127] It should be noted that during the data comparison process, the contents of the production data table are strictly isolated and cannot be displayed. Only when there are inconsistencies in the comparison can the relevant fields of the inconsistent related tables be displayed after data deformation.

[0128] For example, see Figure 3FThe data flow comparison module operation flow chart is shown in the figure. Specifically, step 1, create a data flow comparison task. Step 2, select a completed data flow acquisition task. Step 3, Figure 3C Taking the reference program shown as an example, the data processing results for each candidate area layer are compared. Specifically, the temporary area layer compares Lin-DATA-1 with XLin-DATA-1, Lin-DATA-2 with XLin-DATA-2, Lin-DATA-3 with XLin-DATA-3, Lin-DATA-4 with XLin-DATA-4, and Lin-DATA-5 with XLin-DATA-5; the basic area layer compares Ji-DATA-1 with XJi-DATA-1, and Ji-DATA-2 with XJi-DATA-2; and the summary area layer compares Hui-DATA-1 with HuiDATA-1. Data comparison primarily involves comparing data volume and data details. In the event of inconsistencies, the inconsistency is reviewed. In particular, for fields containing sensitive information, data distortion is applied during presentation. In step 4, if it is confirmed that a program job needs to be modified, after the program job is updated in the test environment, the original data traffic acquisition task can be reused to restart a new round of data traffic acquisition and perform table data comparison again.

[0129] Data volume comparison refers to comparing the reference data volume corresponding to a candidate verification combination with the data volume to be verified. Data detail comparison refers to comparing the reference data details corresponding to a candidate verification combination with the data details to be verified.

[0130] The application scenario of the above-mentioned module in the embodiment of the present invention can be to apply for, obtain, and compare data traffic based on the user's single-selection program operation to complete the test verification. In the full-process data traffic module, the test verification of the entire batch framework is completed in an automated manner through the acquisition and comparison of data traffic.

[0131] For example, Figure 3GThe following figure shows the operational flow chart for the full-process data flow module. Specifically, in step 1, create a full-process data flow task. In step 2, enter the preset data source time, which is the specific date used to run the program in production. This can be a historical date before the current date or a future date after the current date. Historical dates cannot be earlier than the cleanup date, and future dates require waiting. In step 3, associate the dependency configuration table to confirm the reference data tables corresponding to the reference programs at each candidate layer during the full-process batch run. Step 4: Start a full-process batch run. All reference programs (including the old database's temporary, basic, and summary area programs) are automatically loaded sequentially in the order of dependency on the old production database. The corresponding old data tables (including the old database's temporary, basic, and summary area data tables) are temporarily stored in the traffic library, and the reference runtime of each reference program is recorded. Simultaneously, all pending verification programs (including the new database's temporary, basic, and summary area programs) are automatically loaded sequentially in the order of dependency on the new test database. The corresponding new data tables (including the new database's temporary, basic, and summary area data tables) are temporarily stored in the traffic library, and the pending verification runtime of each pending verification program is recorded. Step 5: After the full process is completed, the data content of each data table (including data volume and data details) is automatically compared, and a list of inconsistencies is listed for users to review. Field content involving sensitive information will be distorted when displayed. Step 6: For the modified part of the program to be verified, you can continue to use the data traffic application module to test and verify the local program.

[0132] The embodiment of the present invention proposes a data flow system for rewriting, testing and verifying massive program operations during database transformation. In the test and verification of the currently common information-based innovation database transformation, it effectively improves the test efficiency, quickly locates the problem, saves a lot of time and manpower investment, lays a good foundation for the successful switching of the final database, ensures the high quality of database transformation, improves system stability, and brings higher benefits to the information-based innovation transformation work.

[0133] It should be noted that the information collected in the embodiments of the present invention is information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of relevant data comply with the relevant laws, regulations and standards of relevant countries and regions, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse; if the user chooses to refuse, the expert decision-making process will be entered.

[0134] Example 4

[0135] Figure 4This is a schematic diagram of the structure of a database migration verification device provided in Example 4 of the present invention. This embodiment is applicable to situations where a migrated database is to be verified. The method can be performed by a database migration verification device, which can be implemented using software and / or hardware and can be configured in an electronic device that carries the database migration verification function.

[0136] like Figure 4 As shown, the device includes: a program group determination module 310, a verification data determination module 320, a verification combination generation module 330, a verification data comparison module 340 and a migration verification module 350.

[0137] The program group determination module 310 is configured to obtain a preset data source time, determine the data source and a reference program group in the old database, and determine a program group to be verified in the new database based on the reference program group; wherein the reference program group is determined based on a reference benchmark program determined from a candidate region layer in the old database and a reference association configuration table;

[0138] a verification data determination module 320 configured to obtain migration verification data from the data source according to the preset data source time, determine a reference data table and reference runtime corresponding to each reference program in the reference program group based on the migration verification data, and determine a to-be-verified data table and to-be-verified runtime corresponding to each to-be-verified program in the to-be-verified program group based on the migration verification data;

[0139] A verification combination generating module 330 is configured to generate a candidate verification combination including a reference program and a program to be verified having the same program identifier;

[0140] A verification data comparison module 340 is configured to compare the reference runtime corresponding to the candidate verification combination with the runtime to be verified to determine the program running performance of the program to be verified in the candidate verification combination, and to compare the reference data table corresponding to the candidate verification combination with the data table to be verified to determine the program code status of the program to be verified in the candidate verification combination;

[0141] The migration verification module 350 is used to determine the program migration verification results of the programs to be verified in each candidate verification combination according to the program running performance and / or program code status corresponding to each candidate verification combination, and perform migration verification on the new database according to the program migration verification results.

[0142] An embodiment of the present invention provides a database migration verification solution, which determines a reference program group in a data source and an old database by obtaining a preset data source time, and determines a program group to be verified in a new database based on the reference program group; wherein the reference program group is determined based on a reference benchmark program and a reference associated configuration table determined from a candidate area layer of the old database; migration verification data is obtained from the data source according to the preset data source time, and based on the migration verification data, a reference data table and a reference running time corresponding to each reference program in the reference program group are determined, and based on the migration verification data, a data table and a reference running time corresponding to each program to be verified in the program group to be verified are determined. Running time; generating a candidate verification combination including a reference program and a program to be verified with the same program identifier; comparing the reference running time corresponding to the candidate verification combination with the running time to be verified, determining the program running performance of the program to be verified in the candidate verification combination, and comparing the reference data table corresponding to the candidate verification combination with the data table to be verified, determining the program code status of the program to be verified in the candidate verification combination; determining the program migration verification result of the program to be verified in each candidate verification combination according to the program running performance and / or program code status corresponding to each candidate verification combination, and performing migration verification on the new database according to the program migration verification result. The above scheme realizes the automation of database migration verification and improves the efficiency of database migration verification by determining the program migration verification result of the candidate verification combination according to the program running performance and program code status of the candidate verification combination, and then performing migration verification on the new database; at the same time, the embodiment of the present invention performs migration verification on the new database from two aspects of program running performance and program code status, thereby improving the accuracy of database migration verification.

[0143] Optionally, the program group determination module 310 includes:

[0144] a benchmark program parsing unit, configured to select a reference benchmark program from a candidate area layer of the old database, parse the execution of the reference benchmark program, and determine whether a preceding data table exists based on the parsing result; wherein the candidate area layer includes a temporary area, a basic area, and a summary area;

[0145] a predecessor program determining unit, configured to determine a reference predecessor program corresponding to the predecessor data table, if any;

[0146] a subsequent program determining unit, configured to determine whether the reference benchmark program has a reference subsequent program according to a reference association configuration table corresponding to the reference benchmark program;

[0147] a program group generating unit, configured to generate a reference program group including the reference base program, the reference predecessor program, and the reference successor program, if any;

[0148] The data source determining unit is configured to determine a data source according to the reference program in the reference program group that is in the temporary area.

[0149] Optionally, the device further includes:

[0150] The program traversal unit is configured to traverse each reference program in the reference program group and update the reference programs in the reference program group according to the traversal result.

[0151] Optionally, the verification data comparison module 340 includes:

[0152] a comparison data determination unit, configured to determine the amount of reference data and the details of the reference data in the reference data table corresponding to the candidate verification combination, and to determine the amount of data to be verified and the details of the data to be verified in the to-be-verified data table corresponding to the candidate verification combination;

[0153] a comparison result determining unit, configured to compare the reference data volume with the to-be-verified data volume to obtain a data volume comparison result, and to compare the reference data details with the to-be-verified data details to obtain a data detail comparison result;

[0154] The program code status determining unit is configured to determine the program code status of the to-be-verified program in the candidate verification combination according to the data volume comparison result and the data detail comparison result corresponding to the candidate verification combination.

[0155] Optionally, the device further includes:

[0156] an operation abnormality data determining unit, configured to determine, if the program code status of the program to be verified in the candidate verification combination is a program code abnormality, the operation abnormality data in the to-be-verified data table corresponding to the candidate verification combination based on a data volume comparison result and / or a data detail comparison result corresponding to the candidate verification combination;

[0157] The to-be-verified program updating unit is configured to update the to-be-verified program in the candidate verification combination according to the abnormal operation data.

[0158] Optionally, the verification data comparison module 340 includes:

[0159] a running time deviation value determining unit, configured to compare the reference running time corresponding to the candidate verification combination with the running time to be verified to obtain a running time deviation value;

[0160] The program running performance determining unit is configured to determine the program running performance of the program to be verified in the candidate verification combination according to the running time deviation value and a preset deviation threshold.

[0161] Optionally, the program execution performance determination unit is specifically configured to:

[0162] If the running time deviation value is greater than the preset deviation threshold, determining that the program running performance of the program to be verified in the candidate verification combination is abnormal;

[0163] If the running time deviation value is less than or equal to the preset deviation threshold, it is determined that the program running performance of the program to be verified in the candidate verification combination is normal.

[0164] The database migration verification device provided in the embodiment of the present invention can execute the database migration verification method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing each database migration verification method.

[0165] In the technical solution of the present invention, the collection, storage, use, processing, transmission, provision and disclosure of preset data source time and migration verification data, etc., all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0166] Example 5

[0167] Figure 5 4 is a structural diagram of an electronic device for implementing a database migration verification method provided by Example 5 of the present invention. The electronic device 410 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0168] like Figure 5As shown, the electronic device 410 includes at least one processor 411, and a memory connected to the at least one processor 411, such as a read-only memory (ROM) 412, a random access memory (RAM) 413, etc., wherein the memory stores a computer program that can be executed by the at least one processor, and the processor 411 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 412 or the computer program loaded from the storage unit 418 to the random access memory (RAM) 413. Various programs and data required for the operation of the electronic device 410 can also be stored in the RAM 413. The processor 411, ROM 412 and RAM 413 are connected to each other via a bus 414. An input / output (I / O) interface 415 is also connected to the bus 414.

[0169] Multiple components in electronic device 410 are connected to I / O interface 415, including an input unit 416, such as a keyboard, mouse, etc.; an output unit 417, such as various types of displays, speakers, etc.; a storage unit 418, such as a magnetic disk, optical disk, etc.; and a communication unit 419, such as a network card, modem, wireless communication transceiver, etc. The communication unit 419 allows electronic device 410 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0170] Processor 411 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of processor 411 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. Processor 411 executes the various methods and processes described above, such as the database migration verification method.

[0171] In some embodiments, the database migration verification method can be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as storage unit 418. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 410 via ROM 412 and / or communication unit 419. When the computer program is loaded into RAM 413 and executed by processor 411, one or more steps of the database migration verification method described above can be performed. Alternatively, in other embodiments, processor 411 can be configured to perform the database migration verification method in any other appropriate manner (e.g., via firmware).

[0172] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0173] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0174] In the context of the present invention, computer-readable storage media can be tangible media that can contain or store a computer program for use with an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. Computer-readable storage media can include but are not limited to electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, computer-readable storage media can be machine-readable signal media. More specific examples of machine-readable storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0175] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0176] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0177] A computing system may include clients and servers. The clients and servers are typically remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host. This server is a hosting product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosting and VPS services.

[0178] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved. This is not limited herein.

[0179] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A database migration verification method, characterized in that: include: Obtaining a preset data source time, determining the data source and a reference program group in the old database, and determining a program group to be verified in the new database based on the reference program group; wherein the reference program group is determined based on a reference benchmark program and a reference association configuration table determined from a candidate area layer of the old database; Obtaining migration verification data from the data source according to the preset data source time, determining a reference data table and a reference running time corresponding to each reference program in the reference program group based on the migration verification data, and determining a data table to be verified and a running time to be verified corresponding to each program to be verified in the program to be verified group based on the migration verification data; generating a candidate verification combination including a reference program and a program to be verified having the same program identifier; Comparing the reference running time corresponding to the candidate verification combination with the running time to be verified to determine the program running performance of the program to be verified in the candidate verification combination, and comparing the reference data table corresponding to the candidate verification combination with the data table to be verified to determine the program code status of the program to be verified in the candidate verification combination; According to the program running performance and / or program code status corresponding to each candidate verification combination, the program migration verification result of the program to be verified in each candidate verification combination is determined, and according to the program migration verification result, the new database is migrated and verified.

2. The method according to claim 1, characterized in that The determination of the data source and the reference program group in the old database includes: Selecting a reference benchmark program from a candidate area layer of the old database, parsing the execution of the reference benchmark program, and determining whether a preceding data table exists based on the parsing result; wherein the candidate area layer includes a temporary area, a basic area, and a summary area; If so, determining the reference preamble program corresponding to the preamble data table; Determining whether a reference subsequent program exists for the reference benchmark program according to a reference association configuration table corresponding to the reference benchmark program; If so, generating a reference program group including the reference base program, the reference predecessor program, and the reference successor program; A data source is determined according to the reference programs in the reference program group that are in the temporary area.

3. The method according to claim 2, characterized in that The method further comprises: Each reference program in the reference program group is traversed, and the reference programs in the reference program group are updated according to the traversal result.

4. The method according to claim 1, wherein The step of comparing the reference data table corresponding to the candidate verification combination with the to-be-verified data table to determine the program code status of the to-be-verified program in the candidate verification combination includes: Determine the reference data amount and reference data details in the reference data table corresponding to the candidate verification combination, and determine the to-be-verified data amount and to-be-verified data details in the to-be-verified data table corresponding to the candidate verification combination; Comparing the reference data volume with the data volume to be verified to obtain a data volume comparison result, and comparing the reference data details with the data details to be verified to obtain a data detail comparison result; The program code status of the program to be verified in the candidate verification combination is determined according to the data volume comparison result and the data detail comparison result corresponding to the candidate verification combination.

5. The method according to claim 4, characterized in that The method further comprises: If the program code status of the program to be verified in the candidate verification combination is a program code abnormality, determining the operation abnormality data in the to-be-verified data table corresponding to the candidate verification combination based on the data volume comparison result and / or data detail comparison result corresponding to the candidate verification combination; The programs to be verified in the candidate verification combination are updated according to the abnormal operation data.

6. The method according to claim 1, characterized in that The comparing the reference running time corresponding to the candidate verification combination with the running time to be verified to determine the program running performance of the program to be verified in the candidate verification combination includes: Comparing the reference running time corresponding to the candidate verification combination with the running time to be verified to obtain a running time deviation value; The program running performance of the program to be verified in the candidate verification combination is determined according to the running time deviation value and the preset deviation threshold.

7. The method according to claim 6, characterized in that The determining, based on the running time deviation value and the preset deviation threshold, the program running performance of the program to be verified in the candidate verification combination includes: If the running time deviation value is greater than the preset deviation threshold, determining that the program running performance of the program to be verified in the candidate verification combination is abnormal; If the running time deviation value is less than or equal to the preset deviation threshold, it is determined that the program running performance of the program to be verified in the candidate verification combination is normal.

8. A database migration verification device, characterized in that: include: a program group determination module, configured to obtain a preset data source time, determine the data source and a reference program group in the old database, and determine a program group to be verified in the new database based on the reference program group; wherein the reference program group is determined based on a reference benchmark program determined from a candidate region layer in the old database and a reference association configuration table; a verification data determination module, configured to obtain migration verification data from the data source according to the preset data source time, determine a reference data table and a reference running time corresponding to each reference program in the reference program group based on the migration verification data, and determine a to-be-verified data table and a to-be-verified running time corresponding to each to-be-verified program in the to-be-verified program group based on the migration verification data; A verification combination generating module, configured to generate a candidate verification combination including a reference program and a program to be verified having the same program identifier; a verification data comparison module, configured to compare the reference runtime corresponding to the candidate verification combination with the runtime to be verified to determine the program running performance of the program to be verified in the candidate verification combination, and to compare the reference data table corresponding to the candidate verification combination with the data table to be verified to determine the program code status of the program to be verified in the candidate verification combination; The migration verification module is used to determine the program migration verification results of the programs to be verified in each candidate verification combination according to the program running performance and / or program code status corresponding to each candidate verification combination, and perform migration verification on the new database according to the program migration verification results.

9. An electronic device, characterized in that: include: one or more processors; a memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement a database migration verification method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, a database migration verification method as described in any one of claims 1 to 7 is implemented.

Images