Database migration verification method and device, equipment and medium

Abstract

The embodiment of the invention discloses a database migration verification method and device, equipment and a medium, and relates to the technical field of big data. The method comprises the following steps: determining a data source and a reference program group, and determining a to-be-verified program group; obtaining migration verification data from the data source according to preset data source time, determining a reference data table and a reference running time length based on the migration verification data, and determining a to-be-verified data table and a to-be-verified running time length; generating a candidate verification combination; comparing the reference operation duration corresponding to the candidate verification combination with the to-be-verified operation duration to determine program operation performance, and comparing the reference data table corresponding to the candidate verification combination with the to-be-verified data table to determine a program code state; and determining a program migration verification result according to the program running performance and / or the program code state corresponding to each candidate verification combination, and performing migration verification on the new database according to the program migration verification result. And the efficiency and accuracy of database migration verification are improved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of big data technology, and in particular to a database migration verification method, device, equipment and medium. Background Art

[0002] With the continuous development of the current big data industry, in the future transformation of various databases, the migration of databases with massive data will face more challenges. The database transformation of big data applications (such as data warehouses) is a huge project. Due to the differences in structure and syntax of different types of databases, the database transformation process involves the transformation of thousands of program operations, which is a huge workload. Therefore, it is crucial to improve the accuracy and efficiency of database migration verification. Summary of the invention

[0003] The present invention provides a database migration verification method, device, equipment and medium to improve the accuracy and efficiency of database migration verification.

[0004] According to one aspect of the present invention, a database migration verification method is provided, comprising:

[0005] Acquire a preset data source time, determine a data source and a reference program group in an old database, and determine a program group to be verified in a new database based on the reference program group; wherein the reference program group is determined based on a reference benchmark program and a reference association configuration table determined from a candidate area layer of the old database;

[0006] Acquire migration verification data from the data source according to the preset data source time, determine a reference data table and a reference running time corresponding to each reference program in the reference program group based on the migration verification data, and determine a data table to be verified and a running time to be verified corresponding to each program to be verified in the program group to be verified based on the migration verification data;

[0007] generating a candidate verification combination including a reference program and a program to be verified having the same program identifier;

[0008] Compare the reference running time corresponding to the candidate verification combination with the running time to be verified to determine the program running performance of the program to be verified in the candidate verification combination, and compare the reference data table corresponding to the candidate verification combination with the data table to be verified to determine the program code status of the program to be verified in the candidate verification combination;

[0009] According to the program running performance and / or program code status corresponding to each of the candidate verification combinations, the program migration verification result of the program to be verified in each of the candidate verification combinations is determined, and according to the program migration verification result, the migration verification is performed on the new database.

[0010] According to another aspect of the present invention, there is provided a database migration verification device, comprising:

[0011] A program group determination module, used to obtain a preset data source time, determine a data source and a reference program group in an old database, and determine a program group to be verified in a new database based on the reference program group; wherein the reference program group is determined based on a reference benchmark program determined from a candidate area layer of the old database and a reference association configuration table;

[0012] a verification data determination module, configured to obtain migration verification data from the data source according to the preset data source time, determine a reference data table and a reference running time corresponding to each reference program in the reference program group based on the migration verification data, and determine a to-be-verified data table and a to-be-verified running time corresponding to each to-be-verified program in the to-be-verified program group based on the migration verification data;

[0013] A verification combination generation module, used to generate a candidate verification combination including a reference program and a program to be verified having the same program identifier;

[0014] A verification data comparison module is used to compare the reference running time corresponding to the candidate verification combination with the running time to be verified, to determine the program running performance of the program to be verified in the candidate verification combination, and to compare the reference data table corresponding to the candidate verification combination with the data table to be verified, to determine the program code status of the program to be verified in the candidate verification combination;

[0015] The migration verification module is used to determine the program migration verification results of the programs to be verified in each candidate verification combination according to the program running performance and / or program code status corresponding to each candidate verification combination, and perform migration verification on the new database according to the program migration verification results.

[0016] According to another aspect of the present invention, there is provided an electronic device, comprising:

[0017] one or more processors;

[0018] A memory for storing one or more programs;

[0019] When one or more programs are executed by one or more processors, the one or more processors can execute any one of the database migration verification methods provided by the embodiments of the present invention.

[0020] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement any one of the database migration verification methods provided in the embodiments of the present invention when executed.

[0021] The embodiment of the present invention provides a database migration verification scheme, which determines a reference program group in a data source and an old database by obtaining a preset data source time, and determines a program group to be verified in a new database based on the reference program group; wherein the reference program group is determined based on a reference benchmark program and a reference associated configuration table determined from a candidate area layer of the old database; migration verification data is obtained from the data source according to the preset data source time, and a reference data table and a reference running time corresponding to each reference program in the reference program group are determined based on the migration verification data, and a data table and a reference running time corresponding to each program to be verified in the program group to be verified are determined based on the migration verification data. Running time; generating a candidate verification combination including a reference program and a program to be verified with the same program identifier; comparing the reference running time corresponding to the candidate verification combination with the running time to be verified, determining the program running performance of the program to be verified in the candidate verification combination, and comparing the reference data table corresponding to the candidate verification combination with the data table to be verified, determining the program code status of the program to be verified in the candidate verification combination; determining the program migration verification result of the program to be verified in each candidate verification combination according to the program running performance and / or program code status corresponding to each candidate verification combination, and performing migration verification on the new database according to the program migration verification result. The above scheme realizes the automation of database migration verification and improves the efficiency of database migration verification by determining the program migration verification result of the candidate verification combination according to the program running performance and program code status of the candidate verification combination, and then performing migration verification on the new database; at the same time, the embodiment of the present invention performs migration verification on the new database from two aspects of program running performance and program code status, thereby improving the accuracy of database migration verification.

[0022] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 is a flowchart of a database migration verification method provided by Embodiment 1 of the present invention;

[0025] Figure 2 is a flowchart of a database migration verification method provided by Embodiment 2 of the present invention;

[0026] Figure 3A It is a schematic diagram of an overall data loading hierarchical framework of a data warehouse provided by the third embodiment of the present invention;

[0027] Figure 3B This is a data flow link diagram provided by Embodiment 3 of the present invention;

[0028] Figure 3C is a schematic diagram of dependency relationships between reference programs of different candidate area layers provided in Embodiment 3 of the present invention;

[0029] Figure 3D This is an operation flow chart of a data flow application module provided in Embodiment 3 of the present invention;

[0030] Figure 3E is an operation flow chart of a data flow acquisition module provided in Embodiment 3 of the present invention;

[0031] Figure 3F is an operation flow chart of a data flow comparison module provided in Embodiment 3 of the present invention;

[0032] Figure 3G This is an operation flow chart of a full-process data flow module provided in Embodiment 3 of the present invention;

[0033] Figure 4 It is a structural diagram of a database migration verification device provided by a fourth embodiment of the present invention;

[0034] Figure 5 It is a schematic diagram of the structure of an electronic device for implementing a database migration verification method provided in Embodiment 5 of the present invention. DETAILED DESCRIPTION

[0035] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0036] Embodiment 1

[0037] Figure 1 It is a flowchart of a database migration verification method provided in Example 1 of the present invention. This embodiment is applicable to the situation of verifying the migrated database. The method can be executed by a database migration verification device. The device can be implemented in software and / or hardware and can be configured in an electronic device that carries the database migration verification function.

[0038] See also Figure 1 The database migration verification method shown includes:

[0039] S110, obtaining a preset data source time, determining the data source and a reference program group in the old database, and determining a program group to be verified in the new database according to the reference program group.

[0040] The preset data source time refers to the time when data is obtained from the data source, that is, the data corresponding to the preset data source time is obtained from the data source. The data source can be used to provide migration verification data. Exemplarily, the data source can be different business systems. Migration verification data refers to data obtained from the data source for database migration verification.

[0041] The old database refers to the original database. Exemplarily, the old database may be an old data warehouse. The reference program group refers to a combination of reference programs participating in database migration verification. The reference program refers to a program in the old database participating in migration verification.

[0042] The new database refers to a database obtained by migrating the old database. Exemplarily, the new database may be a new data warehouse. The group of programs to be verified refers to a combination of programs to be verified that participate in database migration verification. Programs to be verified refer to programs in the new database that participate in migration verification.

[0043] Exemplarily, the reference program group is determined based on the reference benchmark program determined from the candidate area layer of the old database and the reference association configuration table. The candidate area layer refers to the structural area layer in the old database. The reference benchmark program can be used to determine other reference programs in the reference program group except the reference benchmark program. The reference association configuration table can also be called a dependency configuration table, which can be used to record the dependency relationship between various programs in the old database.

[0044] It should be noted that the reference programs in the reference program group correspond one-to-one to the programs to be verified in the program group to be verified.

[0045] S120. Obtain migration verification data from a data source according to a preset data source time, determine a reference data table and a reference running time corresponding to each reference program in the reference program group based on the migration verification data, and determine a data table to be verified and a running time to be verified corresponding to each program to be verified in the program group to be verified based on the migration verification data.

[0046] The reference data table refers to the running results of the reference program. The reference running time refers to the running time of the reference program. The data table to be verified refers to the running results of the program to be verified. The running time to be verified refers to the running time of the program to be verified.

[0047] S130: Generate a candidate verification combination including a reference program and a program to be verified having the same program identifier.

[0048] The program identifier can be used to uniquely represent the identity of the program. The candidate verification combination refers to a set of reference programs and programs to be verified that have the same program identifier. It should be noted that the reference programs and programs to be verified that have a corresponding relationship have the same program identifier.

[0049] S140. Compare the reference running time corresponding to the candidate verification combination with the running time to be verified to determine the program running performance of the program to be verified in the candidate verification combination, and compare the reference data table corresponding to the candidate verification combination with the data table to be verified to determine the program code status of the program to be verified in the candidate verification combination.

[0050] The program running performance may represent the running performance of the program to be verified. Exemplarily, the program running performance may include normal program performance and abnormal program performance. Normal program performance means that the running performance of the program to be verified is normal. Abnormal program performance means that the running performance of the program to be verified is abnormal.

[0051] The program code status may represent the code writing status in the program to be verified. Exemplarily, the program code status may include normal program code and abnormal program code. Normal program code means that the code writing in the program to be verified is accurate. Abnormal program code means that there are errors in the code writing in the program to be verified.

[0052] In an optional embodiment, the reference running time corresponding to the candidate verification combination is compared with the running time to be verified to determine the program running performance of the program to be verified in the candidate verification combination, including: comparing the reference running time corresponding to the candidate verification combination with the running time to be verified to obtain a running time deviation value; determining the program running performance of the program to be verified in the candidate verification combination based on the running time deviation value and a preset deviation threshold.

[0053] The running time deviation value can represent the difference between the reference running time and the running time to be verified. Exemplarily, the difference between the reference running time corresponding to the reference program in any candidate verification combination and the running time to be verified corresponding to the program to be verified in the candidate verification combination is used as the running time deviation value of the candidate verification combination.

[0054] The embodiment of the present invention does not impose any limitation on the size of the preset deviation threshold, which can be set by a technician based on experience or needs, or determined repeatedly through a large number of experiments.

[0055] It can be understood that by introducing the running time deviation value and the preset deviation threshold to determine the program running performance of the program to be verified, the accuracy and reliability of the determined program running performance are improved.

[0056] In an optional embodiment, the program running performance of the program to be verified in the candidate verification combination is determined based on the running time deviation value and the preset deviation threshold, including: if the running time deviation value is greater than the preset deviation threshold, then the program running performance of the program to be verified in the candidate verification combination is determined to be abnormal program performance; if the running time deviation value is less than or equal to the preset deviation threshold, then the program running performance of the program to be verified in the candidate verification combination is determined to be normal program performance.

[0057] Exemplarily, for any candidate verification combination, if the running time deviation value of the candidate verification combination is greater than the preset deviation threshold, then the program running performance of the program to be verified in the candidate verification combination is determined to be abnormal program performance; if the running time deviation value of the candidate verification combination is less than or equal to the preset deviation threshold, then the program running performance of the program to be verified in the candidate verification combination is determined to be normal program performance.

[0058] It can be understood that by specifically limiting the program running performance of the program to be verified in the candidate verification combination according to the preset deviation threshold and the running time deviation value, the accuracy of the determined program running performance is improved.

[0059] Exemplarily, for any candidate verification combination, if the program running performance of the program to be verified in the candidate verification combination is abnormal, the program to be verified in the candidate verification combination is optimized.

[0060] In an optional embodiment, the reference data table corresponding to the candidate verification combination and the data table to be verified are compared to determine the program code status of the program to be verified in the candidate verification combination, including: determining the reference data volume and reference data details in the reference data table corresponding to the candidate verification combination, and determining the reference data volume and data details to be verified in the data table to be verified corresponding to the candidate verification combination; comparing the reference data volume and the data volume to be verified to obtain a data volume comparison result, and comparing the reference data details and the data details to be verified to obtain a data detail comparison result; determining the program code status of the program to be verified in the candidate verification combination based on the data volume comparison result and the data detail comparison result corresponding to the candidate verification combination.

[0061] The reference data volume refers to the number of data in the reference data table. The reference data details refer to the field contents in the reference data table. The data volume to be verified refers to the number of data in the data table to be verified. The data details to be verified refer to the field contents in the data table to be verified.

[0062] The data volume comparison result refers to the result obtained by comparing the reference data volume corresponding to a candidate verification combination with the data volume to be verified. Exemplarily, the data volume comparison result may include normal data volume comparison and abnormal data volume comparison. Normal data volume comparison means that the reference data volume corresponding to a candidate verification combination is the same as the data volume to be verified. Abnormal data volume comparison means that the reference data volume corresponding to a candidate verification combination is different from the data volume to be verified.

[0063] Among them, the data detail comparison result refers to the result obtained by comparing the reference data details corresponding to a candidate verification combination with the data details to be verified. Exemplarily, the data detail comparison result may include normal data detail comparison and abnormal data detail comparison. Normal data detail comparison means that the reference data details corresponding to a candidate verification combination are the same as the data details to be verified. Abnormal data detail comparison means that the reference data details corresponding to a candidate verification combination are different from the data details to be verified.

[0064] Exemplarily, the program code status of the program to be verified in the candidate verification combination is determined based on the data volume comparison result and / or data detail comparison result corresponding to the candidate verification combination, including: for any candidate verification combination, if the data volume comparison result of the candidate verification combination is that the data volume comparison is abnormal, then the program code status of the program to be verified in the candidate verification combination is determined to be program code abnormal; if the data detail comparison result of the candidate verification combination is that the data detail comparison is abnormal, then the program code status of the program to be verified in the candidate verification combination is determined to be program code abnormal; if the data volume comparison result of the candidate verification program combination is that the data volume comparison is normal, and the data detail comparison result of the candidate verification program combination is that the data detail comparison is normal, then the program code status of the program to be verified in the candidate verification combination is determined to be program code normal.

[0065] It can be understood that by determining the program code status of the program to be verified in the candidate verification combination based on the quantity comparison results and the data detail comparison results, the program code status is determined from multiple angles, thereby improving the accuracy of the determined program code status.

[0066] In an optional embodiment, the method also includes: if the program code status of the program to be verified in the candidate verification combination is a program code exception, then determining the operation exception data in the data table to be verified corresponding to the candidate verification combination based on the data volume comparison results and / or data detail comparison results corresponding to the candidate verification combination; and updating the program to be verified in the candidate verification combination based on the operation exception data.

[0067] The abnormal operation data refers to abnormal data in the data table to be verified of the program to be verified.

[0068] Exemplarily, if the data volume comparison result of the candidate verification combination is a data volume comparison anomaly, the data corresponding to the data volume comparison anomaly is determined from the data table to be verified as at least part of the operation abnormality data; if the data detail comparison result of the candidate verification combination is a data detail comparison anomaly, the data corresponding to the data detail comparison anomaly is determined from the data table to be verified as at least part of the operation abnormality data.

[0069] For example, when comparing the data volume and data details of the reference data table and the data table to be verified corresponding to the candidate verification combination, the comparison is automatic inside the device; the device only displays the abnormal operation data, and when there is sensitive data in the abnormal operation data, the sensitive data in the abnormal operation data will be deformed first, and then the processed abnormal operation data will be displayed. The advantage is that the security of the data is improved.

[0070] Exemplarily, for any candidate verification combination, if the program code status of the program to be verified in the candidate verification combination is a program code exception, then based on the data volume comparison results and / or data detail comparison results corresponding to the candidate verification combination, the operation exception data is determined from the data table to be verified corresponding to the program to be verified in the candidate verification combination; based on the operation exception data, the program to be verified in the candidate verification combination is updated.

[0071] The abnormal operation data in the embodiment of the present invention can be understood as data in which there are differences between the reference data table corresponding to a candidate verification combination and the data table to be verified.

[0072] It can be understood that by updating the program to be verified based on the abnormal operation data, the accuracy of the program update is improved.

[0073] Specifically, for any candidate verification combination, the reference running time corresponding to the reference program in the candidate verification combination is compared with the running time to be verified corresponding to the program to be verified in the candidate verification combination to determine the program running performance of the program to be verified in the candidate verification combination; the reference data table corresponding to the reference program in the candidate verification combination is compared with the data table to be verified corresponding to the program to be verified in the candidate verification combination to determine the program code status of the program to be verified in the candidate verification combination.

[0074] S150. Determine the program migration verification result of the program to be verified in each candidate verification combination according to the program running performance and / or program code status corresponding to each candidate verification combination, and perform migration verification on the new database according to the program migration verification result.

[0075] Among them, the program migration verification result refers to the result obtained by performing migration verification on the program to be verified in the new database. Exemplarily, the program migration verification result may include program migration verification success and program migration verification failure. Program migration verification success means that there is no abnormality in the program to be verified, that is, it can be characterized that there is no abnormality in the migration of the program to be verified in the new database. Program migration verification failure means that there is an abnormality in the program to be verified, that is, it can be characterized that there is an abnormality in the migration of the program to be verified in the new database.

[0076] Exemplarily, for any candidate verification combination, if the program running performance corresponding to the candidate verification combination is normal program performance, and the program code status corresponding to the candidate verification combination is normal program code, then the program migration verification result of the program to be verified in the candidate verification combination is determined to be a successful program migration verification; if not, then the program migration verification result of the program to be verified in the candidate verification combination is determined to be a failed program migration verification.

[0077] Furthermore, if the program migration verification results corresponding to each candidate verification combination are all successful, then the new database migration verification is determined to be passed; if there is at least one program migration verification failure among the program migration verification results corresponding to each candidate verification combination, then the new database migration verification is determined to be unsuccessful.

[0078] Specifically, the program migration verification results of the programs to be verified in each candidate verification combination are determined according to the program running performance and / or program code status corresponding to each candidate verification combination, and the new database is migrated and verified according to the program migration verification results of each candidate verification combination.

[0079] The embodiment of the present invention provides a database migration verification scheme, which determines a reference program group in a data source and an old database by obtaining a preset data source time, and determines a program group to be verified in a new database based on the reference program group; wherein the reference program group is determined based on a reference benchmark program and a reference associated configuration table determined from a candidate area layer of the old database; migration verification data is obtained from the data source according to the preset data source time, and a reference data table and a reference running time corresponding to each reference program in the reference program group are determined based on the migration verification data, and a data table and a reference running time corresponding to each program to be verified in the program group to be verified are determined based on the migration verification data. Running time; generating a candidate verification combination including a reference program and a program to be verified with the same program identifier; comparing the reference running time corresponding to the candidate verification combination with the running time to be verified, determining the program running performance of the program to be verified in the candidate verification combination, and comparing the reference data table corresponding to the candidate verification combination with the data table to be verified, determining the program code status of the program to be verified in the candidate verification combination; determining the program migration verification result of the program to be verified in each candidate verification combination according to the program running performance and / or program code status corresponding to each candidate verification combination, and performing migration verification on the new database according to the program migration verification result. The above scheme realizes the automation of database migration verification and improves the efficiency of database migration verification by determining the program migration verification result of the candidate verification combination according to the program running performance and program code status of the candidate verification combination, and then performing migration verification on the new database; at the same time, the embodiment of the present invention performs migration verification on the new database from two aspects of program running performance and program code status, thereby improving the accuracy of database migration verification.

[0080] Embodiment 2

[0081] Figure 2 It is a flowchart of a database migration verification method provided by the second embodiment of the present invention. Based on the above embodiments, this embodiment further refines the operation of "determining the data source and the reference program group in the old database" into "selecting a reference benchmark program from the candidate area layer of the old database, parsing the execution of the reference benchmark program, and determining from the parsing result whether there is a predecessor data table; wherein the candidate area layer includes a temporary area, a basic area, and a summary area; if so, determining the reference predecessor program corresponding to the predecessor data table; determining whether there is a reference successor program for the reference benchmark program according to the reference association configuration table corresponding to the reference benchmark program; if so, generating a reference program group including a reference benchmark program, a reference predecessor program, and a reference successor program; determining the data source according to the reference program in the reference program group in the temporary area", so as to improve the determination mechanism of the data source and the reference program group in the old database. It should be noted that for the part not described in detail in the embodiment of the present invention, reference can be made to the description of other embodiments.

[0082] See also Figure 2 The database migration verification method shown includes:

[0083] S200: Obtain preset data source time.

[0084] S210, selecting a reference benchmark program from the candidate area layer of the old database, analyzing the execution status of the reference benchmark program, and determining whether a preceding data table exists from the analysis result.

[0085] The preceding data table may be understood as a reference data table corresponding to a reference program prior to the reference benchmark program.

[0086] Exemplarily, the candidate area layer includes a temporary area, a basic area, and a summary area. The reference program in the temporary area can obtain a reference data table corresponding to the reference program in the temporary area based on the migration verification data obtained from the data source. The reference program in the basic area can obtain a reference data table corresponding to the reference program in the basic area based on the reference data table in the temporary area. The reference program in the summary area can obtain a reference data table corresponding to the reference program in the summary area based on the reference data table in the basic area.

[0087] Exemplarily, a reference area layer is determined from candidate area layers of an old database; a reference benchmark program is selected from the reference area layer; the execution of the reference benchmark program is analyzed, and whether a preceding data table is required during the execution of the reference benchmark program is determined from the analysis result.

[0088] The reference region layer refers to a region layer that can provide a reference benchmark program. Exemplarily, the reference region layer can be a basic region or a summary region.

[0089] For example, if a reference benchmark program is selected from the basic area, the corresponding preceding data table includes the reference data table corresponding to the reference program in the temporary area; if a reference benchmark program is selected from the summary area, the corresponding preceding data table includes the reference data table corresponding to the reference program in the temporary area and the reference data table corresponding to the reference program in the summary area.

[0090] S220: If it exists, determine the reference preceding program corresponding to the preceding data table.

[0091] The reference predecessor program refers to a program that is associated with the reference base program and is executed before the reference base program.

[0092] Specifically, if there is a previous data table in the analysis result, it indicates that the execution of the reference benchmark program requires the participation of the reference data table of the previous candidate layer, and the corresponding reference previous program is determined according to the previous data table. It should be noted that there is a one-to-one correspondence between the program and the data table.

[0093] S230. Determine whether the reference benchmark program has a reference subsequent program according to the reference association configuration table corresponding to the reference benchmark program.

[0094] The reference subsequent program refers to a reference program that is associated with the reference base program and is executed after the reference base program.

[0095] Exemplarily, other programs associated with the reference benchmark program are searched in the reference association configuration table to determine whether there is a program executed after the reference benchmark program among the other programs, that is, whether there is a reference subsequent program.

[0096] Exemplarily, if the reference base program is a reference program in the basic area, the reference subsequent program may be a reference program in the summary area; if the reference base program is a reference program in the summary area, there is no reference subsequent program.

[0097] Specifically, it is determined whether the reference base program has a reference subsequent program according to the reference association configuration table of the old database. The reference association configuration table is a pre-set table used to record the dependency relationship between programs in the old database.

[0098] S240 . If so, generate a reference program group including a reference base program, a reference predecessor program, and a reference successor program.

[0099] Exemplarily, if the reference benchmark program is a reference program in a basic area, there is a reference subsequent program, and a reference program group including the reference benchmark program, the reference preceding program and the reference subsequent program is generated; if the reference benchmark program is a reference program in a summary area, there is no reference subsequent program, and a reference program group including the reference benchmark program and the reference preceding program is generated.

[0100] S250: Determine a data source according to the reference programs in the temporary area in the reference program group, and determine a program group to be verified in the new database according to the reference program group.

[0101] Exemplarily, there is a one-to-one correspondence between the reference program in the temporary area and the data source, and the corresponding data source can be determined according to the reference program in the temporary area.

[0102] S260. Obtain migration verification data from a data source according to a preset data source time, determine a reference data table and a reference running time corresponding to each reference program in the reference program group based on the migration verification data, and determine a data table to be verified and a running time to be verified corresponding to each program to be verified in the program group to be verified based on the migration verification data.

[0103] S270: Generate a candidate verification combination including a reference program and a program to be verified having the same program identifier.

[0104] S280. Compare the reference running time corresponding to the candidate verification combination with the running time to be verified to determine the program running performance of the program to be verified in the candidate verification combination, and compare the reference data table corresponding to the candidate verification combination with the data table to be verified to determine the program code status of the program to be verified in the candidate verification combination.

[0105] S290. Determine the program migration verification result of the program to be verified in each candidate verification combination according to the program running performance and / or program code status corresponding to each candidate verification combination, and perform migration verification on the new database according to the program migration verification result.

[0106] An embodiment of the present invention provides a database migration verification solution, which refines the operation of determining a data source and a reference program group in an old database into selecting a reference benchmark program from a candidate area layer of the old database, analyzing the execution status of the reference benchmark program, and determining from the analysis result whether a predecessor data table exists; wherein the candidate area layer includes a temporary area, a basic area, and a summary area; if so, determining a reference predecessor program corresponding to the predecessor data table; determining whether the reference benchmark program has a reference subsequent program based on a reference association configuration table corresponding to the reference benchmark program; if so, generating a reference program group including a reference benchmark program, a reference predecessor program, and a reference subsequent program; determining a data source based on a reference program in the temporary area in the reference program group, thereby improving a determination mechanism for a data source and a reference program group in the old database. The above scheme improves the accuracy of the reference program group by determining the reference predecessor program according to the analysis results of the reference benchmark program, and determining the reference subsequent program according to the reference association configuration table, thereby generating a reference program group including the reference benchmark program, the reference predecessor program and the reference subsequent program; at the same time, by analyzing the execution status of the reference benchmark program and determining from the analysis results whether there is a predecessor data table, if so, determining the reference predecessor program corresponding to the predecessor data table, it is possible to determine the reference predecessor program from the perspective of the actual operation status of the reference benchmark program, thereby improving the accuracy of the determined reference predecessor program.

[0107] Based on the above technical solution, an embodiment of the present invention further provides a method for updating a reference program in a reference program group. In an optional embodiment, the method further includes: traversing each reference program in the reference program group, and updating the reference program in the reference program group according to the traversal result.

[0108] The traversal result refers to the result obtained by traversing the reference programs in the reference program group.

[0109] Exemplarily, each reference program in the reference program group is executed sequentially to obtain a traversal result; if the traversal result is a normal traversal, there is no need to update the reference programs in the reference program group; if the traversal result is a traversal exception, the program exception data of the traversal exception is determined, and the reference programs in the reference program group are updated according to the program exception data.

[0110] Among them, normal traversal can be used to indicate that the reference program code in the reference program group is accurate and the program execution chain is complete. Abnormal traversal can be used to indicate that the reference program code in the reference program group has errors and / or the program execution chain is incomplete. An incomplete program execution chain means that at least one reference program is missing from the reference program group. A complete program execution chain means that there is no missing reference program in the reference program group.

[0111] The program abnormality data may indicate the cause of the traversal abnormality. For example, the program abnormality data may include a certain abnormality of a reference program in the reference program group, and a lack of a reference program in the reference program group.

[0112] In the embodiment of the present invention, since the contents recorded in the reference association configuration table may be incomplete, and thus the reference subsequent program determined according to the reference association configuration table may also be incomplete, it is necessary to traverse the reference programs in the reference program group and update the reference programs in the reference program group in a timely manner, thereby improving the comprehensiveness and accuracy of the reference programs in the reference program group.

[0113] Embodiment 3

[0114] The embodiment of the present invention provides an optional example based on the above embodiment. It should be noted that for the part not described in detail in the embodiment of the present invention, reference can be made to the description of other embodiments.

[0115] The development of the information and innovation industry has become the key to achieving digital transformation and promoting the development of the industrial chain. The existing technology has a process of migrating from the TeraData database to various distributed databases (including MppDB and Hadoop databases). In the future, the migration of various databases will also become a common project. For the post-testing and verification of database migration, it is necessary to confirm the result data after running each batch program to ensure the high consistency of the data. However, artificial number verification obviously cannot meet the test requirements, and production data cannot be used as test data.

[0116] The embodiment of the present invention proposes a framework for a database migration verification system for a large number of batch program job modifications involved in the database migration process. The database migration verification system is a closed production data flow system. On the old database in the production environment, a series of table data generated during the batch program operation is obtained. At the same time, the same production data source is used to run and load each modified new program job on the new database in the test environment to obtain a series of table data on the new database, and then the table data content on the old and new databases is compared to determine the correctness of the migration and modification of each program job (wherein, due to the confidentiality of production data, only when inconsistencies are found can the data details after partial deformation be displayed). Among them, the database migration verification system can be used to verify database migration.

[0117] The present invention takes a common big data application - data warehouse as an example to specifically illustrate the database migration verification method.

[0118] Figure 3A It shows the overall data loading hierarchical framework of the data warehouse: at the temporary area layer, each temporary area program job corresponds to a data source, and the corresponding temporary area data table is obtained after loading and running; at the basic area layer, each basic area program job will use one or more temporary area data tables or other basic area program jobs in the same layer to run the loaded data table, and then get the corresponding basic area data table after running and loading; at the summary area layer, each summary area program job will use one or more basic area data tables or other summary area program jobs in the same layer to run the loaded data table, and then get the corresponding summary area data table after running and loading.

[0119] The database migration verification system in the embodiment of the present invention is mainly divided into the following five modules: user management module, data flow application module, data flow acquisition module, data flow comparison module and full-process data flow module.

[0120] Among them, the user management module can realize the management of various system users, permissions, resources and other modules, and provide a unified login entrance, so that users can maintain tasks at each stage. The data flow application module can be used to determine a data flow link information (i.e. determine the data source and reference program group) and the running date (i.e. the preset data source time), and record the corresponding data table set (i.e. the reference data table set) of the program job involved during the running process. The data flow acquisition module can obtain the migration verification data under the running date, and run and load the corresponding program in the old database of the production environment and the new database of the test environment respectively. The data flow comparison module can compare the table data content obtained in the data flow acquisition module and the running time of the entire program to obtain the program running performance and program code status. The full-process data flow module can complete the full-process test verification of all programs under the entire batch framework in an automated manner through the acquisition and comparison of data flow.

[0121] Among them, the user management module provides a portal homepage for the database migration verification system with unified login. Users can add, modify, discard, retrieve, and approve related tasks (including the data flow application module, data flow acquisition module, data flow comparison module, and full-process data flow module mentioned later), allowing users to more conveniently maintain tasks at each stage.

[0122] The data flow application module is mainly used to determine the information of a data flow link and record the contents of each reference data table during the program running process. Figure 3B For example: the migration verification data obtained from the corresponding data source is loaded into the temporary area through the temporary area programs (i.e., the reference program of the temporary area) Lin1, Lin2, Lin3, Lin4 and Lin5 respectively, and the temporary area data tables (i.e., the reference data tables of the temporary area) Lin-DATA-1, Lin-DATA-2, Lin-DATA-3, Lin-DATA-4 and Lin-DATA-5 are obtained respectively; through the basic area program (i.e., the reference program of the basic area) Ji1, the temporary area data tables Lin-DATA-1, Lin-DATA-2, Lin-DATA-3, Lin-DATA-4 and Lin-DATA-5 are loaded. DATA-2, Lin-DATA-3, get the basic area data table (i.e. the reference data table of the basic area) Ji1-DATA-1; through the basic area program Ji2, run and load the temporary area data tables Lin-DATA-4 and Lin-DATA-5, get the basic area data table Ji2-DATA-1; through the summary area program (i.e. the reference program of the summary area) Hui1, load the basic area data tables Ji1-DATA-1 and Ji2-DATA-1, get the summary area data table (i.e. the reference data table of the summary area) Hui1-DATA-1. For details, see Figure 3CSchematic diagram of the dependency relationship between reference procedures at different candidate layer levels shown.

[0123] For example, see Figure 3D The operation flow chart of the data traffic application module shown. Specifically, step one, create a data traffic application task. Step two, select the reference benchmark program that needs to be tested and verified (usually a program in the basic area or a program in the summary area). Step three, automatically parse the job content of the reference benchmark program to determine whether there is a pre-order data table that needs to be used (the reference program in the general temporary area does not have a pre-order program job); if so, record the reference pre-order program and the corresponding pre-order data table. Step four, based on the dependency configuration table (that is, a common application dependency parameter table), determine whether the reference benchmark program has a reference post-order program; if so, record the reference post-order program and the corresponding post-order data table. Step five, the traversal of the selected program job is completed. Step six, confirm whether all reference programs involved in the traversal process have been traversed; if not, repeat steps three to five for reference programs that have not yet been traversed. For example, for Figure 3C A set of reference programs. If the reference benchmark program Ji1 is initially selected, after finding the reference predecessor programs Lin1, Lin2, and Lin3, continue to find the reference successor program Hui1. For the reference successor program Hui1, continue to traverse to find another reference predecessor program Ji2, and then find the reference predecessor program Lin4 and program Lin5. Step 7. After traversing all reference programs with forward and backward dependencies, confirm the entire reference data flow framework, including the reference programs at each layer in the reference data flow framework and the corresponding reference data tables. Step 8. Enter the preset data source time, that is, the specific date for running the program on production. It can be a historical date before the current date or a future date after the current date. It should be noted that the historical date cannot be earlier than the cleanup date, and future dates need to wait. Step 9. This task is created.

[0124] For example, see Figure 3E The operation flow chart of the data flow acquisition module shown in the figure obtains the migration verification data of the preset data source time, and respectively runs the relevant reference program for loading the old database of the production environment and the relevant verification program for the corresponding new database of the test environment.

[0125] Specifically, step 1, create a data traffic acquisition task. Step 2, in response to the task selection instruction, select a data traffic application task and obtain the required preset data source time. If the preset data source time is a future date, it will automatically start after the corresponding date; if the preset data source time is a historical date, it will enter the traffic acquisition process. Step 3, Figure 3CTaking a set of reference programs shown as an example, first obtain the migration verification data under the preset data source time: run and load the temporary area programs Lin1-Lin5 in the production on the old database of the production environment to obtain the temporary area data tables Lin-DATA-1, Lin-DATA-2, Lin-DATA-3, Lin-DATA-4 and Lin-DATA-5, which are temporarily stored in the traffic library; based on the temporary area data tables Lin-DATA-1, Lin-DATA-2 and Lin-DATA-3, load the basic area program Ji1 to obtain the basic area data table Ji-DATA-1 corresponding to the basic area program; based on the temporary area data tables Lin-DATA-4 and Lin-DATA-5, load the basic area program Ji2 to obtain the basic area data table Ji-DATA-2 corresponding to the basic area program, and temporarily store it in the traffic library; based on the basic area data tables Ji-DATA-1 and Ji-DATA-2, load the summary area program Hui1 to obtain the summary area data table Hui-DATA-1; at the same time, record the reference running time of each reference program on the old database of the production environment.Step 4. On the other hand, use the migration verification data with the same preset data source time: run and load the new temporary area program (i.e. the program to be verified in the temporary area of ​​the new database) XLin1-XLin5 (the migrated or modified data table applicable to the new database, corresponding to the temporary area program Lin1-Lin5 of the old database) on the new database in the test environment, and obtain the new temporary area data tables (i.e. the data tables to be verified corresponding to the temporary area in the new database) XLin-DATA-1, XLin-DATA-2, XLin-DATA-3, XLin-DATA-4 and XLin-DATA-5, which are temporarily stored in the traffic library; based on the new temporary area data tables XLin-DATA-1, XLin-DATA-2 and XLin-DATA-3, run the new basic area program (i.e. the program to be verified in the basic area of ​​the new database) XJi1 (the migrated or modified data table applicable to the new database, corresponding to the old database) on the new database in the test environment. Based on the new temporary area data tables XLin-DATA-4 and XLin-DATA-5, the new basic area program XJi2 is run on the new database of the test environment to obtain the data table XJi-DATA-2 corresponding to the new basic area program, and temporarily stored in the traffic library; based on the data tables XJi-DATA-1 and XJi-DATA-2 to be verified in the basic area of ​​the new database, the new summary area program (i.e. the program to be verified in the summary area of ​​the new database) XHui1 (which is suitable for the new database after migration or transformation and corresponds to the old database summary area program Hui1) is run on the new database of the test environment to obtain the data table XHui-DATA-1 to be verified in the new summary area, and temporarily stored in the traffic library; at the same time, the verification running time of each program to be verified on the new database of the test environment is recorded. Step 5: Set the status of this data flow acquisition task to completed.

[0126] Exemplarily, the data flow comparison module is mainly based on the table data content obtained in the above data flow acquisition module to compare and draw data comparison conclusions. On the other hand, during the program running process, the running time will also be recorded at the same time, and the performance will be compared to confirm whether there are performance problems. Among them, the task selection instruction can be used to select the data flow application task to be executed this time from the existing data flow application tasks.

[0127] It should be noted that during the data comparison process, the content of the production data table is strictly isolated and cannot be displayed. Only when there is inconsistency in the comparison can the relevant fields of the inconsistent related tables be displayed after data deformation.

[0128] For example, see Figure 3FThe operation flow chart of the data flow comparison module is shown in FIG. Specifically, step 1 is to create a data flow comparison task. Step 2 is to select a completed data flow acquisition task. Step 3 is to Figure 3C Taking the reference program shown as an example, the data processing results of each candidate area layer are compared, namely: the temporary area layer includes the data comparison of Lin-DATA-1 and XLin-DATA-1, Lin-DATA-2 and XLin-DATA-2, Lin-DATA-3 and XLin-DATA-3, Lin-DATA-4 and XLin-DATA-4, Lin-DATA-5 and XLin-DATA-5; the basic area layer includes the data comparison of Ji-DATA-1 and XJi-DATA-1, Ji-DATA-2 and XJi-DATA-2; the summary area layer includes the data comparison of Hui-DATA-1 and HuiDATA-1. Among them, the data comparison mainly includes the comparison of data volume and data details. Only in the case of inconsistency, the inconsistent content can be viewed, especially for the field content involving sensitive information, the data will be deformed when displayed. Step 4: If it is confirmed that a program job needs to be modified, after the program job is updated in the test environment, the original data flow acquisition task can be reused to restart a new round of data flow acquisition and perform table data comparison again.

[0129] The data volume comparison refers to comparing the reference data volume corresponding to a candidate verification combination with the data volume to be verified. The data detail comparison refers to comparing the reference data details corresponding to a candidate verification combination with the data details to be verified.

[0130] The application scenario of the above-mentioned module in the embodiment of the present invention can be to apply for, obtain, and compare data traffic based on the user's single-select program operation to complete the test verification. In the full-process data traffic module, the test verification of the entire process of the entire batch framework is completed in an automated manner by obtaining and comparing data traffic.

[0131] For example, Figure 3GThe operation flow chart of the full-process data flow module is shown. Specifically, step one, create a full-process data flow task. Step two, enter the preset data source time, that is, the specific date used to run the program on production. It can be a historical date before the current date or a future date after the current date. The historical date cannot be earlier than the cleanup date, and future dates need to wait. Step three, associate the dependency configuration table to confirm the reference data tables corresponding to the reference programs of each candidate area layer during the full-process batch running process. Step 4: Start the whole process batch run. On the old database in the production environment, all reference programs (including temporary area programs, basic area programs and summary area programs of the old database) are automatically loaded in sequence according to the dependency order. The corresponding old data tables (including temporary area data tables, basic area data tables and summary area data tables of the old database) are temporarily stored in the traffic library, and the reference running time of each reference program is recorded. At the same time, on the new database in the test environment, all programs to be verified (including temporary area programs, basic area programs and summary area programs of the new database) are automatically loaded in sequence according to the dependency order. The corresponding new data tables (including temporary area data tables, basic area data tables and summary area data tables of the new database) are temporarily stored in the traffic library, and the running time of each program to be verified is recorded. Step 5: After the whole process is completed, the data content of each data table (including data volume and data details) is automatically compared, and a list of inconsistencies is listed for users to view the inconsistent content. The field content involving sensitive information will be deformed when displayed. Step 6: For the modified part of the program to be verified, you can continue to use the data traffic application module to test and verify the local program.

[0132] The embodiment of the present invention proposes a data flow system for rewriting and testing massive program operations during database transformation. In the test and verification of the currently common information-based innovation database transformation, it effectively improves the test efficiency, quickly locates the problem, saves a lot of time and manpower investment, lays a good foundation for the successful switching of the final database, ensures the high quality of the database transformation, improves the system stability, and brings higher benefits to the information-based innovation transformation work.

[0133] It should be noted that the information collected in the embodiments of the present invention is information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of relevant data comply with the relevant laws, regulations and standards of relevant countries and regions, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse; if the user chooses to refuse, the expert decision-making process will be entered.

[0134] Embodiment 4

[0135] Figure 4This is a schematic diagram of the structure of a database migration verification device provided in Embodiment 4 of the present invention. This embodiment is applicable to the case of verifying a migrated database, and the method can be executed by a database migration verification device, which can be implemented in software and / or hardware, and can be configured in an electronic device that carries a database migration verification function.

[0136] like Figure 4 As shown, the device includes: a program group determination module 310, a verification data determination module 320, a verification combination generation module 330, a verification data comparison module 340 and a migration verification module 350. Among them,

[0137] The program group determination module 310 is used to obtain a preset data source time, determine the data source and the reference program group in the old database, and determine the program group to be verified in the new database based on the reference program group; wherein the reference program group is determined based on the reference benchmark program and the reference association configuration table determined from the candidate area layer of the old database;

[0138] A verification data determination module 320 is used to obtain migration verification data from the data source according to the preset data source time, determine the reference data table and reference running time corresponding to each reference program in the reference program group based on the migration verification data, and determine the to-be-verified data table and to-be-verified running time corresponding to each to-be-verified program in the to-be-verified program group based on the migration verification data;

[0139] A verification combination generating module 330, for generating a candidate verification combination including a reference program and a program to be verified having the same program identifier;

[0140] The verification data comparison module 340 is used to compare the reference running time corresponding to the candidate verification combination with the running time to be verified, determine the program running performance of the program to be verified in the candidate verification combination, and compare the reference data table corresponding to the candidate verification combination with the data table to be verified, and determine the program code status of the program to be verified in the candidate verification combination;

[0141] The migration verification module 350 is used to determine the program migration verification results of the programs to be verified in each candidate verification combination according to the program running performance and / or program code status corresponding to each candidate verification combination, and perform migration verification on the new database according to the program migration verification results.

[0142] The embodiment of the present invention provides a database migration verification scheme, which determines a reference program group in a data source and an old database by obtaining a preset data source time, and determines a program group to be verified in a new database based on the reference program group; wherein the reference program group is determined based on a reference benchmark program and a reference associated configuration table determined from a candidate area layer of the old database; migration verification data is obtained from the data source according to the preset data source time, and a reference data table and a reference running time corresponding to each reference program in the reference program group are determined based on the migration verification data, and a data table and a reference running time corresponding to each program to be verified in the program group to be verified are determined based on the migration verification data. Running time; generating a candidate verification combination including a reference program and a program to be verified with the same program identifier; comparing the reference running time corresponding to the candidate verification combination with the running time to be verified, determining the program running performance of the program to be verified in the candidate verification combination, and comparing the reference data table corresponding to the candidate verification combination with the data table to be verified, determining the program code status of the program to be verified in the candidate verification combination; determining the program migration verification result of the program to be verified in each candidate verification combination according to the program running performance and / or program code status corresponding to each candidate verification combination, and performing migration verification on the new database according to the program migration verification result. The above scheme realizes the automation of database migration verification and improves the efficiency of database migration verification by determining the program migration verification result of the candidate verification combination according to the program running performance and program code status of the candidate verification combination, and then performing migration verification on the new database; at the same time, the embodiment of the present invention performs migration verification on the new database from two aspects of program running performance and program code status, thereby improving the accuracy of database migration verification.

[0143] Optionally, the program group determination module 310 includes:

[0144] A benchmark program parsing unit, used to select a reference benchmark program from the candidate area layer of the old database, parse the execution status of the reference benchmark program, and determine whether a preceding data table exists from the parsing result; wherein the candidate area layer includes a temporary area, a basic area, and a summary area;

[0145] a predecessor program determining unit, configured to determine a reference predecessor program corresponding to the predecessor data table, if any;

[0146] a subsequent program determining unit, configured to determine whether the reference benchmark program has a reference subsequent program according to a reference association configuration table corresponding to the reference benchmark program;

[0147] a program group generating unit, for generating a reference program group including the reference base program, the reference predecessor program and the reference successor program, if any;

[0148] The data source determining unit is used to determine the data source according to the reference program in the reference program group that is in the temporary area.

[0149] Optionally, the device further comprises:

[0150] The program traversal unit is used to traverse each reference program in the reference program group and update the reference programs in the reference program group according to the traversal result.

[0151] Optionally, the verification data comparison module 340 includes:

[0152] A comparison data determination unit, used to determine the amount of reference data and the details of reference data in the reference data table corresponding to the candidate verification combination, and to determine the amount of data to be verified and the details of data to be verified in the to-be-verified data table corresponding to the candidate verification combination;

[0153] A comparison result determination unit, configured to compare the reference data volume with the to-be-verified data volume to obtain a data volume comparison result, and to compare the reference data details with the to-be-verified data details to obtain a data detail comparison result;

[0154] The program code status determination unit is used to determine the program code status of the to-be-verified program in the candidate verification combination according to the data volume comparison result and the data detail comparison result corresponding to the candidate verification combination.

[0155] Optionally, the device further comprises:

[0156] an operation abnormality data determining unit, configured to determine the operation abnormality data in the to-be-verified data table corresponding to the candidate verification combination according to the data volume comparison result and / or data detail comparison result corresponding to the candidate verification combination if the program code status of the to-be-verified program in the candidate verification combination is a program code abnormality;

[0157] The to-be-verified program updating unit is used to update the to-be-verified program in the candidate verification combination according to the abnormal operation data.

[0158] Optionally, the verification data comparison module 340 includes:

[0159] A running time deviation value determining unit, used for comparing the reference running time corresponding to the candidate verification combination with the running time to be verified to obtain a running time deviation value;

[0160] The program running performance determination unit is used to determine the program running performance of the program to be verified in the candidate verification combination according to the running time deviation value and the preset deviation threshold.

[0161] Optionally, the program running performance determination unit is specifically used to:

[0162] If the running time deviation value is greater than a preset deviation threshold, determining that the program running performance of the program to be verified in the candidate verification combination is abnormal;

[0163] If the running time deviation value is less than or equal to the preset deviation threshold, it is determined that the program running performance of the program to be verified in the candidate verification combination is normal.

[0164] The database migration verification device provided in the embodiment of the present invention can execute the database migration verification method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing each database migration verification method.

[0165] In the technical solution of the present invention, the collection, storage, use, processing, transmission, provision and disclosure of preset data source time and migration verification data, etc., all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0166] Embodiment 5

[0167] Figure 5 It is a structural diagram of an electronic device for implementing a database migration verification method provided by Embodiment 5 of the present invention. Electronic device 410 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0168] like Figure 5As shown, the electronic device 410 includes at least one processor 411, and a memory connected to the at least one processor 411 in communication, such as a read-only memory (ROM) 412, a random access memory (RAM) 413, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 411 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 412 or the computer program loaded from the storage unit 418 to the random access memory (RAM) 413. In the RAM 413, various programs and data required for the operation of the electronic device 410 can also be stored. The processor 411, the ROM 412, and the RAM 413 are connected to each other via a bus 414. An input / output (I / O) interface 415 is also connected to the bus 414.

[0169] Multiple components in the electronic device 410 are connected to the I / O interface 415, including: an input unit 416, such as a keyboard, a mouse, etc.; an output unit 417, such as various types of displays, speakers, etc.; a storage unit 418, such as a disk, an optical disk, etc.; and a communication unit 419, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 419 allows the electronic device 410 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0170] The processor 411 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 411 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 411 executes the various methods and processes described above, such as a database migration verification method.

[0171] In some embodiments, the database migration verification method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 418. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 410 via the ROM 412 and / or the communication unit 419. When the computer program is loaded into the RAM 413 and executed by the processor 411, one or more steps of the database migration verification method described above may be performed. Alternatively, in other embodiments, the processor 411 may be configured to perform the database migration verification method in any other appropriate manner (e.g., by means of firmware).

[0172] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0173] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0174] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0175] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0176] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0177] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0178] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0179] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A database migration verification method, characterized in that: include: Acquire a preset data source time, determine a data source and a reference program group in an old database, and determine a program group to be verified in a new database based on the reference program group; wherein the reference program group is determined based on a reference benchmark program and a reference association configuration table determined from a candidate area layer of the old database; Acquire migration verification data from the data source according to the preset data source time, determine a reference data table and a reference running time corresponding to each reference program in the reference program group based on the migration verification data, and determine a data table to be verified and a running time to be verified corresponding to each program to be verified in the program group to be verified based on the migration verification data; generating a candidate verification combination including a reference program and a program to be verified having the same program identifier; Compare the reference running time corresponding to the candidate verification combination with the running time to be verified to determine the program running performance of the program to be verified in the candidate verification combination, and compare the reference data table corresponding to the candidate verification combination with the data table to be verified to determine the program code status of the program to be verified in the candidate verification combination; According to the program running performance and / or program code status corresponding to each of the candidate verification combinations, the program migration verification result of the program to be verified in each of the candidate verification combinations is determined, and according to the program migration verification result, the migration verification is performed on the new database.

2. The method according to claim 1, characterized in that The determination of the reference program group in the data source and the old database includes: Selecting a reference benchmark program from the candidate area layer of the old database, parsing the execution of the reference benchmark program, and determining whether a preceding data table exists from the parsing result; wherein the candidate area layer includes a temporary area, a basic area, and a summary area; If so, determining the reference preamble program corresponding to the preamble data table; Determining whether there is a reference subsequent program for the reference benchmark program according to the reference association configuration table corresponding to the reference benchmark program; If so, generating a reference program group including the reference base program, the reference predecessor program and the reference successor program; A data source is determined according to the reference programs in the reference program group that are in the temporary area.

3. The method according to claim 2, characterized in that The method further comprises: Each reference program in the reference program group is traversed, and the reference programs in the reference program group are updated according to the traversal result.

4. The method according to claim 1, characterized in that: The step of comparing the reference data table corresponding to the candidate verification combination with the to-be-verified data table to determine the program code status of the to-be-verified program in the candidate verification combination includes: Determine the amount of reference data and the details of reference data in the reference data table corresponding to the candidate verification combination, and determine the amount of data to be verified and the details of data to be verified in the to-be-verified data table corresponding to the candidate verification combination; Compare the reference data volume with the data volume to be verified to obtain a data volume comparison result, and compare the reference data details with the data details to be verified to obtain a data detail comparison result; According to the data volume comparison result and the data detail comparison result corresponding to the candidate verification combination, the program code status of the program to be verified in the candidate verification combination is determined.

5. The method according to claim 4, characterized in that The method further comprises: If the program code status of the program to be verified in the candidate verification combination is a program code abnormality, determining the operation abnormality data in the to-be-verified data table corresponding to the candidate verification combination according to the data volume comparison result and / or data detail comparison result corresponding to the candidate verification combination; According to the abnormal operation data, the program to be verified in the candidate verification combination is updated.

6. The method according to claim 1, characterized in that The step of comparing the reference running time corresponding to the candidate verification combination with the running time to be verified to determine the program running performance of the program to be verified in the candidate verification combination includes: Compare the reference running time corresponding to the candidate verification combination with the running time to be verified to obtain a running time deviation value; The program running performance of the program to be verified in the candidate verification combination is determined according to the running time deviation value and the preset deviation threshold.

7. The method according to claim 6, characterized in that The determining, according to the running time deviation value and the preset deviation threshold, the program running performance of the program to be verified in the candidate verification combination includes: If the running time deviation value is greater than a preset deviation threshold, determining that the program running performance of the program to be verified in the candidate verification combination is abnormal; If the running time deviation value is less than or equal to the preset deviation threshold, it is determined that the program running performance of the program to be verified in the candidate verification combination is normal.

8. A database migration verification device, characterized in that: include: A program group determination module, used to obtain a preset data source time, determine a data source and a reference program group in an old database, and determine a program group to be verified in a new database based on the reference program group; wherein the reference program group is determined based on a reference benchmark program determined from a candidate area layer of the old database and a reference association configuration table; a verification data determination module, configured to obtain migration verification data from the data source according to the preset data source time, determine a reference data table and a reference running time corresponding to each reference program in the reference program group based on the migration verification data, and determine a to-be-verified data table and a to-be-verified running time corresponding to each to-be-verified program in the to-be-verified program group based on the migration verification data; A verification combination generation module, used to generate a candidate verification combination including a reference program and a program to be verified having the same program identifier; A verification data comparison module is used to compare the reference running time corresponding to the candidate verification combination with the running time to be verified, to determine the program running performance of the program to be verified in the candidate verification combination, and to compare the reference data table corresponding to the candidate verification combination with the data table to be verified, to determine the program code status of the program to be verified in the candidate verification combination; The migration verification module is used to determine the program migration verification results of the programs to be verified in each candidate verification combination according to the program running performance and / or program code status corresponding to each candidate verification combination, and perform migration verification on the new database according to the program migration verification results.

9. An electronic device, characterized in that: include: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement a database migration verification method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, a database migration verification method as described in any one of claims 1 to 7 is implemented.

Images