Data verification method, device and equipment based on heterogeneous data and medium

Abstract

The invention discloses a data verification method and device based on heterogeneous data, equipment and a medium. The invention relates to the technical field of artificial intelligence. The method comprises the steps of obtaining a data maintenance list, a data verification demand, a target database and a source database corresponding to a target data verification task; determining a target mapping rule in a preset mapping rule base based on the data verification demand; determining a target data table in a target database based on the data maintenance list and determining a source data table in a source database; based on a target mapping rule, performing mapping relation construction on the target data table to determine a target library mapping relation, and performing mapping relation construction on the source data table to determine a source library mapping relation; and performing difference comparison on the source library mapping relation and the target library mapping relation to obtain a data verification result. According to the technical scheme, the data problem can be found in time according to the characteristic difference of different databases, and the efficiency and accuracy of data verification are improved.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of artificial intelligence technology, and in particular to a data verification method, apparatus, device and medium based on heterogeneous data. Background Technology

[0002] In the field of information technology, with the implementation of projects such as system iteration, data warehouse construction, and cloud migration, data migration between databases has become a routine but crucial operation. After data migration is completed, ensuring the consistency, integrity, and accuracy of data between the source and target databases is the core criterion for measuring the success of the migration. Therefore, data verification is an indispensable and critical step.

[0003] Currently, the most common verification method in data migration practice is to sample or compare the data at both the source and target ends by writing specific query statements or simple scripts after the migration script has been executed. However, existing verification methods often involve developers or operations personnel manually triggering and executing the verification script, and then exporting inconsistent records as intermediate files. Staff then need to manually screen and verify these exported files to locate data discrepancies.

[0004] However, this semi-manual approach is not only time-consuming and labor-intensive, but its verification time cost increases exponentially when faced with massive amounts of data, severely slowing down project progress. Furthermore, manual comparison is highly susceptible to overlooking data differences due to visual fatigue or negligence, reducing the accuracy of data verification results. Therefore, how to automate, efficiently, and accurately execute the post-data migration verification process, promptly identify data problems, and improve the efficiency and accuracy of data verification is a pressing issue that needs to be addressed. Summary of the Invention

[0005] This invention provides a data verification method, apparatus, device, and medium based on heterogeneous data to solve the problem of low efficiency and accuracy in data verification during database migration.

[0006] In a first aspect, embodiments of the present invention provide a data verification method based on heterogeneous data, the method comprising:

[0007] Obtain the data maintenance list, data verification requirements, and database to be verified corresponding to the target data verification task; wherein, the database to be verified includes the target database and the source database;

[0008] Based on the data verification requirements, determine the target mapping rule corresponding to the target data verification task in the preset mapping rule base;

[0009] determine the target data table corresponding to the target data verification task in the target database based on the data maintenance list, and determine the source data table corresponding to the target data verification task in the source database based on the data maintenance list;

[0010] construct mapping relationship of the target data table based on the target mapping rule, determine the target library mapping relationship corresponding to the target database, and construct mapping relationship of the source data table based on the target mapping rule, determine the source library mapping relationship corresponding to the source database;

[0011] differentially compare the source library mapping relationship and the target library mapping relationship, and obtain the data verification result corresponding to the target data verification task.

[0012] In a second aspect, the embodiment of the present application further provides a data verification device based on heterogeneous data, which comprises:

[0013] a data acquisition module, configured to acquire a data maintenance list, a data verification requirement and a database to be verified corresponding to a target data verification task, wherein the database to be verified comprises a target database and a source database;

[0014] a rule determination module, configured to determine a target mapping rule corresponding to the target data verification task in a preset mapping rule library based on the data verification requirement;

[0015] a data table determination module, configured to determine the target data table corresponding to the target data verification task in the target database based on the data maintenance list, and determine the source data table corresponding to the target data verification task in the source database based on the data maintenance list;

[0016] a mapping determination module, configured to construct mapping relationship of the target data table based on the target mapping rule, determine the target library mapping relationship corresponding to the target database, and construct mapping relationship of the source data table based on the target mapping rule, determine the source library mapping relationship corresponding to the source database;

[0017] a data verification module, configured to differentially compare the source library mapping relationship and the target library mapping relationship, and obtain the data verification result corresponding to the target data verification task.

[0018] In a third aspect, the embodiment of the present application further provides an electronic device, comprising a memory, a processor and a computer program stored in the memory and capable of running on the processor, wherein the processor executes the program to realize the data verification method based on heterogeneous data as any of the embodiments of the present application.

[0019] In a fourth aspect, the embodiments of the present application further provide a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the data verification method based on heterogeneous data according to any one of the embodiments of the present application.

[0020] In a fifth aspect, the embodiments of the present application further provide a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the data verification method based on heterogeneous data according to any one of the embodiments of the present application.

[0021] In the embodiments of the present application, the data maintenance list corresponding to the target data verification task, the data verification requirement and the to-be-verified database are acquired, wherein the to-be-verified database comprises a target database and a source database. Then, the target mapping rule corresponding to the target data verification task is determined in the preset mapping rule library based on the data verification requirement. Meanwhile, the target data table corresponding to the target data verification task is determined in the target database based on the data maintenance list, and the source data table corresponding to the target data verification task is determined in the source database based on the data maintenance list. Further, the mapping relationship of the target data table is constructed based on the target mapping rule to determine the target library mapping relationship corresponding to the target database, and the mapping relationship of the source data table is constructed based on the target mapping rule to determine the source library mapping relationship corresponding to the source database. Finally, the data verification result corresponding to the target data verification task is obtained by comparing the source library mapping relationship and the target library mapping relationship. Since the mapping relationship is constructed in different data verification requirements according to the characteristic differences of different databases, the data comparison of the target data table and the source data table is performed through the mapping relationship, which can automatically, efficiently and accurately perform the verification process after data migration, discover data problems in time and improve the efficiency and accuracy of data verification. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0023] Figure 1 A flowchart of a data verification method based on heterogeneous data provided by the embodiments of the present application;

[0024] Figure 2 A flowchart of another data verification method based on heterogeneous data provided by the embodiments of the present application;

[0025] Figure 3 A flowchart of an optional data verification method based on heterogeneous data provided by the embodiments of the present application;

[0026] Figure 4 A structural schematic diagram of a data verification device based on heterogeneous data provided by an embodiment of the present application is provided.

[0027] Figure 5 A structural schematic diagram of an electronic device for implementing a data verification method based on heterogeneous data of an embodiment of the present application is provided. DETAILED DESCRIPTION

[0028] The present application will be further described below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely intended for the purpose of interpretation of the present application and are not limiting to the present application. In addition, it should be noted that, for the purpose of description, only the parts related to the present application are shown in the drawings and not all the structures.

[0029] It should be noted that: similar reference numerals and letters represent similar items in the following drawings, therefore, once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings. Meanwhile, in the description of the present application, the terms "first", "second" and the like are only used for distinguishing description and cannot be understood as indicating or implying relative importance. The acquisition, storage, use, processing and the like of data in the technical solution of the present application all comply with the relevant provisions of the national laws and regulations. It should be noted that, in the embodiments of the present application, some industry existing solutions, components, models and the like may be mentioned, which should be considered as exemplary, and the purpose is only to illustrate the feasibility of the implementation of the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the related content of the solution.

[0030] Figure 1 A flowchart of a data verification method based on heterogeneous data provided by an embodiment of the present application is provided, the present embodiment can be applicable to the case of data verification of database migration data after data migration is completed, the method can be executed by a data verification device based on heterogeneous data, which can be realized in the form of hardware and / or software, for example, can be configured in an electronic device. As shown in the figure, the method comprises: Figure 1

[0031] S101, obtaining a data maintenance list, data verification requirements and a to-be-verified database corresponding to a target data verification task; wherein the to-be-verified database comprises a target database and a source database.

[0032] ​The data check task can refer to a series of checking, comparing and verifying operations performed on the migrated data after the data is migrated from a source to a target. For example, the data check task can include the object to be checked and the data check method. Generally, the data check task can ensure that the migrated data is complete, accurate and consistent. The target data check task can refer to the selected data check task to be implemented. Generally, the target data check task can be determined according to the actual application requirement, and the embodiments of the present application do not make specific limitation on this. The database can refer to a data collection organized and stored according to a specific rule. Generally, the database has a strict structure and model to manage the data. Through the database, the application program can efficiently add, delete, modify and query the data. The data table can refer to the basic unit for actually storing the data of a specific subject or type in the database. Generally, the data table is a two-dimensional structure composed of rows and columns. The column is also called a field, which is used to define the type and attribute of the data. For example, a user table can have columns of user identity document (ID), name, age and email. The row is also called a record, which is used to store the complete information of an entity. For example, in the user table, a row represents all the information of a specific user. The data table name can refer to an identifier used to uniquely identify the data table. Generally, the data table name can accurately find and operate the corresponding data table and the stored data. It is worth noting that in the same database, each data table name must be unique. The data maintenance list can refer to a metadata list used to manage and track all the data tables in the database structure that need to be maintained. Generally, the data maintenance list can include the table names of each data table in the database. Through the data maintenance list, the data in the database can be continuously, effectively and correctly managed, preventing data confusion, redundancy, expiration or invalidation.

[0033] The data check requirement can refer to a predefined and required check item for ensuring data quality at a key node in a data life cycle, such as after data migration. For example, the data check requirement can include a full-amount check requirement, a statistical-amount check requirement, a structure check requirement, and a conditional check requirement. The full-amount check requirement can refer to a requirement for checking all data in a data table. Generally, the full-amount check requirement can ensure the integrity and absolute accuracy of data without any data loss or tampering. The statistical-amount check requirement can refer to a requirement for checking whether statistical indicators representing overall characteristics of data are consistent, without paying attention to each specific record. For example, the statistical-amount check requirement can be a requirement for checking the amount of data storage of a data table. Generally, the statistical-amount check requirement can quickly verify the overall consistency of data from a macro perspective, and is a highly efficient quality assurance method. The structure check requirement can refer to a requirement for checking whether the structure of a data table meets expectations, without paying attention to specific data in the structure. Generally, the structure check requirement can ensure that the structure of data is correct. The conditional check requirement can refer to a requirement for checking whether data content is reasonable and valid according to specific business logic and rules. Generally, the conditional check requirement can ensure the business correctness and logical consistency of data.

[0034] It is worth noting that in the embodiments of the present application, the conditional check requirement is generally applied to data check tasks related to financial data, and the full-amount check requirement is applied to data check tasks with small amounts of data. The statistical-amount check requirement and the structure check requirement can be used as general data check requirements and applied to various data check tasks.

[0035] The database to be checked can refer to a database that needs to participate in a data check task. Generally, the database to be checked can include a database that stores historical migration data and a database that stores current migration data. For example, the database to be checked can include a target database and a source database. The target database can refer to a target end to which migration data is transmitted. Generally, the target database is the end point of data flow. The source database can refer to the original source end of migration data. Generally, the source database is the starting point of data flow. For example, the source database can be a database used by a system that is running and continuously generating data.

[0036] S102, determining a target mapping rule corresponding to the target data check task in a preset mapping rule library based on the data check requirement.

[0037] The mapping rule can refer to a rule for defining the construction process of the mapping relationship. Generally, the mapping rule can include data objects and a mapping template required for constructing the mapping relationship. The preset mapping rule library can refer to a collection of mapping rules preset for storing various mapping rules. The target mapping rule can refer to a mapping rule matched with the data verification requirement in the preset mapping rule library.

[0038] S103, determining a target data table corresponding to the target data verification task in the target database based on the data maintenance list, and determining a source data table corresponding to the target data verification task in the source database based on the data maintenance list.

[0039] The target data table can refer to a data table stored in the target database and matched with the data maintenance list. The source data table can refer to a data table stored in the source database and matched with the data maintenance list.

[0040] S104, constructing a mapping relationship for the target data table based on the target mapping rule to determine a target library mapping relationship corresponding to the target database, and constructing a mapping relationship for the source data table based on the target mapping rule to determine a source library mapping relationship corresponding to the source database.

[0041] The mapping relationship can refer to a rule for defining the association relationship between the fields in the data table under the database. Generally, one data table can match different mapping relationships according to different data verification requirements. That is, the number of mapping relationships corresponding to the same data table can be one or multiple, which is not specifically limited by the embodiments of the present application. The source database mapping relationship can refer to the mapping relationship corresponding to the data table in the source database. Generally, for the same source database, the source database mapping relationship corresponding to the source database under different data verification requirements is different. For example, the source database mapping relationship can be stored in the form of a key-value pair. Optionally, the source database mapping relationship includes a first index and a first data value. The index can refer to a key for uniquely representing the properties of the data table. The first index can refer to the index corresponding to the source database under the data verification requirement. The field can refer to the representation of the title and rule of each column data in the data table. Generally, one column data in the same data table corresponds to one field, and the field is the most basic and important unit in the database. The field value can refer to the actual data content stored for a specific record in a specific field of the data table. The data value can refer to the specific field value corresponding to the index under different data verification requirements. The first data value can refer to the data value corresponding to the source database under the data verification requirement. The target database mapping relationship can refer to the mapping relationship corresponding to the data table in the target database. Generally, for the same target database, the target database mapping relationship corresponding to the target database under different data verification requirements is different. For example, the target database mapping relationship can be stored in the form of a key-value pair. Optionally, the target database mapping relationship includes a second index and a second data value. The second index can refer to the index corresponding to the target database under the data verification requirement. The second data value can refer to the data value corresponding to the target database under the data verification requirement.

[0042] Specifically, after obtaining the data maintenance list, the data verification requirement, and the database to be verified corresponding to the target data verification task, the target mapping rule corresponding to the target data verification task can be determined in the preset mapping rule library based on the data verification requirement. At the same time, the table name in the data maintenance list is read by using the data interface, and the connection with the source database and the target database is established. The target data table corresponding to the target data verification task is determined in the target database according to the table name in the data maintenance list, and the source data table corresponding to the target data verification task is determined in the source database according to the table name in the data maintenance list. Then, the target mapping rule is used to construct the mapping relationship of the target data table, to determine the target database mapping relationship corresponding to the target database, and the target mapping rule is used to construct the mapping relationship of the source data table, to determine the source database mapping relationship corresponding to the source database. Thus, different target database mapping relationships and source database mapping relationships are constructed through different data verification requirements, which improves the flexibility of data verification and provides an effective basis for subsequent data verification.

[0043] S105, difference comparison is carried out on the source library mapping relationship and the target library mapping relationship, and a data check result corresponding to the target data check task is obtained.

[0044] The difference comparison can refer to an operation of finding out a difference point existing between two mapping relationships by a systematic method. The data check result can refer to a comparison result obtained after the difference comparison. The data check result can be specific data content without difference information or with difference information.

[0045] In the embodiment of the application, the data maintenance list corresponding to the target data check task, the data check requirement and the database to be checked are obtained. The database to be checked includes the target database and the source database. Then, the target mapping rule corresponding to the target data check task is determined in the preset mapping rule library based on the data check requirement. Meanwhile, the target data table corresponding to the target data check task is determined in the target database based on the data maintenance list, and the source data table corresponding to the target data check task is determined in the source database based on the data maintenance list. Further, the mapping relationship of the target data table is constructed based on the target mapping rule, the target library mapping relationship corresponding to the target database is determined, and the mapping relationship of the source data table is constructed based on the target mapping rule, the source library mapping relationship corresponding to the source database is determined. Finally, the source library mapping relationship and the target library mapping relationship are difference compared, and the data check result corresponding to the target data check task is obtained. Since the mapping relationship is constructed in different data check requirement modes according to the characteristic difference of different databases, the data comparison of the target data table and the source data table is performed through the mapping relationship, the check process after the data migration can be automatically, efficiently and accurately performed, the data problem can be found in time, and the efficiency and accuracy of the data check are improved.

[0046] Figure 2A flowchart of another data verification method based on heterogeneous data provided by an embodiment of the present application is shown. The embodiment is refined based on the above-described embodiment. In the embodiment, the "mapping relationship construction of the target data table based on the target mapping rule, determination of the target database corresponding target library mapping relationship, mapping relationship construction of the source data table based on the target mapping rule, and determination of the source database corresponding source library mapping relationship" is refined. Specifically, it can include: mapping data extraction of the source data table based on the target mapping rule, determination of the first mapping data set corresponding to the source data table, and mapping data extraction of the target data table based on the first mapping data set and the target mapping rule, determination of the second mapping data set corresponding to the target data table; data filling of the first mapping data set based on the target mapping rule, obtaining the source database corresponding source library mapping relationship, and data filling of the second mapping data set based on the target mapping rule, obtaining the target database corresponding target library mapping relationship. As shown in Figure 2 the method includes:

[0047] S201, obtaining the data maintenance list, data verification requirement and to-be-verified database corresponding to the target data verification task; wherein the to-be-verified database includes a target database and a source database.

[0048] In an optional implementation, before the obtaining of the data maintenance list corresponding to the target data verification task, the method can further include: obtaining a basic data table name set corresponding to the target data verification task, and storing the basic data table name set based on a preset lightweight text to obtain the data maintenance list corresponding to the target data verification task.

[0049] The basic data table name can refer to the table name of the data table involved in the actual production business of the enterprise institution and required to be verified. Generally, the basic data table name can be determined according to actual application requirements, and the embodiment of the present application does not make specific limitation thereto. The basic data table name set can refer to a set composed of each basic data table name corresponding to the same enterprise institution. The lightweight text can refer to a text data with simple structure, simple format, no complex decoration and easy to be quickly read and processed by programs and devices. For example, the lightweight text can be a plain text file. The preset lightweight text can refer to a lightweight text preset for maintaining the data table name.

[0050] Specifically, before the target data verification task is implemented, the base data table name can be determined according to the data verification object contained in the target data verification task, that is, the data table name that needs to be verified, and a base data table name set is formed. Then, the base data table name set is stored by using the preset lightweight text to obtain the data maintenance list corresponding to the target data verification task. Thus, an effective basis is provided for the smooth execution of subsequent data verification tasks.

[0051] S202, determining the target mapping rule corresponding to the target data verification task in the preset mapping rule library based on the data verification requirement.

[0052] S203, determining the target data table corresponding to the target data verification task in the target database based on the data maintenance list, and determining the source data table corresponding to the target data verification task in the source database based on the data maintenance list.

[0053] S204, performing mapping data extraction on the source data table based on the target mapping rule to determine the first mapping data set corresponding to the source data table, and performing mapping data extraction on the target data table based on the first mapping data set and the target mapping rule to determine the second mapping data set corresponding to the target data table.

[0054] The mapping data can refer to the data content in the data table of the database that matches the data object set by the mapping rule. For example, the mapping data can be a field name or a specific field value. The first mapping data can refer to the mapping data in the source data table that matches the target mapping rule. The first mapping data set can refer to a set of first mapping data corresponding to the source data table under the same target mapping rule. The second mapping data can refer to the mapping data in the target data table that matches the target mapping rule. The second mapping data set can refer to a set of second mapping data corresponding to the target data table under the same target mapping rule.

[0055] In an optional embodiment, if the data verification requirement is a full-quantity verification requirement, the performing mapping data extraction on the source data table based on the target mapping rule to determine the first mapping data set corresponding to the source data table comprises: obtaining the source table primary key value and the source table residual field value set corresponding to the source data table as the first mapping data set corresponding to the source data table.

[0056] The primary key can refer to a column in a data table that uniquely identifies each record. Generally, the primary key has uniqueness, non-nullity, and unique identification. The uniqueness can refer to the primary key values of any two records in the entire table cannot be the same. The non-nullity can refer to the value of the primary key cannot be null, and each record must have a valid and specific primary key value. For example, taking a class roll as an example, the primary key can be the student ID. The primary key value can refer to the specific data stored in the primary key field. Generally, the primary key value is the actual content that realizes the function of the primary key. For example, continuing the above example, taking the student ID as the primary key, the primary key value can be the specific ID data. The source table can refer to a source data table. The source table primary key value can refer to the primary key value contained in the source data table. The residual field can refer to the remaining fields in the data table except the primary key. The residual field value can refer to the specific field value corresponding to the residual field. The source table residual field value can refer to the specific field value corresponding to the residual field in the source data table. The source table residual field value set can refer to a set composed of various source table residual field values in the same source data table.

[0057] Specifically, if the data verification requirement is a full-amount verification requirement, the source table primary key value and the source table residual field value set corresponding to the source data table can be obtained as the first mapping data set corresponding to the source data table. Thus, an effective basis is provided for subsequent operations.

[0058] In an optional implementation, if the data verification requirement is a statistical amount verification requirement, the mapping data extraction of the source data table based on the target mapping rule to determine the first mapping data set corresponding to the source data table includes: obtaining the source table name and the source table data amount corresponding to the source data table as the first mapping data set corresponding to the source data table.

[0059] The source table name can refer to the data table name corresponding to the source data table. The data amount can refer to how many data records are stored in a data table. For example, the data amount can be the number of row data contained in the data table. The source table data amount can refer to the data amount corresponding to the source data table.

[0060] Specifically, if the data verification requirement is a statistical amount verification requirement, the source table name and the source table data amount corresponding to the source data table can be obtained as the first mapping data set corresponding to the source data table. Thus, an effective basis is provided for subsequent operations.

[0061] In an optional implementation, if the data verification requirement is a structure verification requirement, the mapping data extraction of the source data table based on the target mapping rule to determine the first mapping data set corresponding to the source data table includes: obtaining the source table name and the source table field structure set corresponding to the source data table as the first mapping data set corresponding to the source data table.

[0062] The field structure can refer to planning and definition of all data columns in the data table. Generally, one field in the data table corresponds to one field structure, and the field structure can specify the name, type, and constraint of each column of data, and the like. For example, the field structure can be a combination of the field name, the field type, and the field length. The source table field structure can refer to the field structure corresponding to the source data table. The source table field structure set can refer to a set of source table field structures in the same source data table.

[0063] Specifically, if the data verification requirement is a structure verification requirement, the source table name and the source table field structure set corresponding to the source data table can be obtained as the first mapping data set corresponding to the source data table. Thus, an effective basis is provided for subsequent operations.

[0064] In an optional embodiment, if the data verification requirement is a conditional verification requirement, the first mapping data set corresponding to the source data table is determined by extracting mapping data of the source data table based on the target mapping rule, including: obtaining the source table conditional field and the source table conditional field value set corresponding to the source data table as the first mapping data set corresponding to the source data table.

[0065] The conditional field can refer to an important field selected for data verification. For example, the conditional field can be determined according to actual application requirements. Generally, the number of conditional fields can be one or more, and the embodiments of the present application do not make specific limitations thereto. The source table conditional field can refer to the conditional field corresponding to the source data table. The conditional field value can refer to the field value corresponding to the conditional field. For example, the conditional field value can be the actual data content stored in each specific record under the conditional field. The source table conditional field value can refer to the conditional field value corresponding to the source table conditional field. The source table conditional field value set can refer to a set of source table conditional field values corresponding to the same source data table.

[0066] Specifically, if the data verification requirement is a conditional verification requirement, the source table conditional field and the source table conditional field value set corresponding to the source data table are obtained as the first mapping data set corresponding to the source data table. Thus, an effective basis is provided for subsequent operations.

[0067] In an optional embodiment, the second mapping data set corresponding to the target data table is determined by extracting mapping data of the target data table based on the first mapping data set and the target mapping rule, including:

[0068] Step a1, extracting mapping data of the target data table based on the target mapping rule to determine the basic mapping data set corresponding to the target data table.

[0069] The basic mapping data can refer to mapping data in a target data table that is preliminarily obtained and that matches a target mapping rule.

[0070] Optionally, if the data verification requirement is a full-amount verification requirement, the extracting mapping data from the target data table based on the target mapping rule to determine the basic mapping data set corresponding to the target data table includes: obtaining a target table primary key value and a target table residual field value set corresponding to the target data table as the basic mapping data set corresponding to the target data table.

[0071] The target table can refer to a target data table. The target table primary key value can refer to a primary key value contained in the target data table. The target table residual field value can refer to a specific field value corresponding to a residual field in the target data table. The target table residual field value set can refer to a set of target table residual field values in the same target data table.

[0072] Specifically, if the data verification requirement is a full-amount verification requirement, the target table primary key value and the target table residual field value set corresponding to the target data table can be obtained as the basic mapping data set corresponding to the target data table. Thus, an effective basis is provided for subsequent operations.

[0073] Optionally, if the data verification requirement is a statistical amount verification requirement, the extracting mapping data from the target data table based on the target mapping rule to determine the basic mapping data set corresponding to the target data table includes: obtaining a target table name and a target table data amount corresponding to the target data table as the basic mapping data set corresponding to the target data table.

[0074] The target table name can refer to a data table name corresponding to the target data table. The target table data amount can refer to a data amount corresponding to the target data table.

[0075] Specifically, if the data verification requirement is a statistical amount verification requirement, the target table name and the target table data amount corresponding to the target data table can be obtained as the basic mapping data set corresponding to the target data table. Thus, an effective basis is provided for subsequent operations.

[0076] Optionally, if the data verification requirement is a structure verification requirement, the extracting mapping data from the target data table based on the target mapping rule to determine the basic mapping data set corresponding to the target data table includes: obtaining a target table name and a target table field structure set corresponding to the target data table as the basic mapping data set corresponding to the target data table.

[0077] The target table field structure can refer to a field structure corresponding to the target data table. The target table field structure set can refer to a set composed of each target table field structure in the same target data table.

[0078] Specifically, if the data verification requirement is a structure verification requirement, the target table name corresponding to the target data table and the target table field structure set can be obtained as the basic mapping data set corresponding to the target data table. Thus, an effective basis is provided for subsequent operations.

[0079] Optionally, if the data verification requirement is a conditional verification requirement, the target table condition field and the target table condition field value set corresponding to the target data table can be obtained as the basic mapping data set corresponding to the target data table.

[0080] The target table condition field can refer to a condition field corresponding to the target data table. The target table condition field value can refer to a condition field value corresponding to the target table condition field. The target table condition field value set can refer to a set composed of each target table condition field value corresponding to the same target data table.

[0081] Specifically, if the data verification requirement is a conditional verification requirement, the target table condition field and the target table condition field value set corresponding to the target data table can be obtained as the basic mapping data set corresponding to the target data table. Thus, an effective basis is provided for subsequent operations.

[0082] Step a2, type conversion is performed on the basic mapping data set based on the first mapping data set, to obtain a second mapping data set corresponding to the target data table.

[0083] The type conversion can refer to an operation of format conversion of the corresponding basic mapping data in the basic mapping data set according to the structure type of the first mapping data in the first mapping data set. For example, the structure type of the first mapping data in the first mapping data set is a high-precision numerical type, and the structure type of the corresponding basic mapping data in the basic mapping data set is a standard integer type. The standard integer structure type of the basic mapping data can be converted into the high-precision numerical type. The high-precision numerical type can refer to a data type specially used for storing accurate numerical values, and is particularly suitable for scenarios requiring accurate decimal calculation. Generally, the high-precision numerical type can store integers and decimals, and ensure the accuracy of calculation by defining the precision, i.e., the total number of bits, and the scale, i.e., the number of decimal places. The standard integer type can refer to a basic data type used for storing integer values in a programming language and a database system. Generally, the standard integer type can only store numbers without decimal parts, and each integer type has a specific value range. Thus, the different field types between the target database and the source database can be kept consistent, providing effective data support for subsequent operations.

[0084] Specifically, for the target data table, the mapping data extraction can be performed on the target data table based on the target mapping rule to obtain a basic mapping data set corresponding to the target data table. Then, the type conversion is performed on the basic mapping data at the corresponding position in the basic mapping data set according to the field type of each first mapping data in the first mapping data set. Thus, a second mapping data set corresponding to the target data table with consistent format types is obtained.

[0085] It should be noted that in the embodiments of the present application, the type conversion can also be performed on the first mapping data set based on the basic mapping data set, i.e., the type conversion is performed on the field type of the source data table in the source database according to the field type of the target data table in the target database. The above embodiments only preferentially perform the type conversion on the field type of the target data table in the target database according to the field type of the source data table in the source database, and the embodiments of the present application do not make specific limitations thereon.

[0086] S205, data filling is performed on the first mapping data set based on the target mapping rule to obtain a source database mapping relationship corresponding to the source database, and data filling is performed on the second mapping data set based on the target mapping rule to obtain a target database mapping relationship corresponding to the target database.

[0087] Exemplarily, taking a target mapping rule expressed as <k, v> as an example, k can be expressed as an index, and v can be expressed as a data value. If the data check requirement is a full-amount check requirement, the source table primary key value in the first mapping data set is a, and the source table residual field values are b, c, d, and e, k in the source-database mapping relationship can be expressed as the source table primary key value, that is, a, and v can be expressed as the value obtained by splicing the source table residual field values in the order of the residual field values, that is, bcde. Thus, the source-database mapping relationship in the form of <a, bcde> is formed.

[0088] If the data check requirement is a statistical-amount check requirement, the source table name in the first mapping data set is A, and the source table data amount is B, k in the source-database mapping relationship can be expressed as the source table name, that is, A, and v can be expressed as the source table data amount, that is, B. Thus, the source-database mapping relationship in the form of <A, B> is formed.

[0089] If the data check requirement is a structure check requirement, the source table name in the first mapping data set is A, and the source table field structure set is [{user identification, standard integer type, 11}, {user name, variable character type, 50}, {user mailbox, variable character type, 100}], k in the source-database mapping relationship can be expressed as the source table name, that is, A, and v can be expressed as the source table field structure set, that is, [{user identification, standard integer type, 11}, {user name, variable character type, 50}, {user mailbox, variable character type, 100}]. Thus, the source-database mapping relationship in the form of <A, [{user identification, standard integer type, 11}, {user name, variable character type, 50}, {user mailbox, variable character type, 100}]> is formed.

[0090] If the data check requirement is a condition check requirement, the source table condition field in the first mapping data set is F, and the source table condition field value set is {f1, f2, f3, f4}, k in the source-database mapping relationship can be expressed as the source table condition field, that is, F, and v can be expressed as the sum of each source table condition field value in the source table condition field value set, that is, f1+f2+f3+f4. Thus, the source-database mapping relationship in the form of <F, f1+f2+f3+f4> is formed.

[0091] It is worth noting that in the embodiment of the present application, the processing procedure of "filling data in the second mapping data set based on the target mapping rule to obtain the target-database corresponding target-database mapping relationship" is consistent with the construction procedure of the source-database mapping relationship, and the embodiment of the present application does not perform additional elaboration.

[0092] Specifically, after the target data table and the source data table corresponding to the target data check task are determined, the source data table can be mapped and data extracted according to the data content contained in the target mapping rule to determine a first mapping data set corresponding to the source data table. Then, the target data table is mapped and data extracted by using the first mapping data set and the target mapping rule to determine a second mapping data set corresponding to the target data table. Further, the first mapping data set is filled with data according to the target mapping rule to obtain a source database mapping relationship corresponding to the source database, and the second mapping data set is filled with data according to the target mapping rule to obtain a target database mapping relationship corresponding to the target database. Thus, the source database mapping relationship and the target database mapping relationship constructed are obtained, which provides an effective basis for subsequent data check.

[0093] It is worth noting that in the embodiments of the present application, the construction process of the source database mapping relationship and the target database mapping relationship can be sequentially executed, or can be executed in parallel after the first mapping data set is determined, thereby improving the data processing rate, and the embodiments of the present application do not make specific limitation on this.

[0094] S206, the first index and the second index are compared to obtain an index comparison result.

[0095] The index comparison result can refer to a comparison result obtained by comparing the index in the source database mapping relationship with the index in the target database mapping relationship in the same target data check task. For example, the index comparison result can be consistent index values, or inconsistent index values.

[0096] S207, the first data value and the second data value are compared based on the index comparison result to obtain a data check result corresponding to the target data check task.

[0097] For example, if the index comparison result is consistent index values, the data value in the source database mapping relationship and the data value in the target database mapping relationship are compared to determine whether the data values on both ends are consistent. If the data values on both ends are consistent, a data check result without difference can be obtained. Conversely, if the data values on both ends are inconsistent, the data with difference in the source database mapping relationship can be taken as the data check result.

[0098] Specifically, after obtaining the target library mapping relationship corresponding to the target database and the source library mapping relationship corresponding to the source database, the first index in the source library mapping relationship and the second index in the target library mapping relationship can be obtained first, and the first index and the second index are compared in difference to obtain an index comparison result. If the index comparison result is that the index values are consistent, the first data value in the source library mapping relationship and the second data value in the target library mapping relationship are obtained, and the first data value and the second data value are compared in difference to obtain a data check result corresponding to the target data check task. Thus, the data check task is completed.

[0099] Notably, in the embodiment of the application, after obtaining the data check result corresponding to the target data check task, the data check result can be sent to the user end for visual display, so that the staff can analyze according to the data check result and improve the data quality of the migrated data.

[0100] In the embodiment of the application, the data maintenance list corresponding to the target data check task, the data check requirement and the database to be checked are obtained; wherein the database to be checked includes the target database and the source database. Then, the target mapping rule corresponding to the target data check task is determined in the preset mapping rule library based on the data check requirement. At the same time, the target data table corresponding to the target data check task is determined in the target database based on the data maintenance list, and the source data table corresponding to the target data check task is determined in the source database based on the data maintenance list. Further, the first mapping data set corresponding to the source data table is determined by mapping data extraction on the source data table based on the target mapping rule, and the second mapping data set corresponding to the target data table is determined by mapping data extraction on the target data table based on the first mapping data set and the target mapping rule. Further, the source library mapping relationship corresponding to the source database is obtained by data filling on the first mapping data set based on the target mapping rule, and the target library mapping relationship corresponding to the target database is obtained by data filling on the second mapping data set based on the target mapping rule. Finally, the index comparison result is obtained by comparing the first index and the second index in difference. And the data check result corresponding to the target data check task is obtained by comparing the first data value and the second data value in difference based on the index comparison result. Since the mapping relationship is constructed in different data check requirements according to the characteristics of different databases, the data comparison between the target data table and the source data table is performed through the mapping relationship, which can automatically, efficiently and accurately execute the check process after data migration, discover data problems in time, and improve the efficiency and accuracy of data check.

[0101] Figure 3An optional data verification method based on heterogeneous data provided by the embodiment of the present application is provided. Specifically, first, the data verification object contained in the target data verification task, i.e., the data table name that needs to be verified, is determined to determine the basic data table name, and a basic data table name set is formed. Then, the basic data table name set is stored by using a preset lightweight text to obtain a data maintenance list corresponding to the target data verification task. Further, the data maintenance list is read through a data interface to determine the data table name that needs to be verified, and a data connection is established with the target database and the source database to obtain the data table in the target database consistent with the data maintenance list as the target data table corresponding to the target data verification task, and obtain the data table in the source database consistent with the data maintenance list as the source data table corresponding to the target data verification task. At the same time, the target mapping rule corresponding to the target data verification task is determined in the preset mapping rule library based on the data verification requirement. If the data verification requirement is a full-quantity verification requirement, the source table primary key value and the source table residual field value set corresponding to the source data table are obtained as the first mapping data set corresponding to the source data table, and the target table primary key value and the target table residual field value set corresponding to the target data table are obtained as the basic mapping data set corresponding to the target data table. If the data verification requirement is a statistical quantity verification requirement, the source table name and the source table data quantity corresponding to the source data table are obtained as the first mapping data set corresponding to the source data table, and the target table name and the target table data quantity corresponding to the target data table are obtained as the basic mapping data set corresponding to the target data table. If the data verification requirement is a structure verification requirement, the source table name and the source table field structure set corresponding to the source data table are obtained as the first mapping data set corresponding to the source data table, and the target table name and the target table field structure set corresponding to the target data table are obtained as the basic mapping data set corresponding to the target data table. If the data verification requirement is a conditional verification requirement, the source table condition field and the source table condition field value set corresponding to the source data table are obtained as the first mapping data set corresponding to the source data table, and the target table condition field and the target table condition field value set corresponding to the target data table are obtained as the basic mapping data set corresponding to the target data table. Further, the type conversion is performed on the basic mapping data set based on the first mapping data set to obtain the second mapping data set corresponding to the target data table. Further, the data filling is performed on the first mapping data set based on the target mapping rule to obtain the source database mapping relationship corresponding to the source database, and the data filling is performed on the second mapping data set based on the target mapping rule to obtain the target database mapping relationship corresponding to the target database. Further, the first index in the source database mapping relationship and the second index in the target database mapping relationship are compared in difference to obtain an index comparison result. When the index comparison result is the index value consistent, the first data value in the source database mapping relationship and the second data value in the target database mapping relationship are compared in difference to obtain the data verification result corresponding to the target data verification task.Finally, the data check result corresponding to the target data check task is saved persistently, and the data check result is analyzed differentially to obtain a migration data quality improvement scheme.

[0102] Figure 4 A structure diagram of a data check device based on heterogeneous data is provided for the embodiments of the present application, and the device can execute the data check method based on heterogeneous data involved in each of the embodiments. The device can be realized in the form of software and / or hardware. Figure 4 As shown in the figure, the data check device based on heterogeneous data specifically includes a data acquisition module 310, a rule determination module 320, a data table determination module 330, a mapping determination module 340 and a data check module 350.

[0103] The data acquisition module 310 is configured to acquire a data maintenance list, a data check requirement and a database to be checked corresponding to a target data check task, wherein the database to be checked includes a target database and a source database.

[0104] The rule determination module 320 is configured to determine a target mapping rule corresponding to the target data check task in a preset mapping rule library based on the data check requirement.

[0105] The data table determination module 330 is configured to determine a target data table corresponding to the target data check task in the target database based on the data maintenance list, and determine a source data table corresponding to the target data check task in the source database based on the data maintenance list.

[0106] The mapping determination module 340 is configured to construct a mapping relationship of the target data table based on the target mapping rule, determine a target library mapping relationship corresponding to the target database, and construct a mapping relationship of the source data table based on the target mapping rule, determine a source library mapping relationship corresponding to the source database.

[0107] The data check module 350 is configured to differentially compare the source library mapping relationship and the target library mapping relationship to obtain a data check result corresponding to the target data check task.

[0108] In the embodiment of the present application, the data maintenance list corresponding to the target data verification task, the data verification requirement and the to-be-verified database are acquired, wherein the to-be-verified database comprises a target database and a source database. Then, the target mapping rule corresponding to the target data verification task is determined in the preset mapping rule library based on the data verification requirement. Meanwhile, the target data table corresponding to the target data verification task is determined in the target database based on the data maintenance list, and the source data table corresponding to the target data verification task is determined in the source database based on the data maintenance list. Further, the mapping relationship of the target data table is constructed based on the target mapping rule, the target database corresponding target library mapping relationship is determined, and the mapping relationship of the source data table is constructed based on the target mapping rule, the source database corresponding source library mapping relationship is determined. Finally, the data verification result corresponding to the target data verification task is obtained by comparing the source library mapping relationship and the target library mapping relationship. Since the mapping relationship is constructed in different data verification requirements according to the characteristic difference of different databases, the data comparison of the target data table and the source data table is performed through the mapping relationship, the data verification process after data migration can be automatically, efficiently and accurately performed, the data problem can be found in time, and the efficiency and accuracy of data verification are improved.

[0109] Optionally, the data verification device based on heterogeneous data can further comprise: a list construction module, configured to acquire a basic data table name set corresponding to the target data verification task before acquiring the data maintenance list corresponding to the target data verification task, and perform data storage on the basic data table name set based on a preset lightweight text to obtain the data maintenance list corresponding to the target data verification task.

[0110] Optionally, the mapping determination module 340 can specifically comprise: a mapping data determination unit and a mapping relationship determination unit.

[0111] The mapping data determination unit is configured to extract mapping data from the source data table based on the target mapping rule to determine a first mapping data set corresponding to the source data table, and extract mapping data from the target data table based on the first mapping data set and the target mapping rule to determine a second mapping data set corresponding to the target data table.

[0112] The mapping relationship determination unit is configured to fill data in the first mapping data set based on the target mapping rule to obtain the source library mapping relationship corresponding to the source database, and fill data in the second mapping data set based on the target mapping rule to obtain the target library mapping relationship corresponding to the target database.

[0113] Optionally, the data verification requirement comprises: full-amount verification requirement, statistical-amount verification requirement, structure verification requirement and conditional verification requirement.

[0114] Optionally, if the data verification requirement is a full-amount verification requirement, the mapping data determination unit can be specifically configured to: acquire a source table primary key value and a source table residual field value set corresponding to the source data table as a first mapping data set corresponding to the source data table.

[0115] Optionally, if the data verification requirement is a statistical-amount verification requirement, the mapping data determination unit can be specifically configured to: acquire a source table name and a source table data amount corresponding to the source data table as the first mapping data set corresponding to the source data table.

[0116] Optionally, if the data verification requirement is a structure verification requirement, the mapping data determination unit can be specifically configured to: acquire a source table name and a source table field structure set corresponding to the source data table as the first mapping data set corresponding to the source data table.

[0117] Optionally, if the data verification requirement is a condition verification requirement, the mapping data determination unit can be specifically configured to: acquire a source table condition field and a source table condition field value set corresponding to the source data table as the first mapping data set corresponding to the source data table.

[0118] Optionally, the mapping data determination unit can be specifically configured to:

[0119] extract mapping data from the target data table based on the target mapping rule to determine a basic mapping data set corresponding to the target data table;

[0120] perform type conversion on the basic mapping data set based on the first mapping data set to obtain a second mapping data set corresponding to the target data table.

[0121] Optionally, the source library mapping relationship includes a first index and a first data value, and the target library mapping relationship includes a second index and a second data value.

[0122] Optionally, the data verification module 350 can be specifically configured to:

[0123] differentially compare the first index and the second index to obtain an index comparison result;

[0124] differentially compare the first data value and the second data value based on the index comparison result to obtain a data verification result corresponding to the target data verification task.

[0125] The data verification device based on heterogeneous data provided in the embodiments of the present application can execute the data verification method based on heterogeneous data provided in any of the embodiments of the present application, and has the corresponding function modules and beneficial effects of the execution method.

[0126] Figure 5A structural schematic diagram of an electronic device provided by an embodiment of the present application is shown in Figure 5 The electronic device includes a processor 410, a memory 420, an input device 430, and an output device 440; the number of processors 410 in the electronic device can be one or more, Figure 5 and one processor 410 is taken as an example in the embodiment; the processor 410, the memory 420, the input device 430, and the output device 440 in the electronic device can be connected through a bus or other means, Figure 5 and the connection through the bus is taken as an example.

[0127] The memory 420, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as program instructions / modules corresponding to the data verification method based on heterogeneous data in the embodiment of the present application (for example, the data acquisition module 310, the rule determination module 320, the data table determination module 330, the mapping determination module 340, and the data verification module 350 in the data verification device based on heterogeneous data). The processor 410 executes various function applications and data processing of the electronic device by running the software programs, instructions, and modules stored in the memory 420, that is, implements the data verification method based on heterogeneous data described above.

[0128] The memory 420 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system and application programs required by at least one function; the data storage area can store data created according to the use of the terminal, etc. In addition, the memory 420 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some examples, the memory 420 can further include a memory remotely arranged with respect to the processor 410, and these remote memories can be connected to the electronic device through a network. Examples of the above network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0129] The input device 430 can be used to receive input digital or character information, and generate key signal inputs related to user settings and function control of the electronic device. The output device 440 can include a display device such as a display screen.

[0130] The embodiment of the present application also provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to execute a data verification method based on heterogeneous data, and the method includes:

[0131] acquiring a data maintenance list, data verification requirements, and a to-be-verified database corresponding to a target data verification task; wherein the to-be-verified database includes a target database and a source database;

[0132] determine a target mapping rule corresponding to the target data verification task in a preset mapping rule library based on the data verification requirement;

[0133] determine a target data table corresponding to the target data verification task in the target database based on a data maintenance list, and determine a source data table corresponding to the target data verification task in the source database based on the data maintenance list;

[0134] construct a mapping relationship for the target data table based on the target mapping rule to determine a target library mapping relationship corresponding to the target database, and construct a mapping relationship for the source data table based on the target mapping rule to determine a source library mapping relationship corresponding to the source database;

[0135] differentially compare the source library mapping relationship and the target library mapping relationship to obtain a data verification result corresponding to the target data verification task.

[0136] Of course, the storage medium provided by the embodiment of the present application includes computer executable instructions, which are not limited to the method operations described above, and can also perform related operations in the data verification method based on heterogeneous data provided by any embodiment of the present application.

[0137] Through the above description of the embodiments, those skilled in the art can clearly understand that the present application can be realized by means of software and necessary general hardware, and of course can also be realized by hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a floppy disk, a read-only memory (ROM), a random access memory (RAM), a FLASH, a hard disk or an optical disk, etc., including a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in various embodiments of the present application.

[0138] It is worth noting that the embodiments of the above data verification device based on heterogeneous data include various units and modules only according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific name of each functional unit is only for easy distinction, and does not limit the protection scope of the present application.

[0139] The embodiment of the present application further provides a computer program product comprising a computer program which, when executed by a processor, implements the data verification method based on heterogeneous data provided by any embodiment of the present application.

[0140] The computer program product can be written in any one of a number of programming languages or combinations thereof, including an object oriented programming language such as Java, Smalltalk, C++, or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0141] Note that the above only describes the preferred embodiments of the present application and the principles of the applied technology. Those skilled in the art will understand that the present application is not limited to the specific embodiments described herein, and that various obvious changes, reconfigurations and substitutions can be made by those skilled in the art without departing from the scope of the present application. Therefore, although the present application has been described in detail through the above embodiments, the present application is not limited to the above embodiments, and can include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the appended claims.

Claims

1. A data verification method based on heterogeneous data, characterized in that, include: Obtain the data maintenance list, data verification requirements, and database to be verified corresponding to the target data verification task; wherein, the database to be verified includes the target database and the source database; Based on the data verification requirements, determine the target mapping rule corresponding to the target data verification task in the preset mapping rule base; Based on the data maintenance list, the target data table corresponding to the target data verification task is determined in the target database, and the source data table corresponding to the target data verification task is determined in the source database based on the data maintenance list. Based on the target mapping rules, a mapping relationship is constructed for the target data table to determine the target database mapping relationship corresponding to the target database; and based on the target mapping rules, a mapping relationship is constructed for the source data table to determine the source database mapping relationship corresponding to the source database. By comparing the source library mapping relationship and the target library mapping relationship, the data verification result corresponding to the target data verification task is obtained.

2. The method according to claim 1, characterized in that, Before obtaining the data maintenance list corresponding to the target data verification task, the following steps are also included: Obtain the set of basic data table names corresponding to the target data verification task, and store the set of basic data table names based on a preset lightweight text to obtain the data maintenance list corresponding to the target data verification task.

3. The method according to claim 1, characterized in that, The step of constructing a mapping relationship for the target data table based on the target mapping rules to determine the target database mapping relationship corresponding to the target database, and constructing a mapping relationship for the source data table based on the target mapping rules to determine the source database mapping relationship corresponding to the source database, includes: Based on the target mapping rules, the source data table is mapped and data is extracted to determine the first mapping data set corresponding to the source data table. Based on the first mapping data set and the target mapping rules, the target data table is mapped and data is extracted to determine the second mapping data set corresponding to the target data table. Based on the target mapping rules, the first mapping data set is populated with data to obtain the source database mapping relationship corresponding to the source database. Then, based on the target mapping rules, the second mapping data set is populated with data to obtain the target database mapping relationship corresponding to the target database.

4. The method according to claim 3, characterized in that, The data verification requirements include: full data verification requirements, statistical verification requirements, structural verification requirements, and conditional verification requirements.

5. The method according to claim 4, characterized in that, If the data verification requirement is a full verification requirement, then the step of extracting mapping data from the source data table based on the target mapping rule to determine the first mapping data set corresponding to the source data table includes: Obtain the set of primary key values ​​and residual field values ​​of the source table corresponding to the source data table, and use them as the first mapping data set corresponding to the source data table.

6. The method according to claim 4, characterized in that, If the data validation requirement is a statistical validation requirement, then the step of extracting mapping data from the source data table based on the target mapping rule to determine the first mapping data set corresponding to the source data table includes: Obtain the source table name and the amount of data in the source table, and use them as the first mapping data set corresponding to the source table.

7. The method according to claim 4, characterized in that, If the data validation requirement is a structural validation requirement, then the step of extracting mapping data from the source data table based on the target mapping rule to determine the first mapping data set corresponding to the source data table includes: Obtain the source table name and the set of field structures corresponding to the source data table, and use them as the first mapping data set corresponding to the source data table.

8. The method according to claim 4, characterized in that, If the data validation requirement is a conditional validation requirement, then the step of extracting mapping data from the source data table based on the target mapping rule to determine the first mapping data set corresponding to the source data table includes: Obtain the source table condition fields and the set of source table condition field values ​​corresponding to the source data table, and use them as the first mapping data set corresponding to the source data table.

9. The method according to claim 3, characterized in that, The step of extracting mapping data from the target data table based on the first mapping data set and the target mapping rules to determine the second mapping data set corresponding to the target data table includes: Based on the target mapping rules, the target data table is mapped and the corresponding basic mapping data set is determined. Based on the first mapping data set, the type of the basic mapping data set is converted to obtain the second mapping data set corresponding to the target data table.

10. The method according to claim 1, characterized in that, The source library mapping relationship includes: a first index and a first data value; the target library mapping relationship includes: a second index and a second data value.

11. The method according to claim 10, characterized in that, The difference comparison between the source library mapping relationship and the target library mapping relationship yields the data verification result corresponding to the target data verification task, including: The first index and the second index are compared to obtain the index comparison results; Based on the difference of the index comparison result, the first data value and the second data value are compared to obtain the data verification result corresponding to the target data verification task.

12. A data verification device based on heterogeneous data, characterized in that, include: The data acquisition module is used to acquire the data maintenance list, data verification requirements, and database to be verified corresponding to the target data verification task; wherein, the database to be verified includes the target database and the source database; The rule determination module is used to determine the target mapping rule corresponding to the target data verification task in a preset mapping rule library based on the data verification requirements. The data table determination module is used to determine the target data table corresponding to the target data verification task in the target database based on the data maintenance list, and to determine the source data table corresponding to the target data verification task in the source database based on the data maintenance list. The mapping determination module is used to construct a mapping relationship for the target data table based on the target mapping rules, determine the target database mapping relationship corresponding to the target database, and construct a mapping relationship for the source data table based on the target mapping rules, determine the source database mapping relationship corresponding to the source database. The data verification module is used to compare the source library mapping relationship and the target library mapping relationship to obtain the data verification result corresponding to the target data verification task.

13. An electronic device, characterized in that, The invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, when the processor executes the computer program, it implements the data verification method based on heterogeneous data as described in any one of claims 1-11.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the data verification method based on heterogeneous data as described in any one of claims 1-11.

15. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the data verification method based on heterogeneous data as described in any one of claims 1-11.

Images