Data verification method, apparatus and device, and computer program product

Abstract

The invention discloses a data verification method, apparatus and device, and a computer program product. The method comprises the steps of obtaining information of a to-be-verified data table; querying in a preset associated verification rule configuration table based on the information of the to-be-verified data table; under the condition that a target association verification rule matched with the information of the to-be-verified data table is queried, determining an association data table corresponding to the to-be-verified data table based on the target association verification rule; and based on the target association verification rule, performing association verification on the to-be-verified data table and the corresponding association data table to obtain a data verification result. According to the data verification method, for a configurable language of multi-table data association verification, complicated data verification requirements are classified according to scenes and configured into the association verification rule configuration table, powerful support is provided for multi-table data association verification, the efficiency and accuracy of multi-table data association verification in scenes such as data migration are improved, and the data verification efficiency is improved. The adaptability is high, and the application scene coverage range is wide.

Description

Technical Field

[0001] The present application relates to the technical field of data verification, and particularly to a data verification method, device, equipment, and computer program product. Background Art

[0002] With the rapid development of banking business, outdated business systems will affect business development. Therefore, in order to solve the problems of existing systems, improve business processing capabilities, and meet the requirements of technological innovation, banks inevitably need to update and replace old and new systems. At the same time, data models will also be redesigned. All the data in the old core system must be migrated and transplanted into the new core system through data migration and express the same business meaning.

[0003] However, considering that bank data often has the following characteristics: at the data level, the data scale is large, the data range is wide, the correlation relationships between data are complex, the data model differences between old and new systems are large, and the database table conversion relationships between old and new systems are complex. It is not simply to extract the data from the old table and insert it into the new table as it is. It needs to go through a complex conversion process to become the data that conforms to the business meaning and technical requirements in the new table; at the business level, the business correlation degree of data is high, the business information is sensitive, and the business meaning expressed is rigorous.

[0004] Therefore, in view of the possible quality problems in data migration, a rigorous data verification method must be established to verify the data before it is imported into the new core system. On the one hand, it can prevent errors in data migration, where the business meanings expressed by the data of each data model entity are inconsistent, which will affect customers. On the other hand, it can check the quality problems caused by the lack of unified management of early data and correct them before the new system runs.

[0005] The existing common and most direct implementation solution is to develop code for specific verification content, that is, for the relationship between fields in one table and fields in another table in a specific database, such as equality, conditional equality, within a range, etc., to develop specific code, which is prone to problems such as a large system, poor reusability, and poor response ability. Summary of the Invention

[0006] Embodiments of the present application provide a data verification method, device, equipment, and computer program product to improve the efficiency and accuracy of data verification in scenarios such as data migration.

[0007] Embodiments of the present application adopt the following technical solutions:

[0008] In a first aspect, embodiments of the present application provide a data verification method, and the data verification method includes:

[0009] Obtain information of the data table to be verified;

[0010] Query in the preset associated verification rule configuration table based on the information in the data table to be verified;

[0011] In the case of querying a target associated verification rule that matches the information in the data table to be verified, determine the associated data table corresponding to the data table to be verified based on the target associated verification rule;

[0012] Based on the target associated verification rule, perform an associated verification on the data table to be verified and the corresponding associated data table to obtain a data verification result.

[0013] Optionally, the preset associated verification rule configuration table is configured in the following manner:

[0014] Define the attribute fields of the associated verification rule configuration table, where the attribute fields include the main table, verification type, main table conditions, association conditions, associated table, result type, and result conditions;

[0015] According to the values of each attribute field, use a custom rule generation strategy to generate multiple associated verification rules, and the custom rule generation strategy includes at least one of a custom symbol matching pattern and a custom function method.

[0016] Optionally, the associated verification rule contains one or more groups of main table conditions, and the main table conditions are used to constrain the data range to be verified. Multiple groups of the main table conditions are connected by a first preset separator; each group of the main table conditions consists of a field name, a comparison operator, and a condition value, and one field name corresponds to one or more condition values, and multiple condition values are connected by a second preset separator.

[0017] Optionally, the main table conditions also include at least one of a custom function method and a special connection symbol part. The custom function method is used to extract partial values from the field name of the main table conditions, and the special connection symbol part is used to constrain the information of other data tables that the main table needs to contact except the associated data table.

[0018] Optionally, the associated verification rule also includes an association condition, and the association condition is used to constrain the condition for verifying the associated data table and the data table to be verified. The association condition consists of a field of the data table to be verified, an operator, and a field of the associated data table.

[0019] Optionally, in the case of querying a target associated verification rule that matches the information in the data table to be verified, determining the associated data table corresponding to the data table to be verified based on the target associated verification rule includes:

[0020] In the case of querying a target associated verification rule that matches the information in the data table to be verified, traverse all data records in the data table to be verified;

[0021] For the first target data record in the data table to be verified, traverse all target associated verification rules, where the first target data record is any one of all data records in the data table to be verified;

[0022] For any one target associated verification rule, determine whether the first target data record meets the verification type and main table conditions in the target associated verification rule;

[0023] If it meets, determine the associated data table corresponding to the data table to be verified based on the associated table in the target associated verification rule.

[0024] Optionally, the performing an associated verification on the data table to be verified and the corresponding associated data table based on the target associated verification rule to obtain a data verification result includes:

[0025] Based on the association condition in the target associated verification rule, obtain the data records that meet the association condition from the associated data table corresponding to the data table to be verified to form a result set;

[0026] Based on the result type and result condition in the target associated verification rule, traverse all data records in the result set;

[0027] For the second target data record in the result set, determine whether the second target data record meets the result type and the result condition, where the second target data record is any one of all data records in the result set;

[0028] If the second target data record meets the result type and the result condition, determine that the data verification result of the second target data record is verified passed;

[0029] Otherwise, determine that the data verification result of the second target data record is verified failed.

[0030] In a second aspect, an embodiment of the present application further provides a data verification device, where the data verification device includes:

[0031] An acquisition unit, configured to acquire information of a data table to be verified;

[0032] A query unit, configured to perform a query in a preset associated verification rule configuration table based on the information of the data table to be verified;

[0033] A determination unit, configured to, when a target associated verification rule that matches the information of the to-be-verified data table is queried, determine an associated data table corresponding to the to-be-verified data table based on the target associated verification rule;

[0034] A verification unit, configured to perform an associated verification on the to-be-verified data table and the corresponding associated data table based on the target associated verification rule, so as to obtain a data verification result.

[0035] In a third aspect, an embodiment of the present application further provides a device, including:

[0036] A processor; and a memory arranged to store computer-executable instructions, where the executable instructions, when executed, cause the processor to execute any one of the foregoing data verification methods.

[0037] In a fourth aspect, an embodiment of the present application further provides a computer program product, including a computer program / instructions, where the computer program / instructions, when executed by a processor, implement any one of the foregoing data verification methods.

[0038] The above at least one technical solution adopted in the embodiment of the present application can achieve the following beneficial effects: In the data verification method of the embodiment of the present application, first, the information of the to-be-verified data table is obtained; then, based on the information of the to-be-verified data table, a query is performed in a preset associated verification rule configuration table; then, when a target associated verification rule that matches the information of the to-be-verified data table is queried, an associated data table corresponding to the to-be-verified data table is determined based on the target associated verification rule; finally, based on the target associated verification rule, an associated verification is performed on the to-be-verified data table and the corresponding associated data table to obtain a data verification result. The data verification method of the embodiment of the present application is a configurable language for multi-table data association verification, classifies complicated data verification requirements according to scenarios, and configures them into an associated verification rule configuration table, which provides strong support for multi-table data association verification, improves the efficiency and accuracy of multi-table data association verification in scenarios such as data migration, and has strong adaptability and a wide coverage of application scenarios. Description of the Drawings

[0039] The drawings described herein are used to provide a further understanding of the present application, and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application, and do not constitute an improper limitation to the present application. In the drawings:

[0040] Figure 1 is a schematic diagram of a data migration process in an embodiment of the present application;

[0041] Figure 2 is a schematic diagram of a data verification method in an embodiment of the present application;

[0042] Figure 3It is a schematic diagram of a data verification process in an embodiment of the present application;

[0043] Figure 4 It is a schematic diagram of the structure of a main table condition in an embodiment of the present application;

[0044] Figure 5 It is a schematic diagram of the structure of a data verification device in an embodiment of the present application;

[0045] Figure 6 It is a schematic diagram of the structure of a device in an embodiment of the present application. Detailed implementation manners

[0046] To make the objectives, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be clearly and completely described below in conjunction with specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0047] The following will describe in detail the technical solutions provided by each embodiment of the present application in conjunction with the drawings.

[0048] The embodiment of the present application provides a data verification method. The data verification method in the embodiment of the present application can be applied to scenarios such as data migration. Of course, it can also be applicable to other scenarios with data verification requirements, especially application scenarios with a large business connection and many restrictive requirements between database tables.

[0049] Taking the data migration scenario as an example, as Figure 1 shown, a schematic diagram of a data migration process in an embodiment of the present application is provided. The conventional data migration process is divided into several stages: migration preparation, data export, data conversion, data import, and migration completion. The data verification method provided by the present application occurs after data conversion and before data import in the data migration scenario, that is, the data in the old core database has been converted into data that conforms to the table design, table structure, and business meaning in the new core database according to certain analysis requirements. At this time, the data of each converted table and the configured verification rules related to this table are obtained in turn, and the related table data is queried according to the associated verification rules. If the expected results exist, it can be considered that this piece of data meets the requirements of data meaning and quality and can enter the data import stage. If not, this piece of data cannot be imported into the new core database. Therefore, the implementation process of the present application can be simply summarized into two parts. One is to write verification rules according to the defined configuration language, and the other is to combine the code to obtain all verification rules for the data verification process.

[0050] As Figure 2As shown in the figure, a flowchart of a data verification method in an embodiment of the present application is provided. The data verification method at least includes the following steps S210 to S240:

[0051] Step S210, obtain information of the data table to be verified.

[0052] Combined Figure 3 , a flowchart of a data verification process in an embodiment of the present application is provided. When performing data verification, it is necessary to first obtain information of the data table to be verified (such as the table name or other unique identification information of the table). For example, in a data migration scenario, the table names of all data tables to be verified can be obtained according to the data verification requirements of the data migration scenario. These data tables are the data tables that have completed data conversion but have not been imported into the new system in the data migration scenario.

[0053] Step S220, query in the preset associated verification rule configuration table based on the information of the data table to be verified.

[0054] In the actual application scenario, there are many verification requirements between data. Each verification requirement corresponds to a configuration, forming a verification rule. In the current application scenario, the configurations are recorded in a database table, which is called the associated verification rule configuration table. The associated verification rule configuration table records the data verification rules in various verification scenarios, so as to meet the data verification requirements of different scenarios.

[0055] Based on the table name of the above-mentioned data table to be verified, search in the associated verification rule configuration table to determine whether the associated verification rule corresponding to the information of the data table to be verified can be found. If it can be found, it means that the data table to be verified needs to perform subsequent data association verification. If it cannot be found, it means that the data table to be verified does not need to perform subsequent data association verification for the time being.

[0056] Step S230, in the case of querying a target associated verification rule that matches the information of the data table to be verified, determine the associated data table corresponding to the data table to be verified based on the target associated verification rule.

[0057] If a target associated verification rule that matches the information of the data table to be verified can be found in the associated verification rule configuration table, then extract all the associated verification rules that match this table from the associated verification rule configuration table, and then determine the associated data tables to be associated with the current table for verification based on these verification rules. The associated data table can be understood as a data table actually generated based on the business logic of the new system.

[0058] Step S240, based on the target associated verification rule, perform an association verification on the data table to be verified and the corresponding associated data table to obtain a data verification result.

[0059] After determining the associated data table, the specific verification conditions defined in the target association verification rule can be used to perform association verification on the data records in the data table to be verified and the corresponding data records in the associated data table one by one, and a data verification result can be obtained. The data verification result can specifically include the result of passing or failing the verification of the data table to be verified.

[0060] The data verification method in the embodiments of the present application is a configurable language for multi-table data association verification. It classifies complex data verification requirements according to scenarios and configures them into an association verification rule configuration table, providing strong support for multi-table data association verification, improving the efficiency and accuracy of multi-table data association verification in scenarios such as data migration, and having strong adaptability and a wide coverage of application scenarios.

[0061] In some embodiments of the present application, the preset association verification rule configuration table is configured in the following manner: Define the attribute fields of the association verification rule configuration table, and the attribute fields include the main table, verification type, main table conditions, association conditions, associated table, result type, and result conditions; According to the values of each attribute field, multiple association verification rules are generated using a custom rule generation strategy, and the custom rule generation strategy includes at least one of a custom symbol matching pattern and a custom function method.

[0062] The association verification rules configured in the embodiments of the present application can include attribute fields in multiple dimensions such as the main table, verification type, main table conditions, association conditions, associated table, result type, and result conditions. As shown in Table 1 below, an example of the values corresponding to the multiple attribute fields included in an association verification rule configuration table is given.

[0063] Table 1

[0064]

[0065]

[0066] Among them, the "main table" indicates which table's data is to be verified. The example in Table 1 represents Table A. The "verification type" indicates whether to verify all the data in the main table or the data that meets certain conditions, and can be configured as "default" or "condition" respectively. The example in Table 1 represents that only the data that meets the conditions is verified. The "main table conditions" indicate what conditions the data to be verified should meet, and multiple conditions can be configured simultaneously. The "association conditions" indicate how to locate the data in the associated data table that has a business connection with the data to be verified in the main table. The "associated table" indicates which table's data is to be associated and verified with the main table. The example in Table 1 represents Table B. The "result type" indicates what type of result set is obtained after querying Table B, representing successful verification. Multiple different result types can be abstracted by analyzing the application scenario. The "result conditions" indicate what conditions the data in the result set of Table B should meet to represent successful verification.

[0067] When generating specific association verification rules, they can be generated based on a custom symbol matching pattern and custom function methods. The custom symbol matching pattern defines the matching pattern of the configured symbols to meet the writing of complex conditions. The custom function methods define multiple function methods, and each function method has an adapted code to undertake simple operations on the data. Only the method name needs to be written during configuration.

[0068] Through the above process of generating custom verification rules, various complex association verification rules can be flexibly configured according to different business scenarios and requirements, enabling the solution to be widely applied to various types of data verification scenarios, improving the adaptability and generality of the system. Through the clear definition and configuration of the association verification rules, data can be verified quickly and efficiently according to the predetermined rules. Compared with manual verification or simple rule verification methods, this technical solution can greatly improve the efficiency and accuracy of data verification, saving time and labor costs. In addition, the adopted custom symbol matching pattern and custom function method can provide a rich variety of verification rules for a large amount of data through basic writing logic, covering complex business requirements. And with the development of the business, it is easy to expand new verification scenarios without changing the underlying code design.

[0069] In some embodiments of the present application, the association verification rule includes one or more groups of main table conditions, and the main table conditions are used to restrict the data range to be verified. Multiple groups of the main table conditions are connected by a first preset separator; each group of the main table conditions is composed of a field name, a comparison operator, and a condition value. One field name corresponds to one or more condition values, and multiple condition values are connected by a second preset separator.

[0070] As described above, the "main table conditions" restrict the data for which verification is to be performed under what conditions. Here, the matching pattern of the configuration symbols is defined to meet the writing of complex conditions, and multiple conditions can be configured simultaneously. For example, Figure 4 shows a schematic structural diagram of a main table condition in an embodiment of the present application.

[0071] In a group of main table conditions (①②), the main structure includes several parts such as field name (③), comparison operator (⑤), and condition value (⑥). The writing rules for each group of main table conditions are the same, and each main table condition can be separated by a condition separator (⑦). For part ⑤, common comparison operators such as = (equal to),!= (not equal to), > (greater than), < (less than), >= (greater than or equal to), <= (less than or equal to), etc. can be used. Part ⑥ can also represent multiple values, and they can be separated by a separator such as the "|" symbol. Of course, the specific form of the separator can be flexibly defined according to actual needs and is not specifically limited here.

[0072] In the embodiments of the present application, by presetting separators to connect different main table conditions and condition values, complex verification rules can be flexibly defined to adapt to various business requirements, and new business scenarios and data structure changes can be easily handled. In addition, the rule format is designed to be simple and clear, making it easy to understand, maintain, and expand.

[0073] In some embodiments of the present application, the main table condition further includes at least one of a custom function method and a special connection symbol part. The custom function method is used to obtain partial values in the field name of the main table condition, and the special connection symbol part is used to restrict information of other data tables other than the associated data table that the main table needs to be associated with.

[0074] Continuing to refer to Figure 4 , additional structures such as a custom function method (④) and a special connection symbol part (⑧ and ⑨) can also be configured in the main table condition. The custom function method is used to further restrict the method of obtaining partial values in the field name of the main table. When there is no need to operate on the field value, part ④ may not exist. As shown in Table 2 below, some writable function method names are listed:

[0075] Table 2

[0076] Function method name Function method function getLength() Get the length of the field value substring(n) Extract the first n characters of the field value substring(n,m) Extract the characters from the nth to the mth of the field value lpad(n,m) If the length of the field value is less than n, pad the front with the value m rpad(n,m) If the length of the field value is less than n, pad the back with the value m joint(field,...) Concatenate the values of the field field, multiple fields can be concatenated fixValue(n) Ignore the field name and directly use the fixed value n datelnterval(field) The number of days of the date interval from the value of the field field …… Code can be written according to requirements to adapt to new function names

[0077] Of course, those skilled in the art can flexibly write more function methods according to actual scenario requirements and will not be elaborated here.

[0078] ⑧⑨ are only used in the scenario where the main table needs to be associated with another table other than the associated table. After using a special connection symbol such as the "@" symbol followed by the table name, it indicates that the corresponding field of another table must meet the previous condition (②) for the main table conditions in this group. When another table is not required, the ⑧⑨ part may not exist. The example in Table 1 represents that only the data with field a being 1 in Table A needs to perform this association verification.

[0079] The custom function method allows users to extract specific partial values from field names according to actual needs, greatly enhancing the flexibility and adaptability of data verification. Users can define suitable function methods according to different business scenario requirements. The special connection symbol part enables the system to be associated with other data tables other than the associated data tables, expanding the scope of data association, providing support for complex business scenarios, and improving the overall data verification ability.

[0080] In some embodiments of the present application, the association verification rule further includes an association condition, and the association condition is used to constrain the conditions for the associated data table and the data table to be verified to perform verification. The association condition is composed of the fields of the data table to be verified, operators, and the fields of the associated data table.

[0081] The association condition constrains how to locate the business connection of a piece of data to be verified in the main table (such as Table A in Table 1) with the associated data table (such as Table B in Table 1). The association condition is composed of the fields of the data table to be verified, operators, and the fields of the associated data table. As shown in Table 1, before the operator represents the field of Table A, and after the operator represents the field of Table B. The writing rules of other parts are the same as those of the "main table conditions".

[0082] It should be particularly noted that when the main table condition contains the special connection symbol "@", it is also necessary to add how to locate the business connection of another table (such as Table C) with a piece of data to be verified in Table A, separated by "@", and then write the associated fields of Table A and Table C. Before the operator represents the field of Table A, and after the operator represents the field of Table C. The example in Table 1 represents that when the c field in Table A is equal to the d field in Table B, the data in Table B is used for association verification.

[0083] The writing rules of the association condition are basically the same as those of the main table condition. Users can configure according to the familiar rules, reducing the difficulty of rule writing. In addition, since the association condition can be flexibly configured according to actual business needs, it can adapt to various different business scenarios. Whether it is a simple single-table association or a complex multi-table association, it can be achieved by configuring the corresponding association conditions, improving the generality and scalability of the solution.

[0084] In some embodiments of the present application, "result type" means what kind of result set is obtained after querying the associated data table to represent successful verification. Five result types are abstracted here by analyzing the application scenarios: one is that the number of data items in the result set meets the requirements, which is configured as "query"; the second is that there may be no data in the result set, but if there is data, the conditions must be met, which is configured as "field"; the third is that there is only one data record in the result set and it can meet the conditions, which is configured as "singleField"; the fourth is that there is at least one data record in the result set that meets the conditions, which is configured as "existField"; the fifth is that there must be data records in the result set and all data records meet the conditions, which is configured as "allField". These five types are also taken over by adapted code.

[0085] Of course, other specific result types can be flexibly configured by those skilled in the art according to actual needs, and are not listed here one by one.

[0086] In some embodiments of the present application, "result condition" means the condition that the data in the result set of the associated data table needs to meet. When the "result type" is "query", it only needs to be written as an operator and a number, such as ">0. The writing rules of the "result conditions" corresponding to the other four "result types" are the same as the "main table conditions". At this time, the field name represents the corresponding field in table B. The example in Table 1 is that the value of field b in table B is 0.

[0087] Therefore, if we use words to describe the example in Table 1, it can be described as "When the data in Table A meets the condition field a=1, the value of field c in Table A is equal to the value of field d in Table B to associate with the data in Table B. When the value of field b in the associated data in Table B is 0 or no data in Table B can be associated, it is considered that the verification has passed."

[0088] Through these abstracted configuration scenarios and symbols, most verification requirements can be covered with a very small amount of code, with high readability and fast writing speed.

[0089] In some embodiments of the present application, when a target associated verification rule matching the information of the data table to be verified is found, determining the associated data table corresponding to the data table to be verified based on the target associated verification rule includes: when a target associated verification rule matching the information of the data table to be verified is found, traversing all data records in the data table to be verified; for the first target data record in the data table to be verified, traversing all target associated verification rules, where the first target data record is any one of all data records in the data table to be verified; for any one target associated verification rule, determining whether the first target data record meets the verification type and master table conditions in the target associated verification rule; and when it is met, determining the associated data table corresponding to the data table to be verified based on the associated table in the target associated verification rule.

[0090] Continue to refer to Figure 3 , when verification is required, the information of the data table to be verified (such as the table name) can be first queried in the associated verification rule library. After the target associated verification rule corresponding to the data table to be verified is found, all data records in the data table to be verified can be traversed. For each data record (i.e., the first target data record) in the data table to be verified, all target associated verification rules will be further traversed.

[0091] When traversing each target associated verification rule, the verification type and master table conditions in the rule will be parsed, and the verification type and master table conditions will be used to determine whether the current data record meets the verification conditions. On the one hand, the verification type restricts whether to verify all data in the master table or data meeting certain conditions. If all data in the master table are verified, the current data record meets the requirements of the verification type. If data meeting certain conditions are verified, it is further determined whether the current data record meets the master table conditions. If the master table conditions are met, it indicates that the current data record is the data that needs to be associated and verified, and the associated data table corresponding to the data table to be verified is further determined according to the associated table information in the associated verification rule. Otherwise, the next associated verification rule will be traversed, and so on.

[0092] In the process of traversing the data table to be verified and the target associated verification rule, each data record will be specifically verified according to the verification type and master table conditions. Only data records meeting specific conditions will be subject to subsequent verification operations, avoiding unnecessary calculations and data processing, and improving the efficiency and pertinence of verification. In addition, by traversing the data table to be verified and the target associated verification rule, various different types of data tables and verification types can be processed. Whether it is simple single-table verification or complex multi-table associated verification, it can be achieved by defining corresponding rules and implementing the above technical solutions, improving the scalability of the solution.

[0093] In some embodiments of the present application, the associative verification of the data table to be verified and the corresponding associated data table based on the target associative verification rule to obtain a data verification result includes: obtaining data records that meet the association conditions from the associated data table corresponding to the data table to be verified based on the association conditions in the target associative verification rule, and forming a result set; traversing all the data records in the result set based on the result type and result conditions in the target associative verification rule; for the second target data record in the result set, determining whether the second target data record meets the result type and the result conditions, where the second target data record is any one of all the data records in the result set; if the second target data record meets the result type and the result conditions, determining that the data verification result of the second target data record is verified; otherwise, determining that the data verification result of the second target data record is not verified.

[0094] Continue to refer to Figure 3 , after determining the associated data table, further extract data records that meet the association conditions from the associated data table based on the association conditions configured in the target associative verification rule, and form a result set. The association condition in the example of Table 1 associates the data in Table B by making the value of field c in Table A equal to the value of field d in Table B. Further traverse all the data records in the result set based on the result type and result conditions defined in the target associative verification rule, and for any data record in the result set, determine whether it meets the requirements of the result type and the result conditions at the same time. In the example of Table 1, the result type is "field", and the result condition is b = 0. Then, only when the value of field b in the result set is all 0 or the result set is empty, is it considered that the verification is passed. If any data record in the result set fails the verification, then the result of verification failure is output.

[0095] By verifying each data record in the result set according to the result type and result conditions, it is possible to accurately determine whether the data meets the business requirements, improving the accuracy of data verification. The result type and result conditions configured in the associative verification rule can cover various complex business logics, such as data existence, consistency, integrity, etc. It is possible to comprehensively verify the data according to these rules, improving the reliability of data verification.

[0096] The embodiment of the present application further provides a data verification device 500, as Figure 5 shown, providing a structural schematic diagram of a data verification device in the embodiment of the present application. The data verification device 500 includes: an acquisition unit 510, a query unit 520, a determination unit 530, and a verification unit 540, where:

[0097] An acquisition unit 510, configured to acquire information of a data table to be verified;

[0098] A query unit 520, configured to query in a preset associated verification rule configuration table based on the information of the data table to be verified;

[0099] A determination unit 530, configured to, when a target associated verification rule matching the information of the data table to be verified is queried, determine an associated data table corresponding to the data table to be verified based on the target associated verification rule;

[0100] A verification unit 540, configured to perform an associated verification on the data table to be verified and the corresponding associated data table based on the target associated verification rule to obtain a data verification result.

[0101] In some embodiments of the present application, the preset associated verification rule configuration table is configured in the following manner: defining attribute fields of the associated verification rule configuration table, where the attribute fields include a main table, a verification type, main table conditions, association conditions, an associated table, a result type, and result conditions; generating multiple associated verification rules by using a custom rule generation strategy according to the values of each attribute field, where the custom rule generation strategy includes at least one of a custom symbol matching pattern and a custom function method.

[0102] In some embodiments of the present application, one or more groups of main table conditions are included in the associated verification rule, where the main table conditions are used to constrain the data range to be verified, and multiple groups of the main table conditions are connected by a first preset separator; each group of the main table conditions is composed of a field name, a comparison operator, and a condition value, and one field name corresponds to one or more condition values, and multiple condition values are connected by a second preset separator.

[0103] In some embodiments of the present application, at least one of a custom function method and a special connection symbol part is further included in the main table conditions, where the custom function method is used to extract partial values from the field name of the main table conditions, and the special connection symbol part is used to constrain information of other data tables that the main table needs to be associated with except the associated data table.

[0104] In some embodiments of the present application, an association condition is further included in the associated verification rule, where the association condition is used to constrain the condition for verifying the associated data table and the data table to be verified, and the association condition is composed of a field of the data table to be verified, an operator, and a field of the associated data table.

[0105] In some embodiments of the present application, the determining unit 530 is specifically configured to: when a target associated verification rule that matches the information in the data table to be verified is queried, traverse all data records in the data table to be verified; for a first target data record in the data table to be verified, traverse all target associated verification rules, where the first target data record is any one of all data records in the data table to be verified; for any one target associated verification rule, determine whether the first target data record meets the verification type and master table conditions in the target associated verification rule; if it meets, determine the associated data table corresponding to the data table to be verified based on the associated table in the target associated verification rule.

[0106] In some embodiments of the present application, the verification unit 540 is specifically configured to: based on the association condition in the target associated verification rule, obtain the data records that meet the association condition from the associated data table corresponding to the data table to be verified to form a result set; based on the result type and result condition in the target associated verification rule, traverse all data records in the result set; for a second target data record in the result set, determine whether the second target data record meets the result type and the result condition, where the second target data record is any one of all data records in the result set; if the second target data record meets the result type and the result condition, determine that the data verification result of the second target data record is verified; otherwise, determine that the data verification result of the second target data record is not verified.

[0107] It can be understood that the above data verification device can implement each step of the data verification method provided in the foregoing embodiments. The relevant explanations regarding the data verification method are applicable to the data verification device and will not be elaborated here.

[0108] In summary, the key points and technical effects achieved by the present application mainly include:

[0109] 1) By summarizing the verification categories existing in real application scenarios and defining the pattern meanings of simple symbols, an abstract and configurable rule writing scheme is provided. The present application meets the real demand for the complex relationship of the existence of connections and restrictions among multiple fields of multiple tables, and has high versatility for any system with a large number of business library tables and high data requirements.

[0110] 2) When performing data closed-loop inspection during the data migration process, adopting the technical scheme of the configurable verification rules provided by the present application can quickly respond and take effect in real time. When there are new verification requirements, online updates or repairs can be performed without downtime or restart, greatly improving the ability to ensure data consistency during the migration process.

[0111] 3) The custom symbol matching mode and custom function method adopted in this application can provide a rich variety of verification rules for a large amount of data through basic writing logic, covering complex business requirements. Moreover, with the development of the business, new verification scenarios can be easily extended without changing the underlying code design.

[0112] The method for writing association verification rules between multiple tables provided by this application has high abstraction and is flexible and configurable. Only a small amount of code needs to be combined to cover the verification requirements needed in most real application scenarios. The usability and fault tolerance of the system are greatly enhanced. It can not only be applied in the data migration process, but also has high utilization value for any application scenario with a large business connection and many restrictive requirements between database tables. And through learning the syntax and rules of configuration writing, users do not need to rely on technical personnel to develop background code, and can quickly respond to new verification requirements, efficiently ensuring the consistency and integrity of data during system development or system operation.

[0113] Figure 6 It is a schematic structural diagram of a device in an embodiment of this application. As Figure 6 shown, the device includes one or more processors (or processing units), and may also include one or more memories coupled to the processor, and may also include a communication module coupled to the processor.

[0114] The communication module can be used to communicate with other devices or apparatuses, such as sending or receiving data and / or signals. The communication module can have at least one communication module for communication. The communication module can include any interface necessary for communicating with other devices. Exemplarily, the communication module can be a transceiver, a circuit, a bus, a module or other types of communication modules.

[0115] The processor can include but is not limited to at least one of the following: a general-purpose computer, a special-purpose computer, a microcontroller, a digital signal controller (Digital Signal Processor, DSP), or one or more in a multi-core controller architecture based on a controller. The device can have multiple processors, such as an application-specific integrated circuit chip, which is subordinate to a clock synchronized with the main processor in time.

[0116] The memory may include one or more non-volatile memories and one or more volatile memories. Examples of non-volatile memories include, but are not limited to, at least one of the following: Read-Only-Memory (ROM), Electrically Programmable Read-Only-Memory (EPROM), flash memory, hard disk, Compact Disc (CD), Digital Video Disk (DVD), or other magnetic storage and / or optical storage. Examples of volatile memories include, but are not limited to, at least one of the following: Random Access Memory (RAM), or other volatile memories that do not persist during a power-off duration.

[0117] The computer program includes computer-executable instructions executed by an associated processor. The program may be stored in the ROM. The processor may execute any suitable actions and processes by loading the program into the RAM.

[0118] Possible implementations of the present application may be implemented by means of a program such that the communication device may execute any process discussed in the foregoing embodiments. Possible implementations of the present application may also be implemented by hardware or by a combination of software and hardware.

[0119] In some embodiments, the program may be tangibly embodied in a computer-readable storage medium, which may be included in the device (such as in the memory) or other storage devices accessible by the device. The program may be loaded from the computer-readable storage medium into the RAM for execution. The computer-readable storage medium may include any type of tangible non-volatile memory, such as ROM, EPROM, flash memory, hard disk, CD, DVD, etc.

[0120] The embodiments of the present application also provide a computer-readable storage medium, on which computer instructions or program codes are stored. When the processor runs the instructions or the program codes, the processor is caused to execute the methods and functions involved in any of the above embodiments. The computer-readable medium can be any tangible medium that contains or stores a program for or related to an instruction execution system, apparatus, or device. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any suitable combination thereof. The computer-readable storage medium can be any available medium that the computer can access or a data storage device such as a server or a data center that integrates one or more available media. More detailed examples of the computer-readable storage medium include an electrical connection with one or more wires, magnetic media (such as disks, floppy disks, hard disks, magnetic tapes, magnetic storage devices), optical media (such as optical storage devices, DVDs), semiconductor media (such as solid-state drives), random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), or any suitable combination thereof, etc.

[0121] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The embodiments of the present application also provide at least one computer program product tangibly stored on a non-transitory computer-readable storage medium. The computer program product includes one or more computer-executable instructions, such as instructions included in program modules, which are executed in a device on a target real or virtual processor to execute the processes, methods, and functions involved in any of the above embodiments. When the computer program instructions are loaded and executed on a computer, the processes or functions according to the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (such as coaxial cable, fiber optic, digital subscriber line) or wirelessly (such as infrared, wireless, microwave, etc.).

[0122] Embodiments of the present application also propose a computer program product, including a computer program or instructions. When the computer program or instructions run on a computer, the computer is caused to execute the processes, methods, and functions in the above embodiments. Generally, program modules include routines, programs, libraries, objects, classes, components, data structures, etc. that perform specific tasks or implement specific abstract data types. In various embodiments, the functions of program modules can be combined or divided as needed. Machine-executable instructions for program modules can be executed within local or distributed devices. In a distributed device, program modules can be located in local and remote storage media.

[0123] Generally, various embodiments of the present application can be implemented in hardware or special-purpose circuits, software, logic, or any combination thereof. Some aspects can be implemented in hardware, while other aspects can be implemented in firmware or software, which can be executed by a controller, microprocessor, or other computing device. Although various aspects of the embodiments of the present disclosure are shown and described as block diagrams, flowcharts, or using some other graphical representation, it should be understood that the blocks, devices, systems, techniques, or methods described herein can be implemented as, by way of non-limiting example, hardware, software, firmware, special-purpose circuits or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0124] It should be noted that although the embodiments of the present application are described above in conjunction with the accompanying drawings respectively, the above embodiments are not independent of each other, and they can also be combined to obtain other embodiments. The manners, situations, categories, and divisions of the embodiments in the present application are only for the convenience of description and should not constitute a special limitation. The features in various manners, categories, situations, and embodiments can be combined with each other under logical conditions. The various embodiments of the present application can be combined arbitrarily to achieve different technical effects. The embodiments of the present application will no longer list various combinations.

[0125] In addition, although the operations of the methods of the present disclosure are described in a specific order in the drawings, this does not require or imply that these operations must be performed in that specific order, or that all the illustrated operations must be performed to achieve the desired result. On the contrary, the steps depicted in the flowchart can be changed in the order of execution. Additionally or alternatively, some steps can be omitted, multiple steps can be combined into one step for execution, and / or one step can be decomposed into multiple steps for execution. It should also be noted that the features and functions of two or more devices according to the present disclosure can be embodied in one device. Conversely, the features and functions of one device described above can be further divided and embodied by multiple devices.

[0126] It should also be noted that the term "comprise", "include" or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, commodity or device comprising a series of elements not only includes those elements but also includes other elements not expressly listed, or further includes elements inherent to such process, method, commodity or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, commodity or device comprising said element.

[0127] The above are only embodiments of the present application and are not used to limit the present application. For those skilled in the art, various changes and modifications can be made to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the scope of the claims of the present application.

Claims

1. A data verification method, characterized in that, The data verification method includes: Obtain the information of the data table to be verified; Query in a preset associated verification rule configuration table based on the information of the data table to be verified; When a target associated verification rule that matches the information of the data table to be verified is found, determine the associated data table corresponding to the data table to be verified based on the target associated verification rule; Based on the target associated verification rule, perform an associated verification on the data table to be verified and the corresponding associated data table to obtain a data verification result.

2. The data verification method according to claim 1, wherein The preset associated verification rule configuration table is configured in the following manner: Define the attribute fields of the associated verification rule configuration table, where the attribute fields include a main table, a verification type, main table conditions, association conditions, an associated table, a result type, and result conditions; According to the values of each attribute field, use a custom rule generation strategy to generate multiple associated verification rules, and the custom rule generation strategy includes at least one of a custom symbol matching pattern and a custom function method.

3. The data verification method according to claim 2, wherein One or more groups of main table conditions are included in the associated verification rule, and the main table conditions are used to restrict the data range to be verified, and multiple groups of the main table conditions are connected by a first preset separator; each group of the main table conditions consists of a field name, a comparison operator, and a condition value, and one field name corresponds to one or more condition values, and multiple condition values are connected by a second preset separator.

4. The data verification method according to claim 3, wherein At least one of a custom function method and a special connection symbol part is further included in the main table conditions, the custom function method is used to extract partial values from the field name of the main table conditions, and the special connection symbol part is used to restrict the information of other data tables that the main table needs to be associated with except the associated data table.

5. The data verification method according to claim 1, wherein An association condition is further included in the associated verification rule, and the association condition is used to restrict the condition for verifying the associated data table and the data table to be verified, and the association condition consists of a field of the data table to be verified, an operator, and a field of the associated data table.

6. The data verification method according to claim 1, wherein When a target associated verification rule that matches the information of the data table to be verified is found, determining the associated data table corresponding to the data table to be verified based on the target associated verification rule includes: When a target associated verification rule that matches the information of the data table to be verified is found, traverse all data records in the data table to be verified; For a first target data record in the data table to be verified, traverse all target associated verification rules, and the first target data record is any one of all data records in the data table to be verified; For any one target associated verification rule, determine whether the first target data record meets the verification type and main table conditions in the target associated verification rule; When it is satisfied, determine the associated data table corresponding to the data table to be verified based on the associated table in the target associated verification rule.

7. The data verification method according to claim 6, wherein Based on the target associated verification rule, performing an associated verification on the data table to be verified and the corresponding associated data table to obtain a data verification result includes: Based on the association conditions in the target association verification rule, obtain data records that meet the association conditions from the association data table corresponding to the data table to be verified, and form a result set; Based on the result type and result conditions in the target association verification rule, traverse all data records in the result set; For the second target data record in the result set, determine whether the second target data record meets the result type and the result conditions, where the second target data record is any one of all data records in the result set; If the second target data record meets the result type and the result conditions, determine that the data verification result of the second target data record is verified through; Otherwise, determine that the data verification result of the second target data record is not verified through.

8. A data verification device, characterized in that, The data verification device includes: An acquisition unit for acquiring information of the data table to be verified; A query unit for querying in a preset association verification rule configuration table based on the information of the data table to be verified; A determination unit for, when a target association verification rule matching the information of the data table to be verified is queried, determining the association data table corresponding to the data table to be verified based on the target association verification rule; A verification unit for performing association verification on the data table to be verified and the corresponding association data table based on the target association verification rule to obtain a data verification result.

9. A device, comprising: A processor; And a memory arranged to store computer-executable instructions that, when executed, cause the processor to execute the data verification method according to any one of claims 1 to 7.

10. A computer program product, comprising a computer program / instructions, characterized in that, The computer program / instructions, when executed by the processor, implement the data verification method according to any one of claims 1 to 7.

Images