Data verification method, device, computer equipment and storage medium

Abstract

The present application relates to a data verification method, device, computer equipment, storage medium and computer program product. The method comprises: obtaining business data to be verified, determining the business data type of the business data to be verified; determining a target rule table according to the business data type, and determining a target verification method for the business data to be verified according to the business data type; determining the verification rules to be executed in the target rule table based on the target verification method, and performing local data verification on the business data to be verified based on the verification rules to be executed to obtain the target verification result. The use of this method can reduce the operational complexity of data verification.

Description

Technical Field

[0001] The present application relates to the technical field of data quality verification, and in particular to a data verification method, apparatus, computer equipment, storage medium and computer program product. Background Art

[0002] The current business regulatory reporting system has a significant increase in the number of business data items that require quality verification, and the data structure of business data is becoming more and more complex. The inspection standards for each business data under different businesses are different. Therefore, the verification of business data is becoming more and more difficult.

[0003] In traditional technology, technicians determine configuration parameters and template information based on manually formulated business data verification rules, automatically generate rule SQL (Structured Query Language) based on the configuration parameters and template information, and generate a business data verification program for rule verification by combining rule SQL with application code in the Hadoop (distributed system infrastructure) ecosystem, that is, packaging the rule SQL into a JAR (Java Archive) package, and deploying the JAR package in the Hadoop (distributed system infrastructure) ecosystem to complete the compilation of the business data verification program, and verify the quality of business data through the business data verification program. Among them, in the process of importing data from SQL rules in a relational database into the Hadoop ecosystem for cross-platform analysis, it is necessary to use the big data tool component Sqoop (SQL-to-Hadoop, a data transmission tool) for data transmission.

[0004] However, in traditional technologies, since the business data verification program requires big data tool components to perform cross-platform analysis, the operation and maintenance complexity of the business data verification program is relatively high. Summary of the invention

[0005] Based on this, it is necessary to provide a data verification method, device, computer equipment, computer-readable storage medium and computer program product to address the above technical problems.

[0006] In a first aspect, the present application provides a data verification method, comprising:

[0007] Acquire the business data to be verified, and determine the business data type of the business data to be verified;

[0008] Determine a target rule table according to the business data type, and determine a target verification method for the business data to be verified according to the business data type;

[0009] Based on the target verification method, the verification rules to be executed are determined in the target rule table, and local data verification is performed on the business data to be verified based on the verification rules to be executed to obtain a target verification result.

[0010] In one embodiment, the determining the target verification method of the to-be-verified business data according to the business data type includes:

[0011] Determine the primary key configuration of the reporting model according to the business data type and the primary key information table;

[0012] The business data is matched based on the primary key configuration of the reporting model, and a target verification method is determined based on the matching result.

[0013] In one of the embodiments, the target rule table includes a basic rule configuration table;

[0014] The determining of the verification rules to be executed in the target rule table based on the target verification mode, and performing local data verification on the business data to be verified based on the verification rules to be executed to obtain the target verification result includes:

[0015] Determine the target scanning method based on the target verification method;

[0016] Scan the business data to be verified according to the target scanning method to obtain target verification data;

[0017] For each of the target verification data, determining the verification rule to be executed corresponding to the target verification data in the basic rule configuration table;

[0018] Each target verification data is locally verified according to the verification rules to be executed corresponding to each target verification data to obtain a target verification result.

[0019] In one of the embodiments, the performing local data verification on the to-be-verified business data based on the to-be-executed verification rule to obtain a target verification result includes:

[0020] Performing local data verification on the business data to be verified based on the verification rules to be executed to obtain an initial verification result;

[0021] According to the initial verification result, matching is performed in the problem detail table and the rule definition table respectively to obtain the problem detail information and verification description information corresponding to the initial verification result;

[0022] A verification checklist is constructed according to the problem detail information, the verification description information and the initial verification result as the target verification result.

[0023] In one embodiment, before obtaining the business data to be verified, the method further includes:

[0024] Obtaining initial verification rules, classifying the initial rules according to preset classification rules, and obtaining rationality verification rules, code table type verification rules, function type verification rules and cross table type verification rules;

[0025] According to the rationality verification rules, the code table type verification rules, the function type verification rules and the cross-table type verification rules, a basic rule configuration table and a special rule mapping table are constructed as a target rule table.

[0026] In one of the embodiments, the target rule table includes a special rule mapping table;

[0027] The determining of the verification rules to be executed in the target rule table based on the target verification mode, and performing local data verification on the business data to be verified based on the verification rules to be executed to obtain the target verification result includes:

[0028] In the case where the target verification method includes special verification, determining the verification rules to be executed according to the special rule mapping table;

[0029] The business data to be verified is verified based on the verification rules to be executed to obtain a target verification result.

[0030] In a second aspect, the present application also provides a data verification device, including:

[0031] An acquisition module, used to acquire the business data to be verified and determine the business data type of the business data to be verified;

[0032] A first determination module, configured to determine a target rule table according to the business data type, and determine a target verification method for the business data to be verified according to the business data type;

[0033] The first verification module is used to determine the verification rules to be executed in the target rule table based on the target verification method, and perform local data verification on the business data to be verified based on the verification rules to be executed to obtain the target verification result.

[0034] In one of the embodiments, the first determination module is specifically used to determine the primary key configuration of the reporting model according to the business data type and the primary key information table;

[0035] The business data is matched based on the primary key configuration of the reporting model, and a target verification method is determined based on the matching result.

[0036] In one of the embodiments, the target rule table includes a basic rule configuration table; the first verification module is specifically used to determine the target scanning mode based on the target verification mode;

[0037] Scan the business data to be verified according to the target scanning method to obtain target verification data;

[0038] For each of the target verification data, determining the verification rule to be executed corresponding to the target verification data in the basic rule configuration table;

[0039] Each target verification data is locally verified according to the verification rules to be executed corresponding to each target verification data to obtain a target verification result.

[0040] In one of the embodiments, the first verification module is specifically used to perform local data verification on the business data to be verified based on the verification rule to be executed to obtain an initial verification result;

[0041] According to the initial verification result, matching is performed in the problem detail table and the rule definition table respectively to obtain the problem detail information and verification description information corresponding to the initial verification result;

[0042] A verification checklist is constructed according to the problem detail information, the verification description information and the initial verification result as the target verification result.

[0043] In one embodiment, the device further comprises:

[0044] A classification module is used to obtain initial verification rules, classify the initial rules according to preset classification rules, and obtain rationality verification rules, code table type verification rules, function type verification rules and cross table type verification rules;

[0045] A construction module is used to construct a basic rule configuration table and a special rule mapping table as a target rule table according to the rationality verification rule, the code table type verification rule, the function type verification rule and the cross-table type verification rule.

[0046] In one embodiment, the target rule table includes a special rule mapping table, and the device further includes:

[0047] A second determination module, configured to determine the verification rules to be executed according to the special rule mapping table when the target verification method includes special verification;

[0048] The business data to be verified is verified based on the verification rules to be executed to obtain a target verification result.

[0049] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0050] Acquire the business data to be verified, and determine the business data type of the business data to be verified;

[0051] Determine a target rule table according to the business data type, and determine a target verification method for the business data to be verified according to the business data type;

[0052] Based on the target verification method, the verification rules to be executed are determined in the target rule table, and local data verification is performed on the business data to be verified based on the verification rules to be executed to obtain the target verification result. In a fourth aspect, the present application also provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the following steps are implemented:

[0053] Acquire the business data to be verified, and determine the business data type of the business data to be verified;

[0054] Determine a target rule table according to the business data type, and determine a target verification method for the business data to be verified according to the business data type;

[0055] Based on the target verification method, the verification rules to be executed are determined in the target rule table, and local data verification is performed on the business data to be verified based on the verification rules to be executed to obtain a target verification result.

[0056] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0057] Acquire the business data to be verified, and determine the business data type of the business data to be verified;

[0058] Determine a target rule table according to the business data type, and determine a target verification method for the business data to be verified according to the business data type;

[0059] Based on the target verification method, the verification rules to be executed are determined in the target rule table, and local data verification is performed on the business data to be verified based on the verification rules to be executed to obtain the target verification result. The above-mentioned data verification method, device, computer equipment, storage medium and computer program product obtain the business data to be verified and determine the business data type of the business data to be verified; determine the target rule table according to the business data type, and determine the target verification method of the business data to be verified according to the business data type; determine the verification rules to be executed in the target rule table based on the target verification method, and perform local data verification on the business data to be verified based on the verification rules to be executed to obtain the target verification result. By adopting this method, the method of performing local verification on the business data to be verified through the verification rules contained in the target rule table can be executed and maintained in a single data development environment, without the need to use the big data tool components for cross-platform transmission, and the maintenance cost of the target rule table is much less than the operation and maintenance difficulty of the business data verification program for cross-platform data processing, thereby reducing the operation and maintenance complexity of data verification. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0061] Figure 1 An application environment diagram of a data verification method in an embodiment;

[0062] Figure 2 A schematic diagram of a process for determining a target verification method in an embodiment;

[0063] Figure 3 A schematic diagram of a process for determining a target verification result in one embodiment;

[0064] Figure 4 A schematic diagram of a process for determining a target verification result in another embodiment;

[0065] Figure 5 A schematic diagram of a process for determining a target rule table in one embodiment;

[0066] Figure 6 is a schematic diagram of a rule classification method in one embodiment;

[0067] Figure 7 A schematic diagram of a process for determining a target verification result in another embodiment;

[0068] Figure 8 A flowchart of an example of a data verification method in an embodiment;

[0069] Fig. 9 is a structural block diagram of a data verification device in one embodiment;

[0070] Fig.10 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0071] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0072] In one embodiment, Figure 1 As shown, a data verification method is provided. This embodiment uses the method applied to a terminal as an example for illustration. It can be understood that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0073] Step 102: Acquire the business data to be verified and determine the business data type of the business data to be verified.

[0074] In an embodiment of the present application, the terminal obtains the business data to be verified, and the business data may be the business data contained in the business policies of different business types, and the regulatory authorities have different regulatory standards for different business types. Before reporting the business data, the insurance company may preliminarily verify the business data, including policy data, claims data, premium data, customer data, etc., to ensure the integrity, accuracy and compliance of the business data. The terminal may determine the business data type based on the specific source of the business data to be verified, including data interface, data file, etc., and determine the business data type of the business data to be verified based on the file or interface of the business data to be verified, including FTP (File Transfer Protocol) download or API call. Alternatively, the terminal may determine the business data type of the business data to be verified based on the policy type recorded in the business policy to which the business data to be verified belongs.

[0075] Step 104, determining a target rule table according to the business data type, and determining a target verification method for the business data to be verified according to the business data type.

[0076] In an embodiment of the present application, the terminal may group the business data types in advance, and the business data types in different grouping results correspond to different rule tables. The terminal determines the target rule table in the rule tables corresponding to different business types according to the business field in which the business data type is located and the specific business needs. With the changes in business needs and the evolution of business data, the target rule table needs to be continuously maintained and updated. This includes adding new rules, modifying old rules, deleting outdated rules, etc., to maintain the consistency of the rule table with business needs.

[0077] After the terminal determines the target rule table, it can verify the business data to be verified according to the target verification method. The target verification method is determined according to the specific business data type and verification requirements. For example, the target verification method can include cross-table verification, single-table verification and special verification. Among them, cross-table verification is to verify the business data in a single data table, which usually involves verification of table structure, field rules, data range, etc. to ensure data quality and accuracy. Single-table verification is often used in relatively simple business scenarios; cross-table verification is to jointly verify the data in multiple data tables, which usually involves verification of associations between tables, rule verification of associated fields, data matching, etc. Cross-table verification is usually used in more complex business scenarios, such as business logic verification of data, cross-table data comparison, etc. in target verification. Cross-table verification requires joint query and verification between multiple tables; special verification is to verify special business types through special rules in special rule mapping tables. The special rule mapping table contains the mapping relationship between special business types and special rules corresponding to each special business type.

[0078] Step 106, based on the target verification method, determine the verification rules to be executed in the target rule table, and perform local data verification on the business data to be verified based on the verification rules to be executed to obtain the target verification result.

[0079] In the embodiment of the present application, the terminal supports program calls based on verification requirements, and can support scenarios such as incremental monitoring, inventory scanning, and supplementary reporting verification. Based on the target verification method, the terminal automatically calculates and limits the verification scope in the target rule table, selects the reporting model to wait for the execution of the verification rules, and performs screening, comparison, calculation and other operations in the business data based on the verification rules to be executed to determine whether the data meets the requirements of the verification rules. Optionally, the terminal can support the scanning method of a single rule, a reporting table, or a reporting model.

[0080] In an optional embodiment, when the verification rules of the regulatory authority are updated, since the rule table of the embodiment of the present application is stored in a database, the terminal can dynamically modify and expand the rule table according to the rule update situation, so that it can be adjusted according to business needs during runtime without recompiling and deploying code.

[0081] In the above data verification method, the method of locally verifying the business data to be verified through the verification rules contained in the target rule table can be executed and maintained in a single data development environment without the need to use big data tool components for cross-platform transmission. In addition, the maintenance cost of the target rule table is much lower than the operation and maintenance difficulty of the business data verification program for cross-platform data processing, thereby reducing the operation and maintenance complexity of data verification.

[0082] In an exemplary embodiment, Figure 2 As shown, step 104 includes steps 202 to 204. Among them:

[0083] Step 202: Determine the primary key configuration of the reporting model according to the business data type and the primary key information table.

[0084] In an embodiment of the present application, the primary key information table is a pre-stored primary key information that needs to be verified for each business type, including primary key column information corresponding to different business tables and corresponding business data types. The reporting model can be composed of multiple business data tables, which are used to convert the original business data into a standardized data format that meets the reporting format. The data record of the reporting model contains one or more primary key fields, which are used to identify each data record to facilitate data association, filtering and sorting operations. The terminal obtains the primary key information corresponding to the business data type in the pre-stored primary key information table. For example, in the insurance policy of the insurance business, data such as the policy number, date or name are usually used as the primary key. According to the primary key information in the primary key information table, the terminal configures the column that needs to be used as the primary key to the reporting model. For example, the business data type that is used as the primary key information is determined in multiple business tables, and the primary key configuration of the reporting model is configured according to the business data type that is used as the primary key information.

[0085] In an optional embodiment, after completing the primary key configuration of the reporting model, the terminal can query the data source to check whether the primary key can uniquely identify each data record and there is no duplication, so as to ensure the data accuracy and uniqueness of the reporting model.

[0086] Step 204, matching the business data based on the primary key configuration of the reporting model, and determining the target verification method based on the matching result.

[0087] In an embodiment of the present application, the terminal matches the business data according to the primary key configuration of the reporting model, and re-determines the column information serving as the primary key in multiple business data tables. As a matching result, the column information of the primary key corresponds to the business data type. The terminal can pre-store verification methods corresponding to different business data types, and based on the verification method corresponding to the business data type, determine the target verification method based on the matching result.

[0088] In this embodiment, business data is matched based on the primary key configuration of the reporting model, and the target verification method is determined according to the matching results, which can help achieve data accuracy and completeness to meet data verification requirements.

[0089] In an exemplary embodiment, the target rule table includes a basic rule configuration table, such as Figure 3 As shown, step 106 includes steps 302 to 308. Among them:

[0090] Step 302, determining a target scanning method based on a target verification method.

[0091] In the embodiment of the present application, different target verification methods have corresponding scanning methods, and the terminal determines the target scanning method among multiple scanning methods based on the target verification method.

[0092] Step 304: Scan the business data to be verified according to the target scanning method to obtain target verification data.

[0093] In the embodiment of the present application, the terminal scans the business data to be verified according to the target scanning method determined to be completed, and determines the target verification data with the corresponding verification rules in the business form to which the business data to be verified belongs.

[0094] Step 306: for each target verification data, determine the verification rule to be executed corresponding to the target verification data in the basic rule configuration table.

[0095] In the embodiment of the present application, the target rule table includes a basic rule table and a special rule table. For each target verification data, the terminal determines the verification rules to be executed corresponding to the target verification data in the basic rule configuration table. For example, when the target verification method is single-table verification, the terminal first determines the business data type of the business data to be verified, and scans the business data table containing the business data to be verified and the target rule table corresponding to the business data type to obtain the target verification data and the verification rules corresponding to the target verification data; when the target verification method is cross-table verification, the terminal needs to scan the main table, the right table and the target rule tables corresponding to the main table and the right table respectively to obtain the target verification data with corresponding relationships in the main table and the right table and the verification rules corresponding to the target verification data.

[0096] Step 308: Locally verify each target verification data according to the verification rules to be executed corresponding to each target verification data to obtain the target verification result.

[0097] In the embodiment of the present application, after the terminal determines the corresponding verification rule to be executed according to each target verification data, each target verification data is verified separately. During the verification process, it is necessary to execute the verification rules to be executed one by one, and perform different verification processing for target verification data of different data types and data structures of target verification data.

[0098] In this embodiment, the verification rules corresponding to the target verification business data in the business data to be verified are determined by querying the rule table in the database, and the target verification data is locally verified according to the verification rules to obtain the target verification results, thereby realizing the verification of the data to be verified. When the regulatory requirements change, the updated basic rule configuration table can be scanned to directly realize the verification of the data to be verified, thereby reducing the maintenance complexity of the verification method.

[0099] In an exemplary embodiment, Figure 4 As shown, step 106 includes steps 402 to 406. Among them:

[0100] Step 402: Perform local data verification on the business data to be verified based on the verification rules to be executed to obtain an initial verification result.

[0101] In the embodiment of the present application, the result obtained by the terminal through verification of the business data to be verified based on the verification rules to be executed is a preliminary verification result, and the initial verification result obtained is a verification result for whether the business data to be verified complies with the verification rules.

[0102] Step 404, matching is performed in the problem detail table and the rule definition table respectively according to the initial verification result to obtain the problem detail information and verification description information corresponding to the initial verification result.

[0103] In an embodiment of the present application, after the terminal obtains the initial verification result of the business data to be verified at the data level, when outputting, it is necessary to output the initial verification result of the business data and the description of the related information together, so as to solve the corresponding problem for the initial verification result. The terminal matches the initial verification result in the problem detail table and the verification description information table to obtain the problem detail information and verification description information corresponding to the initial verification result. The verification description information can provide explanations and instructions related to the initial verification result, wherein the matching of the problem detail information and the verification description information with the initial verification result can be performed through the primary key or other key fields to accurately match the corresponding information. For example, in the problem detail table, matching is performed according to the key field value to determine the problem detail information corresponding to the initial verification result. The problem detail information may include detailed verification description information such as error type, error description, and error level.

[0104] Step 406, construct a verification checklist based on the problem detail information, verification description information and initial verification results as the target verification results.

[0105] In an embodiment of the present application, the terminal associates the matched problem detail information and verification description information with the initial verification results, constructs the target verification results, and stores the target verification results of the business data to be verified of different business types in the verification result table, so as to facilitate subsequent analysis of the problems in the verification results.

[0106] The terminal can also configure the target verification results and data clearing, clear the problematic data, identify key field information such as primary key and serial number, insert statistical information on the problems in the verification results into the verification results table, and automatically output the verification list. Through the rule definition table, the terminal can automatically calculate and output the core indicators that need to be paid attention to in the rule dimension and special dimension data governance work.

[0107] In this embodiment, by associating the initial verification results with the problem details and verification description information, a target verification result with a detailed problem description and verification information is constructed, which can assist the verification personnel to better understand and solve the problem and improve the efficiency and accuracy of analyzing the target verification results after data verification.

[0108] In an exemplary embodiment, Figure 5 As shown, before step 102, the method further includes steps 502 to 504, wherein:

[0109] Step 502, obtain initial verification rules, classify the initial rules according to preset classification rules, and obtain rationality verification rules, code table type verification rules, function type verification rules and cross-table type verification rules.

[0110] In the embodiment of the present application, the initial verification rules issued by the regulator are classified into intra-table verification rules and inter-table verification rules. The intra-table verification rules are verification rules for verifying the data to be verified in a single form in a single-table verification method in a single reporting model. The inter-table verification rules are verification rules for jointly verifying the models to be verified with related relationships in at least two forms in a cross-table verification method in multiple reporting models. The inter-table verification rules are generally divided into existence-type verification and logic-type verification.

[0111] like Figure 6 As shown, Figure 6This is an example diagram of the rule classification method. The terminal obtains the initial verification rule, which includes multiple verification rules. Among them, the rationality verification rule includes the rules for the business data to be verified that are non-empty and enumerated values. For example, the terminal decomposes the rule text of the initial verification rule into two parts: judgment text and verification text. The judgment text is used to judge the prerequisite for the rule verification. That is, when there are judgment conditions such as "if a data item meets the conditions...", the current verification rule is used as the judgment text; the verification text is used to verify the object that needs to be verified by the verification rule (that is, the master data item corresponding to the judgment text). If the rule does not need to be judged, the default value is directly filled in. The verification text part of this type of rule cannot be empty.

[0112] The code table type verification rule verifies the data items that are required to be filled with code values ​​in the "Business Code Table" based on supervision. The fixed association table of this type of verification rule is the "Business Code Table". Based on the same principle as the rationality rule, the initial verification rule is decomposed into two parts: data domain association condition text and verification text. The data domain association text is to associate the data domain to which this data item belongs, and the verification text is to judge whether the code value of the data item exists in the data domain.

[0113] Functional validation rules are validation rules that include multiple judgments and verifications for scenarios that include complex validation rules. For example, consistency verification of ID numbers, ID numbers that represent gender and gender, and ID numbers that represent date of birth and date of birth based on business specifications; Functional validation rules also include initial validation rules that require nested functions to be used, such as validation of the first few digits or length of a data item; Functional validation rules also include initial validation rules with a high degree of complexity, such as the need to split and verify multiple code values ​​if a data item can be reported with multiple code values; Functional validation rules also include initial validation rules where a judgment or verification text involves multiple fields, such as the amount calculation validation commonly used in the financial module reporting model.

[0114] Cross-table validation rules are for inter-table verification rules. Cross-table validation between different reporting models is usually for two data models, divided into the main table and the right table. The right table is usually the full table, and the main table is the incremental table or the full table. It can also be broken down into two parts: the associated data field and the verification text. The associated data field is the condition for the association between the two tables. The primary key or foreign key is used for association to avoid the Cartesian product association that reduces the verification efficiency. This type of validation can also implement data validation for more than two tables, and the associated data field field can be used to assist in adding associated tables.

[0115] Step 504, construct a basic rule configuration table and a special rule mapping table as a target rule table according to the rationality verification rules, the code table verification rules, the function table verification rules and the cross-table verification rules.

[0116] In the embodiment of the present application, the terminal constructs a basic rule configuration table and a special rule mapping table according to rationality verification rules, code table verification rules, function verification rules and cross-table verification rules, wherein the judgment operation and verification operation text in the target rule table can assist in realizing such rule configuration, and the business function is nested in this field, and the judgment and verification fields and function identifiers are passed in, and the verification text can be automatically output. Among them, the business functions are uniformly developed and maintained using a stored procedure, which is convenient for unifying and reusing templates.

[0117] In an optional embodiment, the configuration fields used to match the business data to be verified in the basic rule configuration table and the special rule mapping table, in addition to the core fields corresponding to different rule modes, may also include common extraction fields such as transaction code, provincial agency number, data main system source, and business system source fields. Such fields are fixed in the data verification program to avoid repeated extraction. In addition, the basic rule configuration table and the special rule mapping table also include mandatory configuration verification fields and verification conditions, as well as configurations that support personalized field verification, such as process fields or auxiliary fields in the problem data field processing process, to facilitate attribution of problems in the data exploration stage. After completing the rule configuration, it is necessary to ensure that the primary key field of the reporting form has been entered, and the verification program can automatically identify the reporting model to which the rule belongs, and automatically insert the primary key information and serial number information into the problem library.

[0118] In this embodiment, by using the initial verification rules, the initial rules are classified according to the preset classification rules, and then the target rule mapping table is constructed, so that the target verification rule table can be automatically configured to improve the efficiency of data verification.

[0119] In an exemplary embodiment, the target rule table includes a special rule mapping table, such as Figure 7 As shown, after step 106, the method further includes steps 702 to 704. Among them:

[0120] Step 702, when the target verification method includes special verification, determine the verification rules to be executed according to the special rule mapping table.

[0121] In an embodiment of the present application, when the target verification method is special verification, the terminal needs to scan the special rule mapping table and the special business data table to obtain the target verification data with the business data type of special business and the verification rules corresponding to the special business.

[0122] Step 704, perform data verification on the business data to be verified based on the verification rules to be executed to obtain the target verification result.

[0123] In an embodiment of the present application, the special rule mapping table contains verification rules with special characteristics in the business dimension. At the terminal, for the business data to be verified of the special business type, a separate call is made to the redirection rule mapping table to determine the verification rules to be executed. Under the verification rules to be executed, based on the same principle as step 308, data verification is performed on the business data to be verified of the special business type to obtain the target verification result. The data verification process will not be elaborated in this embodiment of the present application.

[0124] In this embodiment, by setting a special rule mapping table and using the special rule mapping table to verify the to-be-verified business data of a special business type, the adaptation scope of the data verification can be expanded and the accuracy of the data verification can be improved.

[0125] In an exemplary embodiment, Figure 8 As shown, Figure 8 This is a flowchart of an example of a data verification method. The terminal contains a configured rule definition table, rule configuration table, and special rule mapping table. In the case of incremental monitoring, inventory verification, and supplementary verification, the terminal matches the scanning method according to the preset execution method code. The execution methods include rule scanning, single table scanning, and redirection scanning, and triggers the corresponding verification method according to the rule type code. The initial verification result is associated with the problem detail table, problem statistics table, and rule definition table to obtain a verification list as the target verification result.

[0126] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0127] Based on the same inventive concept, the embodiment of the present application also provides a data verification device for implementing the data verification method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more data verification device embodiments provided below can refer to the limitations on the data verification method above, and will not be repeated here.

[0128] In an exemplary embodiment, Fig. 9As shown, a data verification device 900 is provided, comprising: an acquisition module 901, a first determination module 902 and a first verification module 903, wherein:

[0129] An acquisition module 901 is used to acquire the business data to be verified and determine the business data type of the business data to be verified;

[0130] The first determination module 902 is used to determine the target rule table according to the business data type, and determine the target verification method of the business data to be verified according to the business data type;

[0131] The first verification module 903 is used to determine the verification rules to be executed in the target rule table based on the target verification method, and perform local data verification on the business data to be verified based on the verification rules to be executed to obtain the target verification result.

[0132] In one embodiment, the first determination module 902 is specifically used to determine the primary key configuration of the reporting model according to the business data type and the primary key information table;

[0133] Match business data based on the primary key configuration of the reporting model, and determine the target verification method based on the matching results.

[0134] In one embodiment, the target rule table includes a basic rule configuration table; the first verification module 903 is specifically used to determine the target scanning mode based on the target verification mode;

[0135] Scan the business data to be verified according to the target scanning method to obtain the target verification data;

[0136] For each target verification data, determine the verification rule to be executed corresponding to the target verification data in the basic rule configuration table;

[0137] Each target verification data is locally verified according to the verification rules to be executed corresponding to each target verification data to obtain the target verification result.

[0138] In one embodiment, the first verification module 903 is specifically used to perform local data verification on the business data to be verified based on the verification rules to be executed, and obtain an initial verification result;

[0139] According to the initial verification results, the problem details table and the rule definition table are matched to obtain the problem details information and verification description information corresponding to the initial verification results;

[0140] Construct a verification checklist based on the problem details, verification description information and initial verification results as the target verification results.

[0141] In one embodiment, the apparatus 900 further includes:

[0142] A classification module is used to obtain initial verification rules, classify the initial rules according to preset classification rules, and obtain rationality verification rules, code table type verification rules, function type verification rules and cross table type verification rules;

[0143] The construction module is used to construct a basic rule configuration table and a special rule mapping table as a target rule table according to rationality verification rules, code table verification rules, function type verification rules and cross table type verification rules.

[0144] In one embodiment, the target rule table includes a special rule mapping table, and the apparatus 900 further includes:

[0145] A second determination module is used to determine the verification rules to be executed according to the special rule mapping table when the target verification method includes special verification;

[0146] Based on the verification rules to be executed, data verification is performed on the business data to be verified to obtain the target verification result.

[0147] Each module in the above data verification device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module above.

[0148] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Fig.10 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store XX data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a data verification method is implemented.

[0149] Those skilled in the art will understand that Fig.10The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0150] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0151] Obtain the business data to be verified, and determine the business data type of the business data to be verified;

[0152] Determine the target rule table based on the business data type, and determine the target verification method for the business data to be verified based on the business data type;

[0153] Based on the target verification method, the verification rules to be executed are determined in the target rule table, and local data verification is performed on the business data to be verified based on the verification rules to be executed to obtain the target verification result.

[0154] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0155] Determine the primary key configuration of the reporting model based on the business data type and primary key information table;

[0156] Match business data based on the primary key configuration of the reporting model, and determine the target verification method based on the matching results.

[0157] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0158] Based on the target verification method, the verification rules to be executed are determined in the target rule table, and local data verification is performed on the business data to be verified based on the verification rules to be executed to obtain the target verification results, including:

[0159] Determine the target scanning method based on the target verification method;

[0160] Scan the business data to be verified according to the target scanning method to obtain the target verification data;

[0161] For each target verification data, determine the verification rule to be executed corresponding to the target verification data in the basic rule configuration table;

[0162] Each target verification data is locally verified according to the verification rules to be executed corresponding to each target verification data to obtain the target verification result.

[0163] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0164] Perform local data verification on the business data to be verified based on the verification rules to be executed, and obtain the initial verification results;

[0165] According to the initial verification results, the problem details table and the rule definition table are matched to obtain the problem details information and verification description information corresponding to the initial verification results;

[0166] Construct a verification checklist based on the problem details, verification description information and initial verification results as the target verification results.

[0167] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0168] Obtaining initial verification rules, classifying the initial rules according to preset classification rules, and obtaining rationality verification rules, code table type verification rules, function type verification rules and cross table type verification rules;

[0169] According to the rationality verification rules, code table verification rules, function type verification rules and cross table type verification rules, a basic rule configuration table and a special rule mapping table are constructed as the target rule table.

[0170] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0171] Based on the target verification method, the verification rules to be executed are determined in the target rule table, and local data verification is performed on the business data to be verified based on the verification rules to be executed to obtain the target verification results, including:

[0172] When the target verification method includes special verification, the verification rules to be executed are determined according to the special rule mapping table;

[0173] Based on the verification rules to be executed, data verification is performed on the business data to be verified to obtain the target verification result.

[0174] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0175] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0176] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0177] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0178] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0179] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A data verification method, characterized in that: The method comprises: Acquire the business data to be verified, and determine the business data type of the business data to be verified; Determine the target rule table according to the business data type, and determine the target verification method of the business data to be verified according to the business data type and verification requirements; the target verification method includes cross-table verification, single-table verification and special verification; Determine the verification rules to be executed in the target rule table based on the target verification method, and perform local data verification on the business data to be verified based on the verification rules to be executed to obtain a target verification result; Determining the target rule table according to the business data type, and determining the target verification method of the to-be-verified business data according to the business data type and the verification requirement, includes: Determine the primary key configuration of the reporting model according to the business data type and the primary key information table; the primary key information table is a pre-stored primary key information corresponding to each business type that needs to be verified, including primary key column information corresponding to different business data tables and corresponding business data types; the primary key configuration of the reporting model is the primary key information of the column configuration that needs to be used as the primary key in the primary key information table; Matching the business data based on the primary key configuration of the reporting model, determining the column information as the primary key in the multiple business data tables as a matching result; the column information of the primary key corresponds to the business data type; Based on the matching result, a target verification method is determined from the verification methods corresponding to different types of business data stored in advance.

2. The method according to claim 1, characterized in that The target rule table includes a basic rule configuration table; The step of determining the verification rules to be executed in the target rule table based on the target verification method, and performing local data verification on the business data to be verified based on the verification rules to be executed to obtain the target verification result includes: Determine the target scanning method based on the target verification method; Scan the business data to be verified according to the target scanning method to obtain target verification data; For each of the target verification data, determining the verification rule to be executed corresponding to the target verification data in the basic rule configuration table; Each target verification data is locally verified according to the verification rules to be executed corresponding to each target verification data to obtain a target verification result.

3. The method according to claim 1, characterized in that: The local data verification of the business data to be verified based on the verification rule to be executed to obtain the target verification result includes: Performing local data verification on the business data to be verified based on the verification rules to be executed to obtain an initial verification result; According to the initial verification result, matching is performed in the problem detail table and the rule definition table respectively to obtain the problem detail information and verification description information corresponding to the initial verification result; A verification checklist is constructed according to the problem detail information, the verification description information and the initial verification result as the target verification result.

4. The method according to claim 1, characterized in that: Before obtaining the business data to be verified, the method further includes: Obtaining initial verification rules, classifying the initial rules according to preset classification rules, and obtaining rationality verification rules, code table type verification rules, function type verification rules and cross table type verification rules; According to the rationality verification rules, the code table type verification rules, the function type verification rules and the cross-table type verification rules, a basic rule configuration table and a special rule mapping table are constructed as a target rule table.

5. The method according to any one of claims 1 to 3, characterized in that: The target rule table includes a special rule mapping table; The step of determining the verification rules to be executed in the target rule table based on the target verification method, and performing local data verification on the business data to be verified based on the verification rules to be executed to obtain the target verification result includes: In the case where the target verification method includes special verification, determining the verification rules to be executed according to the special rule mapping table; The business data to be verified is verified based on the verification rules to be executed to obtain a target verification result.

6. A data verification device, characterized in that: The device comprises: An acquisition module, used to acquire the business data to be verified and determine the business data type of the business data to be verified; A first determination module is used to determine a target rule table according to the business data type, and determine a target verification method for the business data to be verified according to the business data type and verification requirements; the target verification method includes cross-table verification, single-table verification and special verification; A first verification module, configured to determine a verification rule to be executed in the target rule table based on the target verification method, and to perform local data verification on the business data to be verified based on the verification rule to be executed to obtain a target verification result; The first determination module is specifically used to determine the primary key configuration of the reporting model according to the business data type and the primary key information table; the primary key information table is a pre-stored primary key information corresponding to each business type that needs to be verified, including primary key column information corresponding to different business data tables and corresponding business data types; the reporting model primary key configuration is the primary key information of the column configuration that needs to be used as the primary key in the primary key information table; Matching the business data based on the primary key configuration of the reporting model, determining the column information as the primary key in the multiple business data tables as a matching result; the column information of the primary key corresponds to the business data type; Based on the matching result, a target verification method is determined from the verification methods corresponding to different types of business data stored in advance.

7. The device according to claim 6, characterized in that The target rule table includes a basic rule configuration table; the first verification module is specifically used to determine the target scanning mode based on the target verification mode; Scan the business data to be verified according to the target scanning method to obtain target verification data; For each of the target verification data, determining the verification rule to be executed corresponding to the target verification data in the basic rule configuration table; Each target verification data is locally verified according to the verification rules to be executed corresponding to each target verification data to obtain a target verification result.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images