Data verification method and device, equipment and storage medium

Abstract

The embodiment of the invention discloses a data verification method and device, equipment and a storage medium, and the method comprises the steps: displaying an initial verification configuration table in a preset interface under the condition that a verification configuration instruction is received; for each to-be-configured item in the initial verification configuration table, generating a configuration result corresponding to the to-be-configured item according to the configuration filling operation, and obtaining a target verification configuration table according to the configuration results of the to-be-configured items; and performing data verification on the to-be-verified data fragment based on the target verification configuration table, and generating a target verification result corresponding to the to-be-verified data fragment. According to the technical scheme, the problems that in an existing data verification technology, the data verification rule is high in modification difficulty and cannot adapt to service requirements are solved, the verification rule can be self-defined for the verification configuration table, data verification can be carried out based on the edited configuration table, bottom layer codes do not need to be modified, the modification difficulty of the verification rule is reduced, and the verification efficiency is improved. And the applicability of data verification is improved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of data processing technology, and in particular to a data verification method, device, equipment and storage medium. Background Art

[0002] In the field of data processing, ETL (Extract Transform Load) technology is a common data processing method used to extract data from the source system and load it into the target system after transformation. Traditional ETL technology usually requires writing a lot of code to implement data processing and verification logic, which has the following shortcomings:

[0003] Complicated code writing: In the traditional ETL process, traditional ETL technology usually adopts static verification rules, which are difficult to modify once defined, lack flexibility, and cannot adapt to changes in business needs. In addition, a large amount of code needs to be written to implement data verification logic, including the definition of verification rules, condition judgment, exception handling, etc., which is labor-intensive and prone to errors. Summary of the invention

[0004] The embodiments of the present invention provide a data verification method, apparatus, device and storage medium, which can customize verification rules for a verification configuration table and perform data verification based on the edited configuration table without modifying the underlying code, thereby reducing the difficulty of modifying the verification rules and improving the applicability of data verification.

[0005] In a first aspect, an embodiment of the present invention provides a data verification method, the method comprising:

[0006] When receiving the calibration configuration instruction, the initial calibration configuration table is displayed in the preset interface;

[0007] For each item to be configured in the initial verification configuration table, a configuration result corresponding to the item to be configured is generated according to the configuration filling operation, and a target verification configuration table is obtained according to the configuration results of the multiple items to be configured; wherein the item to be configured includes: at least one of a field name, a verification type, a verification error description, a verification group, a recording time, an update time and a priority;

[0008] The data segment to be verified is verified based on the target verification configuration table to generate a target verification result corresponding to the data segment to be verified.

[0009] In a second aspect, an embodiment of the present invention provides a data verification device, the device comprising:

[0010] An initial calibration configuration table display module is used to display the initial calibration configuration table in a preset interface when a calibration configuration instruction is received;

[0011] A target verification configuration table determination module is used to generate, for each item to be configured in the initial verification configuration table, a configuration result corresponding to the item to be configured according to a configuration filling operation, and obtain a target verification configuration table according to the configuration results of multiple items to be configured; wherein the item to be configured includes: at least one of a field name, a verification type, a verification error description, a verification group, a recording time, an update time, and a priority;

[0012] The data verification module is used to perform data verification on the data segment to be verified based on the target verification configuration table, and generate a target verification result corresponding to the data segment to be verified.

[0013] In a third aspect, an embodiment of the present invention provides a computer device, the computer device comprising:

[0014] one or more processors;

[0015] A memory for storing one or more programs;

[0016] When the one or more programs are executed by the one or more processors, the one or more processors implement the data verification method described in any embodiment.

[0017] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the data verification method described in any embodiment.

[0018] The technical solution provided by the embodiment of the present invention displays an initial verification configuration table in a preset interface when a verification configuration instruction is received; for each item to be configured in the initial verification configuration table, a configuration result corresponding to the item to be configured is generated according to the configuration filling operation, and a target verification configuration table is obtained according to the configuration results of multiple items to be configured; data verification is performed on the data fragment to be verified based on the target verification configuration table, and a target verification result corresponding to the data fragment to be verified is generated. The technical solution of the embodiment of the present invention solves the problem that the data verification rules in the existing data verification technology are difficult to modify and cannot adapt to business needs. The verification rules can be customized for the verification configuration table, and data verification is performed based on the edited configuration table without modifying the underlying code, thereby reducing the difficulty of modifying the verification rules and improving the applicability of data verification. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a flow chart of a data verification method provided by an embodiment of the present invention;

[0020] Figure 2 It is a flow chart of another data verification method provided by an embodiment of the present invention;

[0021] Figure 3 is a schematic diagram of a target verification configuration table provided by an embodiment of the present invention;

[0022] Figure 4 is a schematic diagram of a target verification type configuration sub-table provided in an embodiment of the present invention;

[0023] Figure 5 This is a workflow diagram for performing data verification provided by an embodiment of the present invention;

[0024] Figure 6 It is a structural schematic diagram of a data verification device provided by an embodiment of the present invention;

[0025] Figure 7 It is a structural schematic diagram of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] Figure 1 It is a flow chart of a data verification method provided by an embodiment of the present invention. The embodiment of the present invention can be applied to a scenario in which data verification is performed based on a verification rule configuration table. The method can be executed by a data verification device, which can be implemented by software and / or hardware.

[0028] like Figure 1 As shown, the data verification method includes the following steps:

[0029] S110. When a verification configuration instruction is received, an initial verification configuration table is displayed in a preset interface.

[0030] Among them, the verification configuration instruction may be a trigger instruction that requires editing of the data verification configuration table. Exemplarily, a trigger control for the verification configuration may be set in a preset interface, and the user may click the control so that the hardware device receives the verification configuration instruction. The preset interface may be a preset interface for interaction. Exemplarily, the display interface of the target terminal may be used as the preset interface. The initial verification configuration table may be an original verification configuration table that has not been edited. By displaying the initial verification configuration table in the preset interface, the user may configure the corresponding data verification content in the initial verification configuration table as needed.

[0031] S120. For each to-be-configured item in the initial verification configuration table, generate a configuration result corresponding to the to-be-configured item according to a configuration filling operation, and obtain a target verification configuration table according to the configuration results of the multiple to-be-configured items.

[0032] Among them, the items to be configured may be options that need to be configured. Among them, the items to be configured include: at least one of field name, check type, check error description, check group, record time, update time and priority. Optionally, the check type includes: at least one of empty and non-empty check, value check, length check, date check, regular check, dictionary enumeration and preset check type. Among them, the preset check type may be a custom check type. Further, the configuration filling operation may be a content filling operation for the items to be configured. Exemplarily, the user may generate the configuration filling operation by voice input, keyboard input or touch screen input. Further, the corresponding configuration filling content may be determined according to the configuration filling operation, and the configuration filling content may be used as the corresponding configuration result to be configured. Further, the target check configuration table may be a table obtained after the initial check configuration table is edited. Specifically, for each item to be configured, the corresponding configuration result may be filled into the configuration item, thereby completing the editing operation of the initial check configuration table. The target check configuration table is obtained.

[0033] S130. Perform data verification on the data segment to be verified based on the target verification configuration table, and generate a target verification result corresponding to the data segment to be verified.

[0034] Among them, the data segment to be verified may be a data segment that needs to be verified. Exemplarily, based on the field name in the target verification configuration table, the data segment corresponding to the field name can be matched from the preset database, and the matched data segment can be used as the data segment to be verified. Furthermore, based on the target verification configuration table, the verification type corresponding to each data segment to be verified can be determined respectively. The target verification result can be the verification result of the data segment in the target verification configuration table. Specifically, for each data segment to be verified, based on the verification type corresponding to the data segment to be verified, the data segment to be verified can be verified, and then the verification result corresponding to the data segment to be verified can be obtained. Finally, the verification results of each data segment to be verified can be summarized to obtain the target verification result.

[0035] The technical solution provided by the embodiment of the present invention displays an initial verification configuration table in a preset interface when a verification configuration instruction is received; for each item to be configured in the initial verification configuration table, a configuration result corresponding to the item to be configured is generated according to the configuration filling operation, and a target verification configuration table is obtained according to the configuration results of multiple items to be configured; data verification is performed on the data fragment to be verified based on the target verification configuration table, and a target verification result corresponding to the data fragment to be verified is generated. The technical solution of the embodiment of the present invention solves the problem that the data verification rules in the existing data verification technology are difficult to modify and cannot adapt to business needs. The verification rules can be customized for the verification configuration table, and data verification is performed based on the edited configuration table without modifying the underlying code, thereby reducing the difficulty of modifying the verification rules and improving the applicability of data verification.

[0036] Figure 2 It is another flow chart of a data verification method provided by an embodiment of the present invention. The embodiment of the present invention can be applied to a scenario in which data verification is performed based on a verification rule configuration table. Based on the above embodiment, this embodiment further illustrates how to generate a configuration result corresponding to a to-be-configured item according to a configuration filling operation, and obtain a target verification configuration table according to the configuration results of multiple to-be-configured items; and how to perform data verification on a to-be-checked data segment based on the target verification configuration table, and generate a target verification result corresponding to the to-be-checked data segment. The device can be implemented by software and / or hardware, and integrated into a computer device with an application development function.

[0037] like Figure 2 As shown, the data verification method includes the following steps:

[0038] S210: When a verification configuration instruction is received, an initial verification configuration table is displayed in a preset interface.

[0039] Among them, the verification configuration instruction may be a trigger instruction that requires editing of the data verification configuration table. Exemplarily, a trigger control for the verification configuration may be set in a preset interface, and the user may click the control so that the hardware device receives the verification configuration instruction. The preset interface may be a preset interface for interaction. Exemplarily, the display interface of the target terminal may be used as the preset interface. The initial verification configuration table may be an original verification configuration table that has not been edited. By displaying the initial verification configuration table in the preset interface, the user may configure the corresponding data verification content in the initial verification configuration table as needed.

[0040] S220. For each to-be-configured item in the initial verification configuration table, generate a configuration result corresponding to the to-be-configured item according to a configuration filling operation, and obtain a target verification configuration table according to the configuration results of the multiple to-be-configured items.

[0041] Among them, the items to be configured may be options that need to be configured. Among them, the items to be configured include: at least one of field name, check type, check error description, check group, record time, update time and priority. Optionally, the check type includes: at least one of empty and non-empty check, value check, length check, date check, regular check, dictionary enumeration and preset check type. Among them, the preset check type may be a custom check type. Further, the configuration filling operation may be a content filling operation for the items to be configured. Exemplarily, the user may generate the configuration filling operation by voice input, keyboard input or touch screen input. Further, the corresponding configuration filling content may be determined according to the configuration filling operation, and the configuration filling content may be used as the corresponding configuration result to be configured. Further, the target check configuration table may be a table obtained after the initial check configuration table is edited. Specifically, for each item to be configured, the corresponding configuration result may be filled into the configuration item, thereby completing the editing operation of the initial check configuration table. The target check configuration table is obtained.

[0042] For example, Figure 3 Schematic diagram of a target verification configuration table provided by an embodiment of the present invention. Figure 3 As shown, the target verification configuration table includes configurable items such as field name, verification type, verification error description, verification group, recording time, update time and priority.

[0043] Optionally, a configuration result corresponding to the item to be configured is generated according to the configuration filling operation, including: when the verification type is a preset verification type, an initial verification type configuration subtable is displayed in a preset interface; for each unconfigured item in the initial verification type configuration subtable, a configuration result corresponding to the unconfigured item is generated according to the configuration filling operation; wherein the unconfigured item includes: at least one of a field name, a custom verification type, a custom verification rule, a basic verification type, a verification error description, a verification label, a rule description and a rule code; a target verification type configuration subtable corresponding to a preset verification type is obtained according to the configuration results of multiple unconfigured items, and the target verification type configuration subtable is used as the configuration result corresponding to the verification type.

[0044] Among them, the initial check type configuration subtable can be a check configuration table corresponding to a preset check type that has not been edited. The user can edit the check content of the preset lesson plan type in the initial check type configuration subtable. Unconfigured items can be items that can be configured in the initial check type configuration subtable. Exemplarily, unconfigured items can include items such as field name, basic verification type, verification label, verification error description, verification group, recording time, update time and priority. Correspondingly, the target check type configuration subtable can be a table obtained after the initial check type configuration subtable is edited. Similarly, the configuration result corresponding to the unconfigured item can also be generated according to the configuration filling operation, and then the target check type configuration subtable corresponding to the preset check type is obtained according to the configuration results of multiple unconfigured items, and the target check type configuration subtable is used as the configuration result corresponding to the check type.

[0045] For example, Figure 4 Schematic diagram of a target verification type configuration sub-table provided by an embodiment of the present invention. Figure 4 As shown, the target verification type configuration subtable includes: field name, basic verification type, verification label, verification error description, recording time, update time and other configurable items.

[0046] S230: Determine multiple data segments to be verified and data verification types corresponding to the data segments to be verified based on the field names and verification types in the target verification configuration table.

[0047] Among them, the data segment to be verified may be a data segment that needs to be verified. Exemplarily, based on the field name in the target verification configuration table, the data segment corresponding to the field name can be matched from the preset database, and the matched data segment can be used as the data segment to be verified. The data verification type can be the type of data verification performed on the data segment to be verified. Exemplarily, the data verification type can also include at least one of empty and non-empty check, value check, length check, date check, regular check, dictionary enumeration and preset verification type. Specifically, the verification type configured for the data segment to be verified in the target verification configuration table can be used as the data verification type corresponding to the data segment to be verified. It should be explained here that the data verification type of the data segment to be verified may not only be of one type, but may be of multiple types, that is, the data segment to be verified can be verified based on multiple data verification types. For example, the length check and date check can be performed on the data segment to be verified.

[0048] S240: For each data segment to be verified, verify the data segment to be verified based on the data verification type to obtain a segment verification result corresponding to the data segment to be verified.

[0049] The segment verification result may be a data verification result of the data segment to be verified. Specifically, the data segment to be verified may be verified based on the data verification type, and the obtained verification result may be used as the segment verification result corresponding to the data segment to be verified.

[0050] Optionally, before performing data verification on the data segment to be verified, verification preconditions of the data segment to be verified may be determined according to the target verification configuration table; when the data segment to be verified meets the verification preconditions, the verification operation on the data segment to be verified is started.

[0051] Among them, the verification precondition can be a precondition for determining whether to perform data verification. Specifically, a fillable item of the verification precondition can also be set in the initial verification configuration table, and the user can fill in the corresponding pre-determination condition in the item. Further, the verification precondition corresponding to the data segment to be verified in the target verification configuration table can be filled in, the verification precondition of the data segment to be verified can be determined, and then it can be determined whether the data segment to be verified meets the verification precondition. If the data segment to be verified meets the verification precondition, the verification operation of the data segment to be verified can be started.

[0052] S250: Determine the target verification result according to the fragment verification results of the multiple data fragments to be verified.

[0053] The target verification result may be a verification result of the data segments in the target verification configuration table. Specifically, after determining the segment verification result of each data segment to be verified, the segment verification results of all the data segments to be verified may be counted, and the statistical result may be used as the target verification result.

[0054] Optionally, when a basic verification configuration instruction is received, a basic verification configuration table can be displayed in a preset interface; wherein the basic verification configuration table is an initial verification configuration table in which all verification types are default verification types; for each configurable item in the basic verification configuration table, a configuration result corresponding to the configurable item is generated according to the configuration item filling operation to obtain a target basic verification configuration table; the data segment corresponding to the target basic verification configuration table is verified based on the default verification type to obtain a basic verification result.

[0055] Among them, the basic check configuration table is a simplified version of the initial check configuration table. There are no configuration items of the check type in the basic check configuration table, that is, the check types in the basic check configuration table are all default check types. Among them, the default check type can be empty and non-empty check, value check, length check, date check, regular check, dictionary enumeration or preset check type. The default check type can be set manually. Further, the configurable item can be an item that can be checked and configured in the basic check configuration table. Optionally, the configurable item can include at least one of the field name, check type, check error description, check group, record time, update time and priority. Similarly, for each configurable item in the basic check configuration table, the configuration result corresponding to the configurable item can be generated according to the configuration item filling operation to obtain the target basic check configuration table. Among them, the target basic check configuration table can be a table obtained after the configuration editing of the basic check configuration table is completed. Further, the data fragment corresponding to the target basic check configuration table can also be checked based on the default check type to obtain the basic check result. By calling the basic check configuration table, the configuration editing task for data verification can be simplified and the efficiency of editing the verification rules can be improved.

[0056] Optionally, in the event of a verification error in the data segment to be verified, a verification error record corresponding to the data segment to be verified may be generated based on the verification error description of the data segment to be verified in the target verification configuration table; and a verification error record table and a verification error data distribution table may be generated based on the verification error records of multiple data segments to be verified.

[0057] Among them, the verification error description can be a way of describing the details of the verification failure. Exemplarily, the verification error description can include ways such as ignoring errors, refusing records or detailed records. The verification error record can be a description of the verification failure details of the data segment to be verified. Specifically, the verification error record corresponding to the data segment to be verified can be generated according to the verification error description of the data segment to be verified. Exemplarily, when the verification error description is to ignore errors, the verification error record corresponding to the data segment to be verified can be blank; when the verification error description is a detailed record, the verification error record corresponding to the data segment to be verified can include the verification error position and error cause of the data segment to be verified. Further, the verification error data distribution table can be a statistical table of the distribution of data segments with verification errors. Exemplarily, the data segments to be verified with verification errors can be marked first, and then the distribution of the marked data segments can be statistically analyzed from the perspectives of data group, data type or data number to obtain the verification error data distribution table.

[0058] For example, in order to better understand the technical solution provided by the present invention, a specific embodiment is introduced below:

[0059] The solution is described from the perspective of product functions:

[0060] Dynamically load validation rules: Users can dynamically load validation rules through configuration files or management interfaces without modifying code or redeploying the system.

[0061] Rich and extensible validation types: The system provides a variety of common validation types, such as empty and non-empty checks, value checks, length checks, date checks, regular expression checks, etc., and supports users to customize validation types according to business needs.

[0062] The verification type of the present invention supports the following types:

[0063] 1. Empty and non-empty checks:

[0064] The invention supports a variety of null and non-null check types, including:

[0065] 1) NOT_BLANK: The verification string cannot be empty and its length must be greater than 0.

[0066] 2) NOT_EMPTY: The check string cannot be empty and its length after removing spaces is greater than 0, or the set cannot be empty and the number of elements is greater than 0.

[0067] 3) NOT_NULL: The verification object cannot be null.

[0068] 4)BLANK: The verification string must be null or its length must be 0.

[0069] 5) EMPTY: The check string must be null or its length is equal to 0 after removing spaces, or the set must be null or the number of elements is equal to 0.

[0070] 6)NULL: The verification object must be null.

[0071] 2. Numerical check:

[0072] The present invention supports checking of value ranges, including:

[0073] 1)Positive: Determine the positive number.

[0074] 2)PositiveOrZero: Determine whether it is a positive number or 0.

[0075] 3)Negative: Determine negative numbers.

[0076] 4)NegativeOrZero: Determines whether it is a negative number or 0.

[0077] 5)Max: The field value must be less than or equal to the specified value.

[0078] 6)Min: The field value must be greater than or equal to the specified value.

[0079] 7)Range: The field value must be within the specified range.

[0080] 3. Length check:

[0081] The present invention supports checking of string length and collection size, including:

[0082] 1)Size: Checks whether the size of the field is within the specified range, which can be an array, collection, map, etc.

[0083] 2)Length: Checks whether the length of the string is within the specified range.

[0084] 4. Date check:

[0085] The present invention supports checking of date values, including:

[0086] 1)Future: must be a future date.

[0087] 2) Past: must be a date in the past.

[0088] 3)Between: must be within the specified date range.

[0089] 5. Regularization check:

[0090] The present invention supports regular expression verification, and the field value must conform to the specified regular expression.

[0091] 6. Dictionary enumeration:

[0092] The present invention supports that the verification string must be in the configured enumeration and fully matched.

[0093] Supports custom verification failure description: Users can set a custom verification failure description for each verification rule and flexibly customize the error message.

[0094] Output result definition: The system outputs detailed verification results, including verification results of individual fields, description of reasons for failure, final verification result summary, etc., to facilitate users to perform subsequent processing and analysis.

[0095] The solution is described from an interactive perspective:

[0096] Dynamic loading validation rules:

[0097] Users can add, modify or delete verification rules through the management interface or configuration files. The system will automatically load and take effect without restarting the service or modifying the code.

[0098] Verification rule configuration interface:

[0099] The system provides a friendly configuration interface, where users can intuitively view and manage all verification rules, including verification type, verification rules, error description and other information.

[0100] Verification results display:

[0101] Users can query the verification results through the management interface or interface. The system will display the results in an easy-to-understand format, including the verification results of individual fields, descriptions of reasons for failure, and the final summary of verification results.

[0102] Through the above functions and interactive design, users can easily configure and manage data verification rules, quickly discover data quality issues, and take corresponding measures to repair and process them in a timely manner to improve data accuracy and reliability.

[0103] For example, Figure 5 This is a workflow diagram for performing data verification provided by an embodiment of the present invention. Figure 5 As shown, the workflow for data verification includes the following steps: based on the database, the basic verification class, the custom verification type and the verification rules related to the list of fields to be verified are obtained, and then the verification processor and the verification rules are initialized, and then the list of fields to be verified and the pre-dependent verification conditions are obtained, and then it is determined whether the list to be verified is completed. If not, the next field to be verified is obtained, and then it is determined whether the field to be verified meets the pre-dependent verification conditions. If not, jump to the next field. If so, the field value is obtained to start data verification, and then it is determined whether the field to be verified is successfully verified. If not, the verification failure information is recorded. After all the fields in the list of fields to be verified are verified, the verification information is summarized and counted.

[0104] The technical solution provided by the embodiment of the present invention is to display an initial verification configuration table in a preset interface upon receiving a verification configuration instruction; for each item to be configured in the initial verification configuration table, generate a configuration result corresponding to the item to be configured according to a configuration filling operation, and obtain a target verification configuration table according to the configuration results of multiple items to be configured; based on the field name and verification type in the target verification configuration table, determine multiple data segments to be verified and the data verification types corresponding to the data segments to be verified; for each data segment to be verified, verify the data segment to be verified based on the data verification type to obtain the segment verification result corresponding to the data segment to be verified; determine the target verification result according to the segment verification results of multiple data segments to be verified. The technical solution of the embodiment of the present invention solves the problem that the data verification rules in the existing data verification technology are difficult to modify and cannot adapt to business needs. By customizing the verification rules for the verification configuration table and performing data verification based on the edited configuration table, there is no need to modify the underlying code, thereby reducing the difficulty of modifying the verification rules and improving the applicability of data verification.

[0105] Figure 6 It is a structural diagram of a data verification device provided by an embodiment of the present invention. The embodiment of the present invention can be applied to the scenario of performing data verification based on the verification rule configuration table. The device can be implemented by software and / or hardware and integrated into a computer device with application development function.

[0106] like Figure 6 As shown, the data verification device includes: an initial verification configuration table display module 310, a target verification configuration table determination module 320 and a data verification module 330.

[0107] Among them, the initial verification configuration table display module 310 is used to display the initial verification configuration table in a preset interface when a verification configuration instruction is received; the target verification configuration table determination module 320 is used to generate a configuration result corresponding to each to-be-configured item in the initial verification configuration table according to a configuration filling operation, and obtain a target verification configuration table according to the configuration results of multiple to-be-configured items; wherein the to-be-configured item includes: at least one of a field name, a verification type, a verification error description, a verification group, a recording time, an update time and a priority; the data verification module 330 is used to perform data verification on the to-be-checked data segment based on the target verification configuration table, and generate a target verification result corresponding to the to-be-checked data segment.

[0108] The technical solution provided by the embodiment of the present invention displays an initial verification configuration table in a preset interface when a verification configuration instruction is received; for each item to be configured in the initial verification configuration table, a configuration result corresponding to the item to be configured is generated according to the configuration filling operation, and a target verification configuration table is obtained according to the configuration results of multiple items to be configured; data verification is performed on the data fragment to be verified based on the target verification configuration table, and a target verification result corresponding to the data fragment to be verified is generated. The technical solution of the embodiment of the present invention solves the problem that the data verification rules in the existing data verification technology are difficult to modify and cannot adapt to business needs. The verification rules can be customized for the verification configuration table, and data verification is performed based on the edited configuration table without modifying the underlying code, thereby reducing the difficulty of modifying the verification rules and improving the applicability of data verification.

[0109] In an optional embodiment, the data verification module 330 is specifically used to: determine multiple data segments to be verified and data verification types corresponding to the data segments to be verified based on the field names and verification types in the target verification configuration table; for each data segment to be verified, verify the data segment to be verified based on the data verification type to obtain a fragment verification result corresponding to the data segment to be verified; and determine the target verification result based on the fragment verification results of the multiple data segments to be verified.

[0110] In an optional embodiment, the target verification configuration table determination module 320 includes: a preset verification type configuration unit, used to: when the verification type is a preset verification type, display an initial verification type configuration subtable in a preset interface; for each unconfigured item in the initial verification type configuration subtable, generate a configuration result corresponding to the unconfigured item according to a configuration filling operation; wherein the unconfigured item includes: at least one of a field name, a custom verification type, a custom verification rule, a basic verification type, a verification error description, a verification label, a rule description and a rule code; according to the configuration results of multiple unconfigured items, obtain a target verification type configuration subtable corresponding to the preset verification type, and use the target verification type configuration subtable as the configuration result corresponding to the verification type.

[0111] In an optional embodiment, the data verification device also includes: a basic verification configuration module, which is used to: display a basic verification configuration table in the preset interface when a basic verification configuration instruction is received; wherein the basic verification configuration table is an initial verification configuration table in which all verification types are default verification types; for each configurable item in the basic verification configuration table, a configuration result corresponding to the configurable item is generated according to a configuration item filling operation to obtain a target basic verification configuration table; and the data segment corresponding to the target basic verification configuration table is verified based on the default verification type to obtain a basic verification result.

[0112] In an optional embodiment, the data verification device also includes: a verification statistics module, which is used to: in the event of a verification error in the data segment to be verified, generate a verification error record corresponding to the data segment to be verified based on the verification error description of the data segment to be verified in the target verification configuration table; and generate a verification error record table and a verification error data distribution table based on the verification error records of multiple data segments to be verified.

[0113] In an optional embodiment, the data verification device also includes: a pre-verification module, which is used to: before performing data verification on the data segment to be verified, determine the verification pre-conditions of the data segment to be verified according to the target verification configuration table; when the data segment to be verified meets the verification pre-conditions, start executing the verification operation on the data segment to be verified.

[0114] In an optional implementation, the check type includes: at least one of: empty and non-empty check, value check, length check, date check, regular check, dictionary enumeration and preset check type.

[0115] The data verification device provided in the embodiment of the present invention can execute the data verification method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0116] Figure 7 A schematic diagram of the structure of a computer device provided in an embodiment of the present invention. Figure 7 A block diagram of an exemplary computer device 12 suitable for use in implementing embodiments of the present invention is shown. Figure 7 The computer device 12 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention. The computer device 12 can be any terminal device with computing capability and can be configured in a data verification device.

[0117] like Figure 7 As shown, the computer device 12 is in the form of a general-purpose computing device. The components of the computer device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that connects various system components (including the system memory 28 and the processing unit 16).

[0118] The bus 18 may be one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. For example, these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0119] The computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the computer device 12, including volatile and non-volatile media, removable and non-removable media.

[0120] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache 32. The computer device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 7 not shown, usually called a "hard drive"). Although Figure 7 Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The system memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present invention.

[0121] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. Program modules 42 generally perform the functions and / or methods of the embodiments described herein.

[0122] The computer device 12 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, displays 24, etc.), one or more devices that enable a user to interact with the computer device 12, and / or any device that enables the computer device 12 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed through an input / output (I / O) interface 22. Furthermore, the computer device 12 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) through a network adapter 20. Figure 7 As shown, the network adapter 20 communicates with other modules of the computer device 12 via the bus 18. It should be understood that although Figure 7 Not shown, other hardware and / or software modules may be used in conjunction with computer device 12, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0123] The processing unit 16 executes various functional applications and data processing by running the program stored in the system memory 28, for example, implementing the data verification method provided in the embodiment of the present invention, which includes:

[0124] When receiving the calibration configuration instruction, the initial calibration configuration table is displayed in the preset interface;

[0125] For each item to be configured in the initial verification configuration table, a configuration result corresponding to the item to be configured is generated according to the configuration filling operation, and a target verification configuration table is obtained according to the configuration results of the multiple items to be configured; wherein the item to be configured includes: at least one of a field name, a verification type, a verification error description, a verification group, a recording time, an update time and a priority;

[0126] The data segment to be verified is verified based on the target verification configuration table to generate a target verification result corresponding to the data segment to be verified.

[0127] This embodiment provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the data verification method provided by any embodiment of the present invention is implemented, including:

[0128] When receiving the calibration configuration instruction, the initial calibration configuration table is displayed in the preset interface;

[0129] For each item to be configured in the initial verification configuration table, a configuration result corresponding to the item to be configured is generated according to the configuration filling operation, and a target verification configuration table is obtained according to the configuration results of the multiple items to be configured; wherein the item to be configured includes: at least one of a field name, a verification type, a verification error description, a verification group, a recording time, an update time and a priority;

[0130] The data segment to be verified is verified based on the target verification configuration table to generate a target verification result corresponding to the data segment to be verified.

[0131] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0132] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0133] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0134] Computer program code for performing the operations of the present invention may be written in one or more programming languages ​​or combinations thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0135] It should be understood by those skilled in the art that the modules or steps of the present invention described above can be implemented by a general-purpose computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, optionally, they can be implemented by a program code executable by a computer device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0136] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A data verification method, characterized in that: include: When receiving the calibration configuration instruction, the initial calibration configuration table is displayed in the preset interface; For each item to be configured in the initial verification configuration table, a configuration result corresponding to the item to be configured is generated according to the configuration filling operation, and a target verification configuration table is obtained according to the configuration results of the multiple items to be configured; wherein the item to be configured includes: at least one of a field name, a verification type, a verification error description, a verification group, a recording time, an update time and a priority; The data segment to be verified is verified based on the target verification configuration table to generate a target verification result corresponding to the data segment to be verified.

2. The method according to claim 1, characterized in that The performing data verification on the data segment to be verified based on the target verification configuration table to generate a target verification result corresponding to the data segment to be verified includes: Determine, based on the field names and verification types in the target verification configuration table, a plurality of data segments to be verified and data verification types corresponding to the data segments to be verified; For each data segment to be verified, verify the data segment to be verified based on the data verification type to obtain a segment verification result corresponding to the data segment to be verified; The target verification result is determined according to the segment verification results of the plurality of data segments to be verified.

3. The method according to claim 1, characterized in that The generating, according to the configuration filling operation, a configuration result corresponding to the item to be configured, comprises: In the case where the verification type is a preset verification type, an initial verification type configuration subtable is displayed in the preset interface; For each unconfigured item in the initial validation type configuration subtable, generate a configuration result corresponding to the unconfigured item according to the configuration filling operation; wherein the unconfigured item includes: at least one of: field name, custom validation type, custom validation rule, basic validation type, validation error description, validation label, rule description and rule code; A target check type configuration subtable corresponding to the preset check type is obtained according to the configuration results of the multiple unconfigured items, and the target check type configuration subtable is used as the configuration result corresponding to the check type.

4. The method according to claim 1, characterized in that: The method further comprises: When a basic verification configuration instruction is received, a basic verification configuration table is displayed in the preset interface; wherein the basic verification configuration table is an initial verification configuration table in which all verification types are default verification types; For each configurable item in the basic verification configuration table, generate a configuration result corresponding to the configurable item according to a configuration item filling operation to obtain a target basic verification configuration table; The data segment corresponding to the target basic verification configuration table is verified based on the default verification type to obtain a basic verification result.

5. The method according to claim 1, characterized in that The method further comprises: In the case of a verification error of the data segment to be verified, generating a verification error record corresponding to the data segment to be verified according to the verification error description of the data segment to be verified in the target verification configuration table; A verification error record table and a verification error data distribution table are generated according to the verification error records of multiple data segments to be verified.

6. The method according to claim 1, characterized in that Before performing data verification on the data segment to be verified, the method further includes: Determining the verification precondition of the data segment to be verified according to the target verification configuration table; When the data segment to be verified meets the verification precondition, a verification operation on the data segment to be verified is started.

7. The method according to claim 1, characterized in that The check type includes: at least one of empty and non-empty check, value check, length check, date check, regular check, dictionary enumeration and preset check type.

8. A data verification device, characterized in that: The device comprises: An initial calibration configuration table display module is used to display the initial calibration configuration table in a preset interface when a calibration configuration instruction is received; A target verification configuration table determination module is used to generate a configuration result corresponding to each to-be-configured item in the initial verification configuration table according to a configuration filling operation, and obtain a target verification configuration table according to the configuration results of multiple to-be-configured items; wherein the to-be-configured items include: at least one of: field name, verification type, verification error description, verification group, recording time, update time and priority; The data verification module is used to perform data verification on the data segment to be verified based on the target verification configuration table, and generate a target verification result corresponding to the data segment to be verified.

9. A computer device, characterized in that: The computer device comprises: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the data verification method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the data verification method as described in any one of claims 1 to 7 is implemented.

Images