Data checking method and device, electronic equipment and storage medium

Abstract

The present disclosure provides a data checking method, device and equipment and a storage medium. It relates to the technical field of data processing. The method comprises: obtaining a checking configuration file of a database to be checked; parsing the checking configuration file and generating a structured query language (SQL) condition statement set based on the parsing result; generating an executable SQL script set based on the checking configuration file and the SQL condition statement set; and executing the SQL script set on the database to be checked to obtain a data checking result of the database to be checked. Thus, the present scheme parses the checking configuration file, generates a SQL condition statement set based on the parsing result, and further generates an executable SQL script set, thereby improving the accuracy of data checking. Executing the SQL script set on the database to be checked can automatically check the database to be checked, thereby improving the efficiency of data checking and saving labor costs.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, and in particular to a data verification method, apparatus, electronic device, and storage medium. Background Technology

[0002] Currently, data migration requires executing numerous data verification rule scripts to verify the accuracy of the imported data. Furthermore, these verification rules are constantly being added, and the Structured Query Language (SQL) generation methods for different rules vary, resulting in a massive and time-consuming workload for data migration. Summary of the Invention

[0003] This disclosure provides a data verification method, apparatus, device, and storage medium to solve the problem in the prior art where the method of generating SQL conditional statements needs to be manually adjusted when facing different verification rules during the data verification process.

[0004] Therefore, one objective of this disclosure is to propose a data verification method.

[0005] The second objective of this disclosure is to provide a data verification device.

[0006] The third objective of this disclosure is to propose an electronic device.

[0007] The fourth objective of this disclosure is to provide a non-transitory computer-readable storage medium.

[0008] The fifth objective of this disclosure is to provide a computer program product.

[0009] To achieve the above objectives, a first aspect of this disclosure proposes a data verification method, comprising: obtaining a verification configuration file of a database to be verified; parsing the verification configuration file and generating a set of Structured Query Language (SQL) conditional statements based on the parsing results; generating a set of executable SQL scripts based on the verification configuration file and the set of SQL conditional statements; and executing the set of SQL scripts on the database to be verified to obtain a data verification result of the database to be verified.

[0010] According to one embodiment of this disclosure, parsing the inspection configuration file and generating a set of structured query language (SQL) conditional statements based on the parsing results includes: parsing the inspection configuration file to obtain a data table to be inspected in the database to be inspected, as well as the fields to be inspected and the inspection types corresponding to the fields in the data table to be inspected; and obtaining the set of SQL conditional statements based on the data table to be inspected, the fields to be inspected, and the corresponding inspection types.

[0011] According to one embodiment of this disclosure, obtaining the SQL condition statement set based on the data table to be checked, the fields to be checked, and the corresponding check types includes: for each field to be checked, obtaining the table name of the target data table to which the field to be checked belongs from the data table to be checked; determining the target SQL condition corresponding to the target check type based on the target check type corresponding to the field to be checked; and assembling the SQL condition statements based on the table name of the target data table to be checked, the field name of the field to be checked, and the target SQL condition to generate the SQL condition statement set.

[0012] According to one embodiment of this disclosure, assembling an SQL condition statement based on the table name of the target data table to be inspected, the field name of the field to be inspected, and the target SQL conditions includes: assembling the table name of the target data table to be inspected, the field name of the target field to be inspected, and the SQL conditions; obtaining the SQL keyword corresponding to the target inspection type; and adding the SQL keyword to the assembled initial SQL condition statement to obtain the SQL condition statement.

[0013] According to one embodiment of this disclosure, the step of generating an executable SQL script set based on the check configuration file and the SQL condition statement set includes: obtaining the identifier and result prompt information of the target check type corresponding to each field to be checked based on the check configuration file; reading the SQL condition statement of each field to be checked from the SQL condition statement set; for each field to be checked, reassembling the identifier and result prompt information of the target check type corresponding to the field to be checked with the SQL condition statement of the field to be checked to obtain an executable SQL statement; configuring the environment variables and database connection operator of the database to be checked for the executable SQL statement to generate the executable SQL script set.

[0014] According to one embodiment of this disclosure, the step of executing the SQL script set on the database to be checked to obtain the data check result of the database to be checked includes: obtaining the configuration information of the database to be checked; executing test statements on the database to be checked to determine whether the configuration information of the database to be checked is correct; if the configuration information is correct, connecting to the database to be checked based on the database connection operator, and executing the SQL script set on the database to be checked to obtain the data check result of the database to be checked.

[0015] According to one embodiment of this disclosure, before executing the SQL script set on the database to be checked to obtain the data check result of the database to be checked, the method further includes: adding a login script command for logging into the database to the executable SQL script set, and executing the login script command to perform a login operation on the database to be checked.

[0016] According to one embodiment of this disclosure, after executing the SQL script set on the database to be checked to obtain the data check result of the database to be checked, the method further includes: converting the data check result into a formatted check result and storing it in a check table.

[0017] To achieve the above objectives, a second aspect of this disclosure provides a data verification device, comprising: an acquisition module for acquiring a verification configuration file of a database to be verified; a first generation module for parsing the verification configuration file and generating a set of Structured Query Language (SQL) conditional statements based on the parsing results; a second generation module for generating a set of executable SQL scripts based on the verification configuration file and the set of SQL conditional statements; and an execution module for executing the set of SQL scripts on the database to be verified to obtain a data verification result for the database to be verified.

[0018] According to one embodiment of this disclosure, the first generation module is further configured to: parse the inspection configuration file to obtain the inspection data table in the inspection database, the inspection fields in the inspection data table and the inspection types corresponding to the inspection fields; and obtain the SQL condition statement set based on the inspection data table, the inspection fields and the corresponding inspection types.

[0019] According to one embodiment of this disclosure, the first generation module is further configured to: for each field to be checked, obtain the table name of the target data table to which the field to be checked belongs from the data table to be checked; determine the target SQL condition corresponding to the target check type according to the target check type corresponding to the field to be checked; and assemble the SQL condition statement according to the table name of the target data table to be checked, the field name of the field to be checked, and the target SQL condition to generate the SQL condition statement set.

[0020] According to one embodiment of this disclosure, the first generation module is further configured to: assemble the table name of the target data table to be inspected, the field name of the target field to be inspected, and the SQL conditions; obtain the SQL keyword corresponding to the target inspection type; and add the SQL keyword to the assembled SQL initial condition statement to obtain the SQL condition statement.

[0021] According to one embodiment of this disclosure, the second generation module is further configured to: obtain the identifier and result prompt information of the target inspection type corresponding to each field to be inspected based on the inspection configuration file; read the SQL condition statement of each field to be inspected from the SQL condition statement set; for each field to be inspected, reassemble the identifier and result prompt information of the target inspection type corresponding to the field to be inspected with the SQL condition statement of the field to be inspected to obtain an executable SQL statement; configure the environment variables and database connection operator of the database to be inspected for the executable SQL statement to generate the executable SQL script set.

[0022] According to one embodiment of this disclosure, the execution module is further configured to: obtain configuration information of the database to be checked; execute test statements on the database to be checked to determine whether the configuration information of the database to be checked is correct; when the configuration information is correct, connect to the database to be checked based on the database connection operator, and execute the SQL script set on the database to be checked to obtain the data check result of the database to be checked.

[0023] According to one embodiment of this disclosure, the execution module is further configured to: add a login script command for logging into the database to the executable SQL script set, and execute the login script command to perform a login operation on the database to be checked.

[0024] According to one embodiment of this disclosure, the execution module is further configured to: convert the data check result into a format to obtain a formatted check result, and store it in a check table.

[0025] To achieve the above objectives, a third aspect of this disclosure provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor to implement the data verification method as described in the first aspect of this disclosure.

[0026] To achieve the above objectives, a fourth aspect of this disclosure provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to implement the data verification method as described in the first aspect of this disclosure.

[0027] To achieve the above objectives, a fifth aspect of this disclosure provides a computer program product, including a computer program that, when executed by a processor, implements the data verification method as described in the first aspect of this disclosure. Attached Figure Description

[0028] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0029] Figure 1 A flowchart illustrating a data verification method provided in this embodiment of the disclosure;

[0030] Figure 2 A flowchart illustrating the data verification process provided in this embodiment of the disclosure;

[0031] Figure 3 A flowchart of another data verification method provided in this disclosure embodiment;

[0032] Figure 4 A flowchart of another data verification method provided in this disclosure embodiment;

[0033] Figure 5 This is a flowchart illustrating the parsing of the verification configuration file provided in an embodiment of this disclosure.

[0034] Figure 6 A flowchart of another data verification method provided in this disclosure embodiment;

[0035] Figure 7 A flowchart illustrating the generation of an executable script provided in this embodiment of the disclosure;

[0036] Figure 8 A flowchart of another data verification method provided in this disclosure embodiment;

[0037] Figure 9 A flowchart illustrating the execution script provided in this embodiment of the disclosure;

[0038] Figure 10 This is a schematic diagram of the structure of the data verification device provided in the embodiments of this disclosure;

[0039] Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure.

[0040] The accompanying drawings have illustrated specific embodiments of this disclosure, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concepts of this disclosure to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0041] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0042] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0043] The acquisition, storage, use, and processing of data in this disclosed technical solution all comply with the relevant provisions of national laws and regulations.

[0044] Figure 1 This is a schematic flowchart illustrating the data verification method provided in an embodiment of this disclosure. Figure 1 As shown, the data verification method includes:

[0045] S101, Obtain the inspection configuration file of the database to be inspected.

[0046] It should be noted that the data verification method disclosed herein is executed by an electronic device, which may be a terminal device or a server. Optionally, the terminal device may be a mobile electronic device or a non-mobile electronic device. For example, mobile electronic devices may be mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, ultra-mobile personal computers (UMPCs), netbooks, or personal digital assistants (PDAs), etc., while non-mobile electronic devices may be network-attached storage (NAS), personal computers (PCs), televisions (TVs), ATMs, or self-service machines, etc. The server may also be a cloud server, a distributed system server, or a server incorporating blockchain technology, etc. This disclosure does not impose specific limitations on the embodiments.

[0047] In some implementations, the verification configuration file for the database to be verified can be determined based on business verification requirements. Optionally, business characteristics can be analyzed to determine verification-related information, such as the data tables to be verified and the fields within those tables, and configuration can be performed based on this information to obtain the verification configuration file for the database to be verified. It should be noted that the verification configuration file is also the verification rule file, which contains the data tables to be verified, the fields within those tables to be verified, and the verification types corresponding to those fields. Verification types, also known as verification rules, include, but are not limited to: non-null verification, uniqueness verification, value range verification, validity verification, association consistency verification, and reconciliation verification.

[0048] It is understandable that the check type corresponding to the field to be checked is configured to the field to be checked in the data table. For example, configuring a uniqueness check to field X indicates that a uniqueness check needs to be performed on field X.

[0049] S102, parse the verification configuration file and generate a set of structured query language (SQL) conditional statements based on the parsing results.

[0050] In some implementations, a pre-defined check rule parser can determine the format of the check configuration file, and then parse the check configuration file to obtain the parsing result. Optionally, the check rule parser can read either file configuration or database table configuration. Optionally, the parsing result includes the data table to be checked, the fields to be checked within the data table, and the check types corresponding to the fields to be checked.

[0051] In some implementations, the parsing results yield the data table to be checked, the fields to be checked, and the corresponding check types. Based on the check types corresponding to the fields to be checked, a corresponding Structured Query Language (SQL) can be generated. Multiple SQL conditional statements are then assembled to obtain a set of SQL conditional statements. In essence, each check type can generate a corresponding SQL conditional statement, and assembling the SQL conditional statements corresponding to different check types results in a set of SQL conditional statements.

[0052] Optionally, any field to be checked can correspond to one check type or multiple check types, meaning that any field to be checked can generate one or more SQL conditional statements.

[0053] S103 generates a set of executable SQL scripts based on the verification configuration file and the set of SQL conditional statements.

[0054] In some implementations, the identification information corresponding to the inspection type and the result prompt information for the field to be inspected can be determined based on the inspection configuration file. By reading the set of SQL condition statements, each SQL condition statement is obtained, and the SQL condition statements are reassembled according to the identification information and result prompt information to obtain executable SQL condition statements.

[0055] Furthermore, the executable SQL conditional statements are assembled into a script based on the database environment variables and database connection operators, resulting in a set of executable SQL scripts. Optionally, the identifier and type of the database to be checked can be used as environment variables, and the access address can be used as the database connection operator.

[0056] S104: Execute a set of SQL scripts on the database to be checked to obtain the data check results of the database to be checked.

[0057] In some implementations, an executable set of SQL scripts can be connected to the database to be checked, the SQL scripts can be executed on the database to be checked, and the check results can be recorded. Optionally, the check results of the database to be checked can be stored in a database table for later use. Optionally, a connection to the database to be checked can be established through the database configuration information, so as to perform the check process on the database through the executable set of SQL scripts and obtain the check results of the database.

[0058] It should be noted that before executing the SQL script set on the database to be checked, test statements need to be executed on the database to determine whether the configuration information of the database is correct.

[0059] Optionally, the database configuration information may include database address, username, password, database environment information, and other related content. The database can be logged into using the address, username, and password, and the database environment information can be securely verified. After successful security verification, a set of executable SQL scripts can be executed on the database.

[0060] In the data verification method provided in this disclosure, a verification configuration file is parsed, and a set of SQL conditional statements is generated based on the parsing results. Based on the verification configuration file, the set of SQL conditional statements is used to generate a set of executable SQL scripts to improve the accuracy of data verification. Then, the set of SQL scripts is executed on the database to be verified to obtain the data verification results. This data verification method can automatically verify the database to be verified, improving the efficiency of data verification, saving labor costs, and effectively shortening the data verification time.

[0061] Figure 2 It is a flowchart for automating data verification, such as Figure 2The process involves parsing the verification configuration file using a parser, processing different verification types differently, and initially assembling SQL conditional statements to form a set of SQL conditional statements. For each SQL conditional statement in the set, a script generator produces a corresponding executable script, creating a set of executable scripts. The executable script set is then read, executed by the script executor, and the data verification results are output.

[0062] Figure 3 This is a flowchart illustrating another data verification method provided in an embodiment of this disclosure. Figure 3 As shown, the data verification method includes:

[0063] S301, Obtain the inspection configuration file of the database to be inspected.

[0064] The details of step S301 can be found in the above embodiments and will not be repeated here.

[0065] S302, parse the inspection configuration file, obtain the inspection data table in the inspection database, as well as the inspection fields and the inspection types corresponding to the inspection fields in the inspection data table.

[0066] In some implementations, the contents of the verification configuration file are obtained by reading it, and a verification configuration file parser is used to parse the read contents to obtain the parsed result. Optionally, the verification configuration file parser supports both file configuration and data table configuration formats.

[0067] Optionally, the parsing result of the verification configuration file includes the data table to be verified in the database to be verified, as well as the fields to be verified and the verification types corresponding to the fields to be verified within the data table to be verified.

[0068] S303: Obtain the set of SQL conditional statements based on the data table to be checked, the fields to be checked, and the corresponding check types.

[0069] In some implementations, the table name and field names of the data table to be checked can be identified based on the parsing results. Then, a framework of SQL conditional statements can be created based on these table and field names. The SQL conditions from the check type, along with the table and field names, are then filled into the created SQL conditional statements to obtain assembled SQL conditional statements corresponding to different check rules. Combining multiple SQL conditional statements yields a set of SQL conditional statements.

[0070] Optionally, the SQL keywords corresponding to the check type can be added to the assembled SQL conditional statement to obtain the SQL conditional statement. These SQL keywords include, but are not limited to, WHERE, AND, QR, and IN.

[0071] S304 generates a set of executable SQL scripts based on the verification configuration file and the set of SQL conditional statements.

[0072] S305 executes a set of SQL scripts on the database to be checked to obtain the data check results of the database to be checked.

[0073] The relevant content of steps S304-S305 can be found in the above embodiments, and will not be repeated here.

[0074] In the data verification method provided in this disclosure, a verification configuration file is parsed to obtain the parsing result. A set of SQL conditional statements is generated based on the data table to be verified, the fields to be verified, and the corresponding verification types. Then, according to the verification configuration file, the set of SQL conditional statements is used to generate a set of executable SQL scripts to improve the accuracy of data verification. The set of SQL scripts is executed on the database to be verified to obtain the data verification result. This data verification method can automatically verify the database to be verified, improving the efficiency of data verification, saving labor costs, and effectively shortening the data verification time.

[0075] Figure 4 A flowchart illustrating another data verification method provided in this disclosure embodiment. (For example...) Figure 4 As shown, the data verification method includes:

[0076] S401, Obtain the inspection configuration file of the database to be inspected.

[0077] S402, parse the inspection configuration file, obtain the inspection data table in the inspection database, as well as the inspection fields and the inspection types corresponding to the inspection fields in the inspection data table.

[0078] The relevant content of steps S401-S402 can be found in the above embodiments, and will not be repeated here.

[0079] S403, for each field to be checked, obtain the table name of the target data table to which the field to be checked belongs from the data table to be checked.

[0080] In some implementations, each field in the data table to be checked has a corresponding table name. The fields to be checked can be identified by examining the structure of the data table, and the table name of the target data table to which the field belongs can be determined by finding the correspondence between the fields and the table names.

[0081] S404, determine the target SQL condition corresponding to the target inspection type based on the target inspection type corresponding to the field to be inspected.

[0082] In some implementations, the target check type corresponding to the field to be checked can be determined based on the parsing results of the check configuration file. For any target check type, checking its corresponding field can determine the target SQL condition for that target check type.

[0083] For example, for non-null check types, the corresponding field to be checked is identified, an IS NULL check is performed, and the corresponding SQL condition is determined; for uniqueness check types, the corresponding field to be checked is identified, a HAVING check is performed, and the corresponding SQL condition is determined; for size check types, the corresponding field to be checked is identified, a comparison check between field 1 and field 2 is performed, and the corresponding SQL condition is determined; for numeric precision check types, the corresponding field to be checked is identified, precision is identified, and the corresponding SQL condition is determined; for specified condition check types, the condition is identified, and the corresponding SQL condition is determined.

[0084] S405: Assemble the SQL condition statements based on the table name of the target data table to be checked, the field name of the field to be checked, and the target SQL conditions to generate a set of SQL condition statements.

[0085] In some implementations, the table name of the target data table to be checked and the field name of the field to be checked can be assembled with the target SQL conditions using SQL keywords. The table name and field name are filled into the corresponding positions in the target SQL conditions to obtain SQL condition statements. Multiple SQL condition statements form a set of SQL condition statements.

[0086] Optionally, the table name of the target data table to be checked, the field names of the fields to be checked, and the SQL conditions can be assembled to obtain the initial SQL condition statement. By obtaining the SQL keywords corresponding to the target check type, the SQL keywords are added to the assembled initial SQL condition statement to obtain the SQL condition statement. Optionally, SQL keywords include, but are not limited to: WHERE, AND, QR, IN, etc.

[0087] S406 generates a set of executable SQL scripts based on the verification configuration file and the set of SQL conditional statements.

[0088] S407 executes a set of SQL scripts on the database to be checked to obtain the data check results of the database to be checked.

[0089] The relevant content of steps S406-S407 can be found in the above embodiments, and will not be repeated here.

[0090] In the data verification method provided in this disclosure, a verification configuration file is parsed to obtain the parsing result. A set of SQL conditional statements is generated based on the data table to be verified, the fields to be verified, and the corresponding verification types. Then, according to the verification configuration file, the set of SQL conditional statements is used to generate a set of executable SQL scripts to improve the accuracy of data verification. The set of SQL scripts is executed on the database to be verified to obtain the data verification result. This data verification method can automatically verify the database to be verified, improving the efficiency of data verification, saving labor costs, and effectively shortening the data verification time.

[0091] Figure 5 This is a flowchart of the check rule parser. It reads and parses the check configuration file, identifying the database tables and field names to be checked based on the parsing results. Depending on the check type, it determines the SQL conditions. It then assembles the table names, field names, and SQL conditions with SQL keywords (such as WHERE, AND, etc.) to obtain the SQL conditional statement, and finally generates a set of SQL statement conditions.

[0092] Figure 6 A flowchart illustrating another data verification method provided in this disclosure embodiment. (For example...) Figure 6 As shown, the data verification method includes:

[0093] S601, Obtain the inspection configuration file of the database to be inspected.

[0094] S602 parses the verification configuration file and generates a set of SQL conditional statements based on the parsing results.

[0095] The relevant content of steps S601-S602 can be found in the above embodiments, and will not be repeated here.

[0096] S603, based on the inspection configuration file, obtains the identifier and result prompt information of the target inspection type corresponding to each field to be inspected.

[0097] In some implementations, the identification information of the verification type, the target verification type corresponding to each field to be verified, and the verification result prompt information can be obtained by reading the verification configuration file. Optionally, the identification information of the verification type can be the verification type number. For example, the number of non-empty verification is 1, the number of uniqueness verification is 2, and the number of value range verification is 3. The identification information of a confirmed non-empty verification is 1, the identification information of a uniqueness verification is 2, and the identification information of a value range verification is 3. Optionally, the result prompt information is the verification result of the field to be verified under the corresponding verification type.

[0098] For example, suppose the field check type is non-empty check, its identification information is 1, and the check result is that the field is empty, indicating that the field failed the non-empty check. Then, a result prompt message is generated, such as "Non-empty check: 1, the field to be checked is empty".

[0099] S604 reads the SQL condition statement for each field to be checked from the SQL condition statement set.

[0100] In some implementations, based on the identification information of the check type, the SQL condition statement for each field to be checked is read from the SQL condition statement set, and the identification of the check type corresponding to each SQL condition statement is determined.

[0101] S605: For each field to be checked, the identifier of the target check type and the result prompt information corresponding to the field to be checked are reassembled with the SQL condition statement of the field to be checked to obtain an executable SQL statement.

[0102] In some implementations, based on the result prompts of the fields to be checked, the check result of each field under the current check type can be determined. Based on the check result, the identifier of the target check type corresponding to the field to be checked is reassembled with the SQL condition statement of the field to be checked. That is, the check type identifier is re-matched with the SQL condition statement of the field to be checked to obtain an executable SQL statement.

[0103] For example, if the identifier of the target check type corresponding to any field A to be checked is a non-empty check, and the check result fails, then the check type corresponding to the SQL condition statement of field A is re-determined. Optionally, the check type identifier 2 is assembled with the SQL condition statement of field A to obtain an executable SQL statement.

[0104] S606 configures the environment variables and database connection operator of the database to be checked for executable SQL statements to generate a set of executable SQL scripts.

[0105] In some implementations, configuring the environment variables and database connection operators of the database to be checked into the executable SQL statements can generate a set of executable SQL scripts. This enables automated data checking, reduces manual costs, and improves the efficiency of data checking.

[0106] Optionally, the environment variables of the database to be checked include the database identifier, database type, database name, etc.; the database connection specifier includes the access address, port, user, etc.

[0107] S607 executes a set of SQL scripts on the database to be checked to obtain the data check results of the database to be checked.

[0108] The details of step S607 can be found in the above embodiments and will not be repeated here.

[0109] In the data verification method provided in this disclosure, a verification configuration file is parsed to obtain the parsing result. A set of SQL conditional statements is generated based on the data table to be verified, the fields to be verified, and the corresponding verification types. Then, according to the verification configuration file, the set of SQL conditional statements is used to generate a set of executable SQL scripts to improve the accuracy of data verification. The set of SQL scripts is executed on the database to be verified to obtain the data verification result. This data verification method can automatically verify the database to be verified, improving the efficiency of data verification, saving labor costs, and effectively shortening the data verification time.

[0110] Figure 7 This is a flowchart for generating an executable script. For example... Figure 7 As shown, the system obtains the target check type identifier and result prompt information for each field to be checked by reading the check configuration file. It then obtains the SQL condition statements for each field by reading the SQL condition statement set. Based on the check type identifier and result prompt information, the SQL condition statements are reassembled to generate executable SQL statements. Furthermore, environment variables (database keywords and identifiers) and database connection operators (access address and port) are configured for the executable SQL statements, generating a set of executable SQL scripts.

[0111] Figure 8 A flowchart illustrating another data verification method provided in this disclosure embodiment. (For example...) Figure 8 As shown, the data verification method includes:

[0112] S801, obtain the inspection configuration file of the database to be inspected.

[0113] S802 parses the inspection configuration file and generates a set of structured query language (SQL) conditional statements based on the parsing results.

[0114] S803 generates a set of executable SQL scripts based on the verification configuration file and the set of SQL conditional statements.

[0115] The relevant content of steps S801-S803 can be found in the above embodiments, and will not be repeated here.

[0116] S804, obtain the configuration information of the database to be checked.

[0117] In some implementations, configuration information can be obtained by reading the database to be checked. Optionally, the configuration information of the database to be checked may include information such as the database name, address, port, and username used to connect to the database.

[0118] S805 executes test statements on the database to be checked to determine whether the configuration information of the database to be checked is correct.

[0119] In some implementations, a separate test statement can be written to determine whether the database configuration information is correct. The configuration information is used to connect the database to be checked to the test statement, and the test statement is executed on the database to determine if the configuration information is correct.

[0120] Optionally, the correctness of the configuration information of the database to be checked can be determined by verifying whether a connection to the database can be established.

[0121] S806, when the configuration information is correct, connects to the database to be checked based on the database connection operator, executes a set of SQL scripts on the database to be checked, and obtains the data check results of the database to be checked.

[0122] In some implementations, when the test statement determines that the configuration information of the database to be checked is correct, it connects the set of executable SQL scripts to the database to be checked based on the database connection operator, executes the set of SQL scripts on the database to be checked, and records the data check results of the database to be checked.

[0123] It should be noted that the login script command in the executable SQL script set can verify the correct database connection, allowing subsequent execution of the SQL script set on the database to be checked. Optionally, a login script command for logging into the database can be added to the executable SQL script set and executed to perform the login operation on the database to be checked. Optionally, the login script command can be added at the beginning of the executable SQL script set to ensure that subsequent executable SQL scripts execute with the correct database connection and avoid connection problems.

[0124] In some implementations, after obtaining the data verification results from the database to be checked, the data verification results can be formatted to obtain formatted verification results, which are then stored in a verification table. Storing the data verification results in a verification table makes it easy to view the results for each check, which is beneficial for the analysis of the results. Formatted verification results can generate visual verification results, helping to intuitively understand the data verification results. Formatted verification results can support automated data processing, reduce the workload of data verification, and effectively shorten the data verification time.

[0125] In the data verification method provided in this disclosure, a verification configuration file is parsed to obtain the parsing result. A set of SQL conditional statements is generated based on the data table to be verified, the fields to be verified, and the corresponding verification types. Then, according to the verification configuration file, the set of SQL conditional statements is used to generate a set of executable SQL scripts to improve the accuracy of data verification. The set of SQL scripts is executed on the database to be verified to obtain the data verification result. This data verification method can automatically verify the database to be verified, improving the efficiency of data verification, saving labor costs, and effectively shortening the data verification time.

[0126] Figure 9 This is a flowchart of the script execution process. For example... Figure 9 As shown, the database configuration information and executable SQL script set are read. Based on the database configuration information, independent test statements are used to verify whether the database can connect to and execute the executable SQL script set. A database connection operator is then used to connect to the database to be checked, and the SQL script set is executed on the database to be checked to obtain the data check results.

[0127] Corresponding to the data verification methods provided in the above embodiments, one embodiment of this disclosure also provides a data verification device. Since the data verification device provided in this disclosure corresponds to the data verification methods provided in the above embodiments, the implementation methods of the above data verification methods are also applicable to the data verification device provided in this disclosure, and will not be described in detail in the following embodiments.

[0128] Figure 10 This is a schematic diagram of the data verification device provided in an embodiment of this disclosure. Figure 10 As shown, the data verification device includes: an acquisition module 1001, a first generation module 1002, a second generation module 1003, and an execution module 1004.

[0129] The acquisition module 1001 is used to acquire the inspection configuration file of the database to be inspected.

[0130] The first generation module 1002 is used to parse the inspection configuration file and generate a set of structured query language (SQL) conditional statements based on the parsing results.

[0131] The second generation module 1003 is used to generate a set of executable SQL scripts based on the verification configuration file and the set of SQL conditional statements.

[0132] Execution module 1004 is used to execute a set of SQL scripts on the database to be checked and obtain the data check results of the database to be checked.

[0133] According to one embodiment of this disclosure, the first generation module 1002 is further configured to: parse the inspection configuration file, obtain the inspection data table in the inspection database, the inspection fields in the inspection data table and the inspection types corresponding to the inspection fields; and obtain a set of SQL conditional statements based on the inspection data table, the inspection fields and the corresponding inspection types.

[0134] According to one embodiment of this disclosure, the first generation module 1002 is further configured to: for each field to be checked, obtain the table name of the target data table to which the field to be checked belongs from the data table to be checked; determine the target SQL condition corresponding to the target check type according to the target check type corresponding to the field to be checked; and assemble the SQL condition statement according to the table name of the target data table to be checked, the field name of the field to be checked, and the target SQL condition to generate a set of SQL condition statements.

[0135] According to one embodiment of this disclosure, the first generation module 1002 is further configured to: assemble the table name of the target data table to be inspected, the field name of the target field to be inspected, and the SQL conditions; obtain the SQL keywords corresponding to the target inspection type; and add the SQL keywords to the assembled SQL initial condition statement to obtain the SQL condition statement.

[0136] According to one embodiment of this disclosure, the second generation module 1003 is further configured to: obtain the identifier and result prompt information of the target inspection type corresponding to each field to be inspected based on the inspection configuration file; read the SQL condition statement of each field to be inspected from the SQL condition statement set; for each field to be inspected, reassemble the identifier and result prompt information of the target inspection type corresponding to the field to be inspected with the SQL condition statement of the field to be inspected to obtain an executable SQL statement; configure the environment variables and database connection operator of the database to be inspected for the executable SQL statement to generate an executable SQL script set.

[0137] According to one embodiment of this disclosure, the execution module 1004 is further configured to: obtain configuration information of the database to be checked; execute test statements on the database to be checked to determine whether the configuration information of the database to be checked is correct; when the configuration information is correct, connect to the database to be checked based on the database connection operator, and execute a set of SQL scripts on the database to be checked to obtain the data check result of the database to be checked.

[0138] According to one embodiment of this disclosure, the execution module 1004 is further configured to: add a login script command for logging into the database to the executable SQL script set, and execute the login script command to perform a login operation on the database to be checked.

[0139] According to one embodiment of this disclosure, the execution module 1004 is further configured to: convert the data check result into a format to obtain a formatted check result, and store it in a check table.

[0140] In the data verification apparatus provided in this embodiment, a set of SQL conditional statements is generated by parsing a verification configuration file. Based on the parsing results, the set of SQL conditional statements is then used to generate a set of executable SQL scripts, thereby improving the accuracy of data verification. The SQL script set is then executed on the database to be verified to obtain the data verification results. This data verification method can automatically verify the database to be verified, improving verification efficiency, saving labor costs, and effectively shortening verification time.

[0141] Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Figure 11 As shown, the electronic device 1100 may include: a transceiver 1101, a processor 1102, and a memory 1103.

[0142] Processor 1102 executes computer execution instructions stored in memory, causing processor 1102 to perform the scheme in the above embodiments. Processor 1102 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0143] The memory 1103 is connected to the processor 1102 via the system bus and completes communication between them. The memory 1103 is used to store computer program instructions.

[0144] Transceiver 1101 can be used to obtain the task to be run and the configuration information of the task to be run.

[0145] The system bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The system bus can be divided into address bus, data bus, control bus, etc. For ease of representation, only one thick line is used in the diagram, but this does not indicate that there is only one bus or one type of bus. Transceivers are used to enable communication between database access devices and other computers (e.g., clients, read-write libraries, and read-only libraries). Memory may include random access memory (RAM) and may also include non-volatile memory.

[0146] The electronic device provided in this disclosure can be the terminal device described in the above embodiments.

[0147] This disclosure also provides a chip for executing instructions, which is used to execute the data verification method described in the above embodiments.

[0148] This disclosure also provides a computer-readable storage medium storing computer instructions that, when executed on a computer, cause the computer to perform the data verification method described in the above embodiments.

[0149] This disclosure also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium, and when the at least one processor executes the computer program, it can implement the technical solution of the data verification method in the above embodiments.

[0150] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0151] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A data verification method, characterized in that, include: Obtain the inspection configuration file for the database to be inspected; The inspection configuration file is parsed to obtain the inspection data table in the inspection database, as well as the inspection fields and the inspection types corresponding to the inspection fields in the inspection data table; For each field to be checked, the table name and field name of the target data table to which the field to be checked belong are identified based on the parsing results, and a framework of SQL conditional statements is created based on the table name and field name. Based on the target check type corresponding to the field to be checked, the target SQL conditions corresponding to the target check type are determined, specifically including: if the target check type is a non-null check type, the corresponding field to be checked is identified, an IS NULL check is performed, and the SQL conditions corresponding to the non-null check type are determined; if the target check type is a unique check type, the corresponding field to be checked is identified, a HAVING check is performed, and the SQL conditions corresponding to the unique check type are determined; if the target check type is a size check type, the corresponding field to be checked is identified, a size comparison check of field 1 and field 2 is performed, and the SQL conditions corresponding to the size check type are determined; if the target check type is a numerical precision check type, the corresponding field to be checked is identified, precision identification is performed, and the SQL conditions corresponding to the numerical precision check type are determined; if the target check type is a specified condition check type, the condition is identified, and the corresponding SQL conditions are determined. The target SQL conditions, table name, and field name are filled into the framework of the SQL condition statement to obtain the assembled SQL condition statement, thereby generating a set of structured query language SQL condition statements. Based on the inspection configuration file, obtain the identifier of the target inspection type and the result prompt information corresponding to each field to be inspected; Read the SQL condition statement for each field to be checked from the set of SQL condition statements; For each of the fields to be checked, the identifier and result prompt information of the target check type corresponding to the field to be checked are reassembled with the SQL condition statement of the field to be checked to obtain an executable SQL statement; Configure the environment variables and database connection specifier of the database to be inspected for the executable SQL statement to generate a set of executable SQL scripts; wherein, the environment variables of the database to be inspected include the database identifier, database type, and database name, and the database connection specifier includes the access address, port, and user; Obtain the configuration information of the database to be inspected; Execute test statements on the database to be inspected to determine whether the configuration information of the database to be inspected is correct; When the configuration information is correct, connect to the database to be checked based on the database connection operator; Add a login script command for logging into the database to the set of executable SQL scripts, and execute the login script command to perform a login operation on the database to be checked; The SQL script set is executed on the database to be checked to obtain the data check results of the database to be checked.

2. The method according to claim 1, characterized in that... Based on the target SQL conditions and the table and field names, the assembled SQL conditional statement is obtained, including: Assemble the table name of the target data table to be checked, the field name of the field to be checked, and the SQL conditions; Obtain the SQL keyword corresponding to the target inspection type; Add the SQL keyword to the assembled SQL initial condition statement to obtain the SQL condition statement.

3. The method according to any one of claims 1-2, characterized in that, After executing the SQL script set on the database to be inspected to obtain the data inspection results of the database to be inspected, the process further includes: The data verification results are formatted to obtain formatted verification results, which are then stored in the verification table.

4. A data verification device, characterized in that, include: The acquisition module is used to acquire the inspection configuration file of the database to be inspected; The first generation module is used to parse the inspection configuration file, obtain the inspection data table in the inspection database, the inspection fields in the inspection data table and the inspection types corresponding to the inspection fields; for each inspection field, the module identifies the table name and field name of the target inspection data table to which the inspection field belongs based on the parsing result, and creates a framework of SQL conditional statements based on the table name and field name; Based on the target check type corresponding to the field to be checked, the target SQL condition corresponding to the target check type is determined, specifically including: if the target check type is a non-null check type, the corresponding field to be checked is identified, an IS NULL check is performed, and the SQL condition corresponding to the non-null check type is determined; if the target check type is a unique check type, the corresponding field to be checked is identified, a HAVING check is performed, and the SQL condition corresponding to the unique check type is determined; if the target check type is a size check type, the corresponding field to be checked is identified, a size comparison check of field 1 and field 2 is performed, and the SQL condition corresponding to the size check type is determined; if the target check type is a numerical precision check type, the corresponding field to be checked is identified, precision identification is performed, and the SQL condition corresponding to the numerical precision check type is determined; if the target check type is a specified condition check type, the condition is identified, and the corresponding SQL condition is determined; the target SQL condition, the table name, and the field name are filled into the framework of the SQL condition statement to obtain the assembled SQL condition statement, thereby generating a set of Structured Query Language (SQL) condition statements; The second generation module is used to obtain the identifier and result prompt information of the target inspection type corresponding to each field to be inspected based on the inspection configuration file; read the SQL condition statement of each field to be inspected from the SQL condition statement set; for each field to be inspected, reassemble the identifier and result prompt information of the target inspection type corresponding to the field to be inspected with the SQL condition statement of the field to be inspected to obtain an executable SQL statement; configure the environment variables and database connection operator of the database to be inspected for the executable SQL statement to generate an executable SQL script set; wherein, the environment variables of the database to be inspected include the database identifier, the database type, and the database name, and the database connection operator includes the access address, port, and user; The execution module is used to obtain the configuration information of the database to be inspected; execute test statements on the database to be inspected to determine whether the configuration information of the database to be inspected is correct; if the configuration information is correct, connect to the database to be inspected based on the database connection operator; add a login script command for logging into the database to the executable SQL script set, and execute the login script command to perform a login operation on the database to be inspected; execute the SQL script set on the database to be inspected to obtain the data inspection result of the database to be inspected.

5. The apparatus according to claim 4, characterized in that, The first generation module is further configured to: Assemble the table name of the target data table to be checked, the field name of the field to be checked, and the SQL conditions; Obtain the SQL keyword corresponding to the target inspection type; Add the SQL keyword to the assembled SQL initial condition statement to obtain the SQL condition statement.

6. The apparatus according to any one of claims 4-5, characterized in that, The execution module is further configured to: The data verification results are formatted to obtain formatted verification results, which are then stored in the verification table.

7. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1-3.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-3.

9. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method of any one of claims 1-3.

Images