Data inspection method, inspection device, terminal equipment and readable storage medium

Abstract

The present application is applicable to the field of data processing technology, and provides a data inspection method, inspection device, terminal equipment and readable storage medium, the method comprising: running a target quality inspection relationship to obtain a first data inspection result; matching a first data identifier in the first data inspection result with a target data identifier in a target record table to determine first data, the first data identifier of the first data matching the target data identifier; removing the first data from the first data inspection result to obtain a second data inspection result. The present application can solve the problem of low efficiency of the data inspection method to a certain extent.

Description

Technical Field

[0001] The present application relates to the field of data processing, and in particular to a data inspection method, an inspection device, a terminal device and a readable storage medium. Background Art

[0002] With the development of Internet technology, there are more and more Internet-based businesses, and the data stored in business databases is also increasing.

[0003] The data stored in the business database usually needs to be checked. The current method for checking the data stored in the business database is: formulate quality inspection relationships according to business rules, then run the quality inspection relationships, filter out the data that does not meet the business rules, and finally correct the data that does not meet the business rules.

[0004] However, there are some special data in the data that do not conform to the business rules. Although these special data do not conform to the business rules, they are correct data. Therefore, after each operation of the quality inspection relationship, if there are special data in the data that do not conform to the business rules, the processing logic of the quality inspection relationship needs to be modified.

[0005] Frequent modification of the processing logic of the quality inspection relationship makes the quality inspection relationship complicated, thereby reducing the efficiency of data inspection. Therefore, the efficiency of the current data inspection method is low. Summary of the invention

[0006] The embodiments of the present application provide a data checking method, a checking device, a terminal device and a readable storage medium, which can solve the problem of low efficiency of current data checking methods to a certain extent.

[0007] In a first aspect, an embodiment of the present application provides a data checking method, comprising:

[0008] Run the target quality inspection relationship to obtain the first data inspection result;

[0009] Matching the first data identifier in the first data check result with the target data identifier in the target record table to determine the first data, wherein the first data identifier of the first data matches the target data identifier;

[0010] The first data is removed from the first data check result to obtain a second data check result.

[0011] In a second aspect, an embodiment of the present application provides a data checking device, including:

[0012] An operation module, used for operating the target quality inspection relationship to obtain a first data inspection result;

[0013] A matching module, used for matching the first data identifier in the first data check result with the target data identifier in the target record table to determine the first data, wherein the first data identifier of the first data matches the target data identifier;

[0014] The elimination module is used to eliminate the first data from the first data inspection result to obtain a second data inspection result.

[0015] In a third aspect, an embodiment of the present application provides a terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method of the first aspect when executing the computer program.

[0016] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method of the first aspect are implemented.

[0017] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when executed on a terminal device, enables the terminal device to execute the data checking method described in any one of the above-mentioned first aspects.

[0018] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here.

[0019] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0020] The present application provides a data inspection method. First, a target quality inspection relationship is run to obtain a first data inspection result. Then, the first data identifier in the first data inspection result is matched with the target data identifier in the target record table to determine the first data (the first data refers to data that does not conform to the business rules but is correct), and the first data identifier of the first data matches the target data identifier. Finally, the first data is removed from the first data inspection result to obtain a second data inspection result. That is, in the present application, a target record table is first set, and then the first data identifier in the first data inspection result is matched with the target data identifier in the target record table. If the first data identifier matches the target data identifier, it means that the first data corresponding to the first data identifier is special data, and then the first data corresponding to the first data identifier is removed from the first data inspection result, thereby reducing the number of special data in the second data inspection result, thereby improving the data inspection efficiency. Therefore, in the present application, there is no need to modify the processing logic of the target quality inspection relationship, and the target quality inspection relationship does not need to be complicated, and the number of special data in the second data inspection result can also be reduced, thereby improving the data inspection efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0022] Figure 1 It is a flowchart of a data checking method provided by an embodiment of the present application;

[0023] Figure 2 is a structural schematic diagram of a data checking device provided by an embodiment of the present application;

[0024] Figure 3 It is a schematic diagram of the structure of the terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0026] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0027] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0028] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0029] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0030] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0031] The data checking method provided in the embodiment of the present application can be applied to terminal devices such as mobile phones, tablet computers, laptop computers, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), etc. The embodiment of the present application does not impose any restrictions on the specific type of terminal devices.

[0032] In order to illustrate the technical solution described in this application, a specific embodiment is provided below for illustration.

[0033] Embodiment 1

[0034] The following is a description of a data checking method provided in Example 1 of the present application. Figure 1 , the method comprising:

[0035] Step S101, run the target quality inspection relationship to obtain a first data inspection result.

[0036] In step S101, the target quality inspection relationship refers to a structured query language (SQL) statement written based on business rules for detecting whether the data in the relational database complies with the business rules. When it is necessary to inspect the data in the database, the terminal device runs the target quality inspection relationship to obtain a first data inspection result.

[0037] Step S102: Match the first data identifier in the first data check result with the target data identifier in the target record table to determine the first data, and the first data identifier of the first data matches the target data identifier.

[0038] In step S102, the first data refers to special data (special data refers to data that does not conform to business rules but is correct). The data identifier of the data that does not conform to business rules but is correct is recorded in the target record table as the target data identifier. After obtaining the first data check result, the terminal device matches the first data identifier in the first data check result with the target data identifier. If the first data identifier matches the target data identifier, it means that the first data corresponding to the first data identifier is special data.

[0039] The target record table refers to a table used to store relevant information of special data. The relevant information of special data includes the identifier of the business corresponding to the special data, the target data identifier, the target quality inspection relationship identifier, the target record reason and the target time, etc. It should be understood that the target record table can be generated according to the detection table of the business. Alternatively, the terminal device can also generate the target record table by itself, and then associate the self-generated target record table with the business. The user can choose the generation method of the corresponding target record table according to the actual situation, and this application does not limit it here.

[0040] Step S103: remove the first data from the first data check result to obtain a second data check result.

[0041] In step S103, after determining the first data, since the first data is special data, the terminal device removes the first data from the first data check result, thereby obtaining a second data check result.

[0042] In some embodiments, the method also includes: obtaining a manual inspection result corresponding to the second data inspection result; determining the data type in the second data inspection result based on the manual inspection result; if there is special data with a data type of the first type in the second data inspection result, recording the data identifier corresponding to the special data in the target record table, and returning the running target quality inspection relationship.

[0043] When special data appears for the first time, the data identifier of the special data has not been recorded in the target record table. At this time, the special data cannot be removed from the first data check result. Therefore, the second data check result may still contain special data. Therefore, it is necessary to manually identify whether the data is special data.

[0044] Therefore, after the terminal device obtains the second data inspection result, it is necessary to manually inspect the second data inspection result to obtain the manual inspection result corresponding to the second data inspection result. Then the terminal device determines the data type in the second data inspection result based on the manual inspection result. If there is special data in the second data inspection result, the terminal device records the data identifier corresponding to the special data as the target data identifier in the target record table, so that when the terminal device runs the target quality inspection relationship again, the special data will not appear in the second data inspection result.

[0045] It should be understood that if there is no corresponding target record table at this time, it is necessary to first generate the target record table, and then record the data identifier corresponding to the special data in the target record table.

[0046] In some other embodiments, the method further includes: if the second data check result contains erroneous data of the second type, correcting the erroneous data to obtain corrected data.

[0047] In this embodiment, if there is erroneous data in the second data check result, the terminal device corrects the erroneous data to obtain modified data, so that when the terminal device runs the target quality inspection relationship again, the erroneous data will no longer appear in the second data check result.

[0048] In some other embodiments, after recording the data identifier corresponding to the special data in the target record table, the method further includes: adding a target record reason corresponding to the special data in the target record table.

[0049] When special data is removed, if the user needs to check the reason why the special data is removed, the user needs to repeatedly check the check result, thereby reducing the data inspection efficiency. Therefore, in this embodiment, after recording the data identifier corresponding to the special data in the target record table, the terminal device can also record the target record reason corresponding to the special data in the target record table for the user to query, so that the user does not need to repeatedly check the special data that has been removed but the reason for removal is unclear, thereby improving the data inspection efficiency. The target record reason refers to the reason why the special data is removed.

[0050] In other embodiments, the method further includes: receiving a query instruction; searching the target record table for first special data corresponding to the query instruction; and outputting a target data identifier corresponding to the first special data and a target record reason corresponding to the first special data.

[0051] This embodiment also includes a query function so that the user can query special data and the target record reason of the special data. When the user needs to query, the user sends a query instruction to the terminal device. After receiving the query instruction, the terminal device searches for the first special data corresponding to the query instruction in the target record table, and then outputs the target data identifier corresponding to the first data and the target record reason corresponding to the first special data.

[0052] It should be understood that the query instruction may also include the first quality inspection relationship. After receiving the query instruction, the terminal device searches the target record table for the first quality inspection relationship corresponding to the query instruction, and then outputs all special data corresponding to the first quality inspection relationship and the target record reasons corresponding to all these special data.

[0053] In summary, the present application provides a data inspection method. First, the target quality inspection relationship is run to obtain a first data inspection result. Then the first data identifier in the first data inspection result is matched with the target data identifier in the target record table to determine the first data (the first data refers to the data that does not meet the business rules but is correct), and the first data identifier of the first data matches the target data identifier. Finally, the first data is removed from the first data inspection result to obtain the second data inspection result. That is, in the present application, a target record table is set, and then the first data identifier in the first data inspection result is matched with the target data identifier in the target record table. If the first data identifier matches the target data identifier, it means that the first data corresponding to the first data identifier is special data, and then the first data corresponding to the first data identifier is removed from the first data inspection result, thereby reducing the number of special data in the second data inspection result, thereby improving the data inspection efficiency. Therefore, in the present application, there is no need to modify the processing logic of the target quality inspection relationship, and the target quality inspection relationship does not need to be complicated, and the number of special data in the second data inspection result can also be reduced, thereby improving the data inspection efficiency.

[0054] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0055] Embodiment 2

[0056] Figure 2 An example of a data inspection device is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown. The device 200 includes:

[0057] The operation module 201 is used to operate the target quality inspection relationship to obtain a first data inspection result.

[0058] The matching module 202 is used to match the first data identifier in the first data check result with the target data identifier in the target record table to determine the first data, and the first data identifier of the first data matches the target data identifier.

[0059] The elimination module 203 is used to eliminate the first data from the first data inspection result to obtain the second data inspection result.

[0060] Optionally, the device 200 further includes:

[0061] The acquisition module is used to receive the manual inspection result corresponding to the second data inspection result.

[0062] The determination module is used to determine the data type in the second data inspection result according to the manual inspection result.

[0063] The recording module is used to record the data identifier corresponding to the special data in the target recording table if there is special data of the first type in the second data inspection result, and return the running target quality inspection relationship.

[0064] Optionally, the device 200 further includes:

[0065] The correction module is used to correct the erroneous data to obtain corrected data if the second data check result contains erroneous data of the second type.

[0066] Optionally, the device 200 further includes:

[0067] The reason recording module is used to add the target recording reasons corresponding to the special data in the target recording table.

[0068] Optionally, the device 200 further includes:

[0069] The receiving module is used to receive the query instruction.

[0070] The search module is used to search the target record table for the first special data corresponding to the query instruction.

[0071] The output module is used to output the target data identifier corresponding to the first special data and the target record reason corresponding to the first special data.

[0072] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the first embodiment of the method of the present application. Their specific functions and technical effects can be found in the part of the method embodiment and will not be repeated here.

[0073] Embodiment 3

[0074] Figure 3Schematic diagram of the terminal device provided in the third embodiment of the present application. Figure 3 As shown, the terminal device 300 of this embodiment includes: a processor 301, a memory 302, and a computer program 303 stored in the memory 302 and executable on the processor 301. When the processor 301 executes the computer program 303, the steps in the above-mentioned method embodiments are implemented. Alternatively, when the processor 301 executes the computer program 303, the functions of the modules / units in the above-mentioned device embodiments are implemented.

[0075] Exemplarily, the computer program 303 may be divided into one or more modules / units, which are stored in the memory 302 and executed by the processor 301 to complete the present application. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program 303 in the terminal device 300. For example, the computer program 303 may be divided into a running module, a matching module, and a rejection module, and the specific functions of each module are as follows:

[0076] Run the target quality inspection relationship to obtain the first data inspection result;

[0077] Matching the first data identifier in the first data check result with the target data identifier in the target record table to determine the first data, wherein the first data identifier of the first data matches the target data identifier;

[0078] The first data is removed from the first data checking result to obtain a second data checking result.

[0079] The terminal device may include, but is not limited to, a processor 301 and a memory 302. Those skilled in the art will appreciate that Figure 3 It is only an example of the terminal device 300 and does not constitute a limitation of the terminal device 300. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the above-mentioned terminal device may also include input and output devices, network access devices, buses, etc.

[0080] The processor 301 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware plug-ins, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0081] The memory 302 may be an internal storage unit of the terminal device 300, such as a hard disk or memory of the terminal device 300. The memory 302 may also be an external storage device of the terminal device 300, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device 300. Further, the memory 302 may also include both an internal storage unit of the terminal device 300 and an external storage device. The memory 302 is used to store the computer program and other programs and data required by the terminal device. The memory 302 may also be used to temporarily store data that has been output or is to be output.

[0082] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the above-mentioned device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0083] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0084] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0085] In the embodiments provided in the present application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are only schematic. For example, the division of the above modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or plug-ins can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0086] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0087] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0088] If the above-mentioned integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned various method embodiments, and can also be completed by instructing the relevant hardware through a computer program. The above-mentioned computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the above-mentioned computer program includes computer program code, and the above-mentioned computer program code can be in source code form, object code form, executable file or some intermediate form. The above-mentioned computer-readable medium may include: any entity or device capable of carrying the above-mentioned computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the above-mentioned computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0089] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some of the technical features therein by equivalents. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A data checking method, characterized in that: include: Running the target quality inspection relationship to obtain a first data inspection result, wherein the target quality inspection relationship refers to a structured query language SQL statement written based on a business rule and used to detect whether the data in the relational database complies with the business rule. When it is necessary to inspect the data in the database, the terminal device runs the target quality inspection relationship to obtain the first data inspection result; Matching the first data identifier in the first data check result with the target data identifier in the target record table to determine the first data, the first data identifier of the first data matches the target data identifier, the first data is special data that does not conform to the business rules, and recording the data identifier of the correct data that does not conform to the business rules as the target data identifier in the target record table, the target record table refers to a table for storing relevant information of special data, and the relevant information of the special data includes the identifier of the business corresponding to the special data, the target data identifier, the target quality inspection relationship identifier, the target record reason and the target time; Eliminate the first data from the first data check result to obtain a second data check result; The method further comprises: Obtaining a manual inspection result corresponding to the second data inspection result; Determining the data type in the second data inspection result according to the manual inspection result; If there is special data of the first type in the second data inspection result, the data identifier corresponding to the special data is recorded in the target record table, and the running target quality inspection relationship is returned.

2. The data checking method according to claim 1, characterized in that: Also includes: If the second data check result contains erroneous data of the second type, the erroneous data is corrected to obtain corrected data.

3. The data checking method according to claim 1, characterized in that: After recording the data identifier corresponding to the special data in the target record table, the method further includes: Add the target record reason corresponding to the special data in the target record table.

4. The data checking method according to claim 3, characterized in that: Also includes: Receive query instructions; Searching the target record table for first special data corresponding to the query instruction; Output the target data identifier corresponding to the first special data and the target record reason corresponding to the first special data.

5. A data checking device, characterized in that: include: An operation module is used to operate the target quality inspection relationship to obtain a first data inspection result, wherein the target quality inspection relationship refers to a structured query language SQL statement written based on a business rule and used to detect whether the data in the relational database complies with the business rule. When it is necessary to inspect the data in the database, the terminal device operates the target quality inspection relationship to obtain the first data inspection result; A matching module, used for matching the first data identifier in the first data inspection result with the target data identifier in the target record table, determining the first data, the first data identifier of the first data matches the target data identifier, the first data is special data that does not conform to the business rules, and recording the data identifier of the correct data that does not conform to the business rules as the target data identifier in the target record table, the target record table refers to a table for storing relevant information of special data, and the relevant information of the special data includes the identifier of the business corresponding to the special data, the target data identifier, the target quality inspection relationship identifier, the target record reason and the target time; A removal module, used for removing the first data from the first data inspection result to obtain a second data inspection result; The device also includes: An acquisition module, configured to receive a manual inspection result corresponding to the second data inspection result; A determination module, configured to determine a data type in the second data inspection result according to the manual inspection result; A recording module is used to record the data identifier corresponding to the special data in the target record table if there is special data of the first type in the second data inspection result, and return the running target quality inspection relationship.

6. The data inspection device according to claim 5, characterized in that: Also includes: The correction module is used to correct the erroneous data to obtain corrected data if the second data check result contains erroneous data of the second type.

7. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 4 is implemented.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images