Data quality detection method and device, electronic equipment, medium and program product
By determining and replacing rule templates from the preset rule template library and generating data quality detection rules, the problems of large workload and high error rate in the prior art are solved, and detection efficiency and accuracy are improved.
Patent Information
- Application Number
- CN202410944285.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-15
- Publication Date
- 2024-11-08
AI Technical Summary
In the prior art, the data quality detection rules are determined by writing a large amount of code, resulting in large development workload and high error probability, which affects the accuracy and reliability of the data.
Provide a data quality detection method, by determining the target rule template matching the detection requirements from the preset rule template library, replacing the template identification information as data identification information, generating quality detection rules, and directly used for data quality detection.
It improves the efficiency of data quality detection, reduces the error rate, ensures the accuracy and reliability of the data, and avoids the process of writing a large amount of code.
Smart Images

Figure CN118916349A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing technology, and in particular to a data quality detection method, device, electronic equipment, medium and program product. Background Art
[0002] With the development of the information age and the advancement of information digital transformation, the amount of data in major enterprises and organizations has shown an exponential growth trend. At present, when analyzing massive and diverse data, the data needs to meet quality requirements in terms of accuracy and completeness, so the need for data quality testing has emerged.
[0003] In the prior art, a series of data quality detection rules are defined through a large amount of code writing work to evaluate data quality.
[0004] However, determining data quality detection rules by writing a large amount of code will result in a high probability of errors due to the large amount of development work, thus affecting the accuracy and reliability of the data. Summary of the invention
[0005] The present application provides a data quality detection method, device, electronic device, medium and program product to solve the problem in the prior art that determining data quality detection rules by writing a large amount of code will result in a high probability of error due to the large development workload, thereby affecting the accuracy and reliability of the data.
[0006] In a first aspect, the present application provides a data quality detection method, the method comprising:
[0007] According to the detection requirements of the data to be detected, determine a target rule template matching the detection requirements from a preset rule template library; wherein the target rule template includes a plurality of template identification information;
[0008] Determine the information content of multiple data identification information of the data to be detected; wherein there is a corresponding relationship between the data identification information and the template identification information;
[0009] Replacing each template identification information in the target rule template with the information content of the corresponding data identification information to obtain the quality detection rule of the data to be detected;
[0010] The data to be detected is detected according to the quality detection rule to obtain a data quality detection result of the data to be detected.
[0011] In a second aspect, the present application provides a data quality detection device, the device comprising:
[0012] A rule template determination module is used to determine a target rule template matching the detection requirement from a preset rule template library according to the detection requirement of the data to be detected; wherein the target rule template includes a plurality of template identification information;
[0013] An information content determination module, used to determine the information content of a plurality of data identification information of the data to be detected; wherein there is a corresponding relationship between the data identification information and the template identification information;
[0014] A replacement module, used to replace each template identification information in the target rule template with the information content of the corresponding data identification information to obtain the quality detection rule of the data to be detected;
[0015] The detection module is used to detect the data to be detected according to the quality detection rule to obtain a data quality detection result of the data to be detected.
[0016] In a third aspect, the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the data quality detection method as described in any embodiment of the present application is implemented.
[0017] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the data quality detection method as described in any embodiment of the present application.
[0018] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the data quality detection method as described in any embodiment of the present application.
[0019] The scheme of the present application, according to the detection requirements of the data to be detected, determines the target rule template that matches the detection requirements from the preset rule template library; wherein the target rule template includes multiple template identification information; determines the information content of multiple data identification information of the data to be detected; wherein there is a corresponding relationship between the data identification information and the template identification information; replaces each template identification information in the target rule template with the information content of the corresponding data identification information, and obtains the quality detection rule of the data to be detected; detects the data to be detected according to the quality detection rule, and obtains the data quality detection result of the data to be detected. That is, the scheme of the present application, on the one hand, determines the target rule template that matches the detection requirements from the preset rule template library, does not need to write a large number of rule templates by itself, and improves the efficiency of data quality detection. On the other hand, by replacing information in the target rule template to obtain the quality detection rule, it is avoided to write a large amount of code to determine the data quality rule, and reduces the error rate caused by code development work in the generation of data quality rules, thereby improving the efficiency of data quality detection and ensuring the accuracy and reliability of data. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.
[0021] Figure 1 It is a flow chart of the data quality detection method provided by this application;
[0022] Figure 2 is another flow chart of the data quality detection method provided by the present application;
[0023] Figure 3 It is a structural schematic diagram of the data quality detection device provided by the present application;
[0024] Figure 4 It is a structural schematic diagram of the electronic device provided by this application. DETAILED DESCRIPTION
[0025] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.
[0026] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0027] Figure 1 The present invention provides a flow chart of a data quality detection method. The method can be performed by a data quality detection device, which can be implemented in software and / or hardware. In a specific embodiment, the device can be applied to an electronic device, which can be a computer. The following embodiments will be described by taking the application of the device in an electronic device as an example. Figure 1 , the method may specifically include the following steps:
[0028] Step 101 , according to the detection requirements of the data to be detected, a target rule template matching the detection requirements is determined from a preset rule template library.
[0029] The target rule template includes multiple template identification information.
[0030] Specifically, the data to be detected is data that needs to be tested for data quality. The data to be detected may have one data value to be detected, or may include multiple data values to be detected, which is not limited in this application. The detection requirements of the data to be detected are aspects of the data to be detected that are determined according to the data characteristics of the data to be detected. For example, when the data to be detected is a time-sensitive data, the validity of the data to be detected needs to be detected. For example, the detection requirements of the data to be detected include at least one of the following: integrity detection requirements, uniqueness detection requirements, timeliness detection requirements, validity detection requirements, consistency detection requirements, and stability detection requirements. The rule template is a standardized template of the data quality detection rules set according to the detection requirements. Based on the rule template, rules can be generated for data quality detection. The rule template includes a variable part and a quantitative part. By changing the variable part in the rule template, the rule template can be applied to the data to be detected, and the quality of the data to be detected can be detected. The preset rule template library is a template library containing multiple rule templates that are pre-set. The multiple rule templates in the preset rule template library can meet one or more detection requirements of the data to be detected. According to the detection requirements of the data to be detected, a rule template that can meet the detection requirements is determined from a preset rule template library as a target rule template.
[0031] Exemplarily, the data to be tested is the data of a person named A as shown in Table 1, and the data to be tested includes three data values to be tested, including the name, age and registration year of A. The detection requirements of the data to be tested are integrity detection requirements and timeliness detection requirements. According to the detection requirements of the data to be tested being integrity detection requirements and timeliness detection requirements, a rule template matching the integrity detection requirements and timeliness detection requirements is determined from a preset rule template library as a target rule template.
[0032] Table 1
[0033]
[0034]
[0035] Optionally, the preset rule template library includes an integrity rule template sub-library, a uniqueness rule template sub-library, a timeliness rule template sub-library, a validity rule template sub-library, a consistency rule template sub-library and a stability rule template sub-library. The integrity rule template sub-library includes at least one of the following: a field null value check template and a field empty string check template. The uniqueness rule template sub-library includes at least one of the following: a field uniqueness check template, a field grouping number check template and a field duplicate value number check template. The timeliness rule template sub-library includes at least one of the following: a time function comparison template, a single table time field comparison template and a two-table time field comparison template. The validity rule template sub-library includes at least one of the following: a field format check template, a field length check template, a field value range check template, a code table reference comparison template and a data standard code table reference comparison template. The consistency rule template sub-library includes at least one of the following: single table field value consistency comparison template, single table field statistical value consistency comparison template, single table field business logic consistency comparison template, two table field value consistency comparison template, two table field statistical value consistency comparison template, two table field business logic consistency comparison template, and cross-source two table field statistical value consistency comparison template. The stability rule template sub-library includes at least one of the following: table stability verification template, table volatility verification template, field stability verification template, and field volatility verification template.
[0036] Specifically, the preset rule template library includes multiple template sub-libraries to meet multiple detection requirements of the data to be detected, which can cover a variety of business scenarios. There is no need to write a large number of different quality detection rules according to different business scenarios, which improves the efficiency of data quality detection.
[0037] Exemplarily, according to the detection requirements of the data to be detected, which are integrity detection requirements and timeliness detection requirements, a rule template that matches the integrity detection requirements and timeliness detection requirements is determined from a preset rule template library, that is, a field null value check template, a field empty string check template, a time function comparison template, a single-table time field comparison template, and a two-table time field comparison template are determined as target rule templates.
[0038] Optionally, if the templates in the preset rule template library cannot meet the detection requirements, the user equipment of the operator is informed so that the operator can write a new template or a new rule according to the detection requirements.
[0039] Step 102: determine the information content of multiple data identification information of the data to be detected.
[0040] There is a corresponding relationship between the data identification information and the template identification information.
[0041] Specifically, the data to be detected includes multiple data identification information, and each data corresponds to a data identification information. For example, the data identification information corresponding to the name data shown in Table 1 is test1. The information content of the data identification information is the actual content of the data to be detected corresponding to the identification information. For example, the information content of the data identification information of the name data shown in Table 1 is name. The template identification information is the variable part in the template. There is a corresponding relationship between the data identification information and the template identification information, which can make the information content of the data identification information replace the variable part in the template, so that the rule template generates rules for detecting data quality.
[0042] For example, in the data to be tested as shown in Table 1, the data identification information is test1, test2 and test3, the information content of test1 is the name, the information content of test2 is the age, and the information content of test3 is the registration year.
[0043] Step 103: Replace each template identification information in the target rule template with the information content of the corresponding data identification information to obtain the quality detection rule of the data to be detected.
[0044] Specifically, according to the correspondence between the data identification information and the template identification information, each template identification information in the target rule template is replaced with the information content of the corresponding data identification information, so as to obtain the quality detection rule of the data to be detected.
[0045] Exemplarily, the target rule template is to check whether X is less than 30, X corresponds to test2, and therefore the quality detection rule of the data to be detected is to check whether the age is less than 30.
[0046] For example, the rule template is represented by Structured Query Language (SQL). For example, to query the statistical value of a field in a table that is less than 10, the following template can be used: "select count(1)from`${validateTable}`where`${validateField}`<10;", where "`${validateTable}`" is the template identification information, which can be replaced by the table name of a table, and "`${validateField}`" is the template identification information, which can be replaced by the field name of a field. The template generated by SQL can be applied to the data of databases such as relational database management system (Mysql), object-relational database management system (PostgreSQL), Oracle database, Kingbase database, Gauss database, etc.
[0047] Optionally, after generating quality inspection rules for the data to be inspected, if new inspection requirements arise, a new target rule template is selected from a preset rule template library according to the new inspection requirements, and quality inspection rules corresponding to the new inspection requirements are generated accordingly.
[0048] Optionally, there are multiple target rule templates, and step 103 can be implemented through steps 1031 to 1032.
[0049] Step 1031: Replace each template identification information in each target rule template with the information content of the corresponding data identification information to obtain a replaced target rule template.
[0050] Specifically, when there are multiple target rule templates, the template identification information in each target rule template needs to be replaced with the information content of the corresponding data identification information to obtain the replaced target rule template.
[0051] Exemplarily, the target rule template is to check whether X is less than 30, and to check whether Y is 2024. X corresponds to test2, and Y corresponds to test3. The replaced target rule template is to check whether the age is less than 30, and to check whether the registration year is 2024.
[0052] Step 1032: determine the multiple replaced target rule templates as quality detection rules for the data to be detected.
[0053] Specifically, after obtaining a plurality of replaced target rule templates, the plurality of replaced target rule templates are determined as quality detection rules for the data to be detected.
[0054] Exemplarily, multiple replaced target rule templates are to check whether the age is less than 30 and to check whether the registration year is 2024. At this time, the quality detection rules for the data to be detected are to check whether the age is less than 30 and to check whether the registration year is 2024.
[0055] Optionally, after each template identification information in the target rule template is replaced with the information content of the corresponding data identification information and the quality detection rule of the data to be detected is obtained, step 31 may be performed.
[0056] Step 31, determining the data quality qualification standard of the data to be tested according to the quality testing rules.
[0057] Specifically, after obtaining the quality inspection rules for the data to be inspected, the data quality qualification standard for the data to be inspected is determined according to the content of the quality inspection rules. For example, if the degree of satisfaction of the quality inspection rules reaches a preset degree of satisfaction, the data quality can be determined to be qualified. At this time, the data quality qualification standard is that the degree of satisfaction of the quality inspection rules reaches the preset degree of satisfaction.
[0058] Exemplarily, the quality detection rule is to check whether the registration year is 2024, and the determined data quality qualification standard is qualified when the registration year is 2024.
[0059] Step 104: Detect the data to be detected according to the quality detection rules to obtain a data quality detection result of the data to be detected.
[0060] Specifically, after determining the quality detection rules, the data to be detected is detected according to the quality detection rules to obtain the data quality detection results of the data to be detected. The data value corresponding to the information content in the data to be detected is detected according to the quality detection rules, and the data quality detection result is determined according to whether the data value can meet the requirements of the quality detection rules. The data quality detection result can be expressed by whether the quality detection is passed or not, or by the quality satisfaction degree obtained after the data quality detection. The quality satisfaction degree can be reflected by numerical values such as the pass rate and qualified rate obtained after detecting the data to be detected. For example, if the data quality completely meets the quality detection rules, the pass rate is 100%.
[0061] Exemplarily, the quality detection rule is to check whether the age is less than 30 and whether the registration year is 2024. The age in the data to be detected is 20 and the registration year is 2023. At this time, the age is less than 30 but the registration year is not 2024, so the data quality detection result is a quality satisfaction degree of 50%.
[0062] Optionally, after executing step 31 , detecting the data to be detected according to the quality detection rule and obtaining the data quality detection result of the data to be detected can be achieved through step 41 .
[0063] Step 41, according to the quality detection rules and the data quality qualification standard, the data to be detected is detected to obtain the data quality detection result of the data to be detected.
[0064] Specifically, after obtaining the data quality qualification standard, the data to be tested is tested according to the quality testing rules and the data quality qualification standard to obtain the data quality test result of the data to be tested.
[0065] Optionally, the data quality detection result includes first information on whether the data to be detected meets the quality detection rule and second information on whether the data to be detected meets the data quality qualification standard. After executing step 104, steps 42 to 43 may also be executed.
[0066] Step 42, when the second information indicates that the data to be detected does not meet the data quality qualification standard, the reason why the data quality of the data to be detected is unqualified is obtained according to the first information.
[0067] Specifically, the first information includes information on whether the data to be detected meets the quality detection rules, for example, the data to be detected does not meet the quality detection rules at all, and the pass rate of the data to be detected is 0. The second information includes information on whether the data to be detected meets the data quality qualification standard, for example, the data to be detected does not meet the quality qualification standard at all, and the pass rate of the data to be detected is 0. When the second information indicates that the data to be detected does not meet the data quality qualification standard, the data to be detected does not meet the quality detection result. At this time, according to the first information, the reason for the data quality failure of the data to be detected is obtained.
[0068] Exemplarily, the quality inspection rule is to check whether the registration year is 2024, the data quality qualification standard is qualified when the registration year is 2024, and the first information is that the registration year is 2023, which does not meet the quality inspection rule. According to the first information, the reason for the data quality failure of the data to be tested is that the registration year is 2023, which does not meet the quality inspection rule for checking whether the registration year is 2024.
[0069] Step 43: Generate first alarm information according to the reason for the data quality failure, and send the first alarm information to the target user terminal.
[0070] Specifically, after obtaining the quality failure reason of the data to be detected, a first alarm message is generated according to the quality failure reason of the data, the first alarm message includes the quality failure reason, and the first alarm message is sent to the target user terminal. The target user may be an operation and maintenance personnel, and the sending method may be SMS, email, etc.
[0071] Optionally, before executing step 104, step 32 may be further executed. After executing step 32, step 104 may be implemented through steps 1041 to 1043.
[0072] Step 32: If the same quality detection rule corresponds to N data to be detected, then the scheduling start time and the scheduling interval time of the N data to be detected are determined according to the data time information corresponding to each of the N data to be detected.
[0073] Wherein, N is an integer greater than 1.
[0074] Specifically, the data time information corresponding to the data to be detected is the time information of the generation of the data to be detected. The same quality detection rule corresponds to N data to be detected, that is, the N data to be detected can be detected by the quality detection rule. The scheduling start time is the time to start the data quality detection of the N data to be detected, and the scheduling interval time is the interval between the detection times of two adjacent data to be detected when the N data to be detected are detected in sequence. According to the data time information corresponding to each of the N data to be detected, the scheduling start time and the scheduling interval time of the N data to be detected are determined, for example, according to the earliest generation time of the N data to be detected, the scheduling start time is determined. When the N data to be detected are generated at the same time, the scheduling interval time is set according to the number of the data to be detected, so that all the N data to be detected can be detected. When the N data to be detected are not generated at the same time, the scheduling interval time is set according to the number of the data to be detected and the generation time of the last data to be detected, so that all the N data to be detected can be detected.
[0075] Exemplarily, the two data to be detected are the data of a person named A and the data of a person named B as shown in Table 2. The quality detection rules corresponding to the data of the person named A and the data of the person named B are the same.
[0076] Table 2
[0077] Name age Year of registration A 20 2023 B 25 2024
[0078] Step 1041 , at the scheduling start time, the first data to be detected among the N data to be detected is detected according to the quality detection rule to obtain the data quality detection result of the first data to be detected.
[0079] Specifically, at the start time of scheduling, the first data to be detected among the N data to be detected is detected according to the quality detection rules to obtain the data quality detection result of the first data to be detected. If the N data to be detected are data generated at the same time, the first data to be detected can be a data selected arbitrarily, or can be determined by keyword recognition and other methods, and this scheme does not limit this. If the N data to be detected are data generated sequentially, the first data to be detected is the data with the earliest data generation time.
[0080] Step 1042, add i scheduling interval times to the scheduling start time, detect the (i+1)th data to be detected among the N data to be detected according to the quality detection rule, and obtain the data quality detection result of the (i+1)th data to be detected.
[0081] Here, i is an integer greater than 0.
[0082] Specifically, the data quality of the i+1th data to be detected should be detected at the scheduling start time plus i scheduling intervals. Therefore, the i+1th data to be detected among the N data to be detected is detected according to the quality detection rule to obtain the data quality detection result of the i+1th data to be detected.
[0083] Exemplarily, when i is 1, the scheduling start time is added with 1 scheduling interval time, and the second data to be detected among N data to be detected is detected according to the quality detection rule to obtain the data quality detection result of the second data to be detected.
[0084] Step 1043, determine the sum of i and 1 as the updated value of i, and return to execute step 1042 until the updated value of i is greater than N-1.
[0085] Specifically, the change of the data to be detected can be realized by determining the sum of i and 1 as the value of i after the update. After determining the sum of i and 1 as the value of i after the update, return to step 1042 to obtain the data quality detection result of the i+1th data to be detected. If the value of i after the update is less than or equal to N-1, continue to return to step 1042. If the value of i after the update is greater than N-1, stop and return to step 1042.
[0086] Exemplarily, the sum of i and 1 is 2, which is determined as the updated value of i, and the process returns to step 1042 to obtain the data quality test result of the third data to be tested. The process loops until the updated value of i is N, and the process returns to step 1042 to obtain the data quality test result of the Nth data to be tested. At this time, the value of i is greater than N-1, and the process stops and returns to step 1042.
[0087] Optionally, a data source quality detection result of a data source of each to-be-detected data is determined according to N data quality detection results.
[0088] Specifically, the data source quality detection result is used to characterize the data quality detection results of multiple data in a data source, for example, the average of the quality satisfaction levels of multiple data quality detection results in the data source is used as the data source quality detection result of the data source. In the loop process, N data quality detection results are obtained, and each data to be detected has its source data source. The data source quality detection result of the data source of each data to be detected is determined according to the data quality detection result of the data to be detected.
[0089] Exemplarily, 3 of the N data to be detected are derived from the same data source, and the data source quality detection results of the data sources of the 3 data to be detected are determined according to the data quality detection results of the 3 data to be detected. The data quality detection results of the 3 data to be detected are respectively satisfying the data quality detection, satisfying the data quality detection, and not satisfying the data quality detection, so the data source quality detection result of the data source is determined to be a data quality satisfaction rate of 66.7%.
[0090] The scheme of the present application, according to the detection requirements of the data to be detected, determines the target rule template that matches the detection requirements from the preset rule template library; wherein the target rule template includes multiple template identification information; determines the information content of multiple data identification information of the data to be detected; wherein there is a corresponding relationship between the data identification information and the template identification information; replaces each template identification information in the target rule template with the information content of the corresponding data identification information, and obtains the quality detection rule of the data to be detected; detects the data to be detected according to the quality detection rule, and obtains the data quality detection result of the data to be detected. That is, the scheme of the present application, on the one hand, determines the target rule template that matches the detection requirements from the preset rule template library, does not need to write a large number of rules by itself, and improves the efficiency of data quality detection. On the other hand, by replacing information in the target rule template to obtain the quality detection rule, it is avoided to write a large amount of code to determine the data quality rule, and reduces the error rate caused by code development work in the generation of data quality rules, thereby improving the efficiency of data quality detection and ensuring the accuracy and reliability of data.
[0091] Figure 2 is another flow chart of the data quality detection method provided by the present application. Figure 1 Based on the illustrated embodiment and various optional implementation schemes, the content of data quality detection performed by the task scheduling execution unit is described in detail. Figure 2 As shown, the method may include the following steps:
[0092] Step 201 : According to the detection requirement of the data to be detected, a target rule template matching the detection requirement is determined from a preset rule template library.
[0093] Step 202: determine the information content of multiple data identification information of the data to be detected.
[0094] Step 203: Replace each template identification information in the target rule template with the information content of the corresponding data identification information to obtain the quality detection rule of the data to be detected.
[0095] Step 204: detect the data to be detected according to the quality detection rules to obtain detection result information.
[0096] Specifically, after the data to be tested is tested according to the quality test rules, test result information is obtained, and the test result information may include the data quality test result, or may only include the test result without including the data quality test result.
[0097] Exemplarily, the test result information is that the test failed and no data quality test result was obtained.
[0098] Optionally, step 204 may be implemented through steps 2041 to 2042 .
[0099] Step 2041, input the quality detection rules and the data to be detected into the task scheduling execution unit, and obtain the execution result output by the task scheduling execution unit after executing the task of detecting the data to be detected according to the quality detection rules.
[0100] Specifically, the task scheduling execution unit is a scheduling rule executor capable of executing task scheduling. For example, the task scheduling execution unit may be a DolphinScheduler scheduling engine. The DolphinScheduler scheduling engine can connect to multiple servers. When the amount of tasks is large, the server load is reduced and the quality detection efficiency is improved by assigning tasks to multiple servers for execution. The quality detection rules scheduled by the DolphinScheduler scheduling engine are all in SQL format. The task scheduling execution unit and the electronic device executing this embodiment may be in the same server or in different servers. After the task scheduling execution unit starts to execute the task, the quality detection rules and the data to be detected are input into the task scheduling execution unit, and the detection task execution result outputted by the task scheduling execution unit after executing the task of detecting the data to be detected according to the quality detection rules is obtained. When the rules in the quality detection rules are generated by a rule template, the task scheduling execution unit only detects the data to be detected according to the rules generated by the rule template. When the rules in the quality detection rules are generated by multiple rule templates, the task scheduling execution unit sequentially detects the data to be detected according to multiple rules generated by multiple rule templates.
[0101] Optionally, if the same quality detection rule corresponds to N data to be detected, the scheduling start time and scheduling interval time of the N data to be detected are determined according to the data time information corresponding to each of the N data to be detected. The scheduling start time and scheduling interval time are input into the task scheduling execution unit.
[0102] The task scheduling execution unit schedules the first data to be detected at the scheduling start time, detects the first data to be detected according to the quality detection rule, and obtains the detection task execution result of the first data to be detected.
[0103] The task scheduling execution unit schedules the i+1th data to be detected among the N data to be detected by adding i scheduling interval times to the scheduling start time, and detects the i+1th data to be detected among the N data to be detected according to the quality detection rules to obtain the detection task execution result of the i+1th data to be detected.
[0104] The sum of i and 1 is determined as the value of i after the update, and the task scheduling execution unit returns to execute the step of "scheduling the i+1th data to be detected among the N data to be detected, and detecting the i+1th data to be detected among the N data to be detected according to the quality detection rules, and obtaining the detection task execution result of the i+1th data to be detected" until the value of i after the update is greater than N-1. At this time, N detection task execution results are obtained. The N detection task execution results are output as the detection task execution results. In addition, the task scheduling execution unit can also generate a quality report, which includes the task execution results. The task scheduling execution unit executes tasks according to the scheduling start time and the scheduling interval time, which reduces the development cost caused by the traditional interface call method of connecting to the database using the Java language to detect data quality because it is not suitable for most databases, and improves the efficiency and performance of task execution.
[0105] After obtaining N detection task execution results, the data source detection task execution results of each data source to be detected, or the data table detection task execution results of each data table to be detected, or the data detection task execution results of multiple data generated at the same time can be summarized. The task execution results may include the number of quality detection rules, the number of abnormal rules in the task execution results, the quality pass rate in the task execution results, the quality score in the task execution results, and the quality ranking obtained according to the quality score.
[0106] Optionally, the task of DolphinScheduler as a scheduling engine to perform quality inspection can also be performed manually by operation and maintenance personnel.
[0107] Step 2042, using the execution result as detection result information.
[0108] Specifically, the execution result can be a task execution log generated by the task scheduling execution unit, and the execution result is output as the detection result information. For example, the DolphinScheduler can also display the scheduling progress in the log output by the scheduling engine to enable the user to understand the quality detection progress. The DolphinScheduler scheduling engine also provides a visual interface to display the scheduling progress in real time, which is convenient for users to view and manage the quality detection progress.
[0109] Optionally, the detection result information includes detection success information and detection failure information. After executing step 204, steps 2043 to 2045 may also be executed.
[0110] Step 2043: If the detection result information is detection success information, determine to execute step 205.
[0111] Specifically, if the detection result information is detection success information, the result of whether the data quality detection satisfies the quality detection rules is output. At this time, the data quality detection result of the data to be detected is determined according to the detection result information.
[0112] Step 2044: If the detection result information is detection failure information, determine the cause of the detection failure based on the detection failure information.
[0113] Specifically, the detection failure information may be a detection failure report, including the detection failure reason. If the detection result information is detection failure information, the detection failure reason is determined according to the detection failure information. In addition, the task that failed to be detected by the task scheduling execution unit may also be recorded for easy viewing by the user. For example, the detection failure reason may be a program error, an operation error, an abnormality in a rule, etc.
[0114] Step 2045: Generate second alarm information according to the detection failure reason, and send the second alarm information to the target user terminal.
[0115] Specifically, after the detection failure reason is obtained, second alarm information is generated according to the detection failure reason, and the second alarm information is sent to the target user terminal.
[0116] Step 205: Determine the data quality detection result of the data to be detected according to the detection result information.
[0117] Specifically, the test result information includes a test result report, and the data quality test result of the data to be tested is determined based on information generated after the data quality test in the test result report.
[0118] The solution of the present application, after scheduling quality detection rules and data to be detected, uses a task scheduling execution unit to perform data quality detection by executing data quality detection tasks. When the amount of data quality detection tasks is large, it can realize concurrent execution of tasks by scheduling tasks to reduce the workload of the server and improve the efficiency of data quality detection.
[0119] Figure 3 is a structural diagram of a data quality detection device provided by the present application, and the device is suitable for executing the data quality detection method provided by the present application. Figure 3 As shown, the device may specifically include:
[0120] The rule template determination module 301 is used to determine a target rule template matching the detection requirement from a preset rule template library according to the detection requirement of the data to be detected; wherein the target rule template includes a plurality of template identification information.
[0121] The information content determination module 302 is used to determine the information content of multiple data identification information of the data to be detected; wherein there is a corresponding relationship between the data identification information and the template identification information.
[0122] The replacement module 303 is used to replace each template identification information in the target rule template with the information content of the corresponding data identification information to obtain the quality detection rule of the data to be detected.
[0123] The detection module 304 is used to detect the data to be detected according to the quality detection rule to obtain a data quality detection result of the data to be detected.
[0124] In one embodiment, the device also includes: a qualification standard determination module, which is used for the replacement module 303 to replace each template identification information in the target rule template with the information content of the corresponding data identification information, and after obtaining the quality detection rules for the data to be detected, determine the data quality qualification standard of the data to be detected according to the quality detection rules; the detection module 304 is specifically used to: detect the data to be detected according to the quality detection rules and the data quality qualification standard, and obtain the data quality detection results of the data to be detected.
[0125] In one embodiment, the data quality detection result includes first information of whether the data to be detected meets the quality detection rules and second information of whether the data to be detected meets the data quality qualification standard; the detection module 304 is also used for: after detecting the data to be detected according to the quality detection rules and obtaining the data quality detection result of the data to be detected, when the second information indicates that the data to be detected does not meet the data quality qualification standard, obtaining the reason for the data quality failure of the data to be detected according to the first information; generating a first alarm information according to the reason for the data quality failure, and sending the first alarm information to the target user terminal.
[0126] In one embodiment, the preset rule template library of the rule template determination module 301 includes an integrity rule template sub-library, a uniqueness rule template sub-library, a timeliness rule template sub-library, a validity rule template sub-library, a consistency rule template sub-library and a stability rule template sub-library; the integrity rule template sub-library includes at least one of the following: a field null value check template and a field empty string check template; the uniqueness rule template sub-library includes at least one of the following: a field uniqueness check template, a field grouping number check template and a field repeated value number check template; the timeliness rule template sub-library includes at least one of the following: a time function comparison template, a single-table time field comparison template and a two-table time field comparison template; the validity rule template The sub-library includes at least one of the following: a field format verification template, a field length verification template, a field value range verification template, a code table reference comparison template, and a data standard code table reference comparison template; the consistency rule template sub-library includes at least one of the following: a single-table field value consistency comparison template, a single-table field statistical value consistency comparison template, a single-table field business logic consistency comparison template, a two-table field value consistency comparison template, a two-table field statistical value consistency comparison template, a two-table field business logic consistency comparison template, and a cross-source two-table field statistical value consistency comparison template; the stability rule template sub-library includes at least one of the following: a table stability verification template, a table volatility verification template, a field stability verification template, and a field volatility verification template.
[0127] In one embodiment, there are multiple target rule templates, and the replacement module 303 is specifically used to: replace each template identification information in each target rule template with the information content of the corresponding data identification information to obtain a replaced target rule template; and determine the multiple replaced target rule templates as the quality detection rules for the data to be detected.
[0128] In one embodiment, the detection module 304 is further used for: detecting the data to be detected according to the quality detection rule, and before obtaining the data quality detection result of the data to be detected, if the same quality detection rule corresponds to N data to be detected, then determining the scheduling start time and the scheduling interval time of the N data to be detected according to the data time information corresponding to each of the N data to be detected; wherein N is an integer greater than 1; the detection module 304 is specifically used for: at the scheduling start time, detecting the first data to be detected among the N data to be detected according to the quality detection rule, and obtaining the first data to be detected The data quality detection result of the data; adding i scheduling intervals to the scheduling start time, detecting the i+1th data to be detected among the N data to be detected according to the quality detection rule, and obtaining the data quality detection result of the i+1th data to be detected; wherein, i is an integer greater than 0; determining the sum of i and 1 as the value of i after the update, returning to execute "adding i scheduling intervals to the scheduling start time, detecting the i+1th data to be detected among the N data to be detected according to the quality detection rule, and obtaining the data quality detection result of the i+1th data to be detected", until the value of i after the update is greater than N-1.
[0129] In one embodiment, the detection module 304 is further used to: until the updated value of i is greater than N-1, determine the data source quality detection result of the data source of each data to be detected according to the N data quality detection results.
[0130] In one embodiment, the detection module 304 is specifically used to: detect the data to be detected according to the quality detection rule to obtain detection result information; and determine the data quality detection result of the data to be detected according to the detection result information.
[0131] In one embodiment, the detection module 304, in terms of detecting the data to be detected according to the quality detection rules and obtaining the detection result information, is specifically used to: input the quality detection rules and the data to be detected into a task scheduling execution unit, and obtain the execution result output by the task scheduling execution unit after executing the task of detecting the data to be detected according to the quality detection rules; and use the execution result as the detection result information.
[0132] In one embodiment, the detection result information includes detection success information and detection failure information, and the device also includes: a detection success module, which is used for the detection module 304 to detect the data to be detected according to the quality detection rule, and after obtaining the detection result information, if the detection result information is detection success information, determine to execute the step of "determining the data quality detection result of the data to be detected according to the detection result information"; a detection failure module, which is used to determine the cause of the detection failure according to the detection failure information if the detection result information is detection failure information; an alarm module, which is used to generate a second alarm information according to the cause of the detection failure, and send the second alarm information to the target user terminal.
[0133] The device of the present application determines the target rule template that matches the detection requirement from the preset rule template library according to the detection requirement of the data to be detected; wherein the target rule template includes multiple template identification information; determines the information content of multiple data identification information of the data to be detected; wherein there is a corresponding relationship between the data identification information and the template identification information; replaces each template identification information in the target rule template with the information content of the corresponding data identification information to obtain the quality detection rule of the data to be detected; detects the data to be detected according to the quality detection rule to obtain the data quality detection result of the data to be detected. That is, the scheme of the present application, on the one hand, determines the target rule template that matches the detection requirement from the preset rule template library, does not need to write a large number of rule templates by itself, and improves the efficiency of data quality detection. On the other hand, by replacing information in the target rule template to obtain the quality detection rule, it is avoided to write a large amount of code to determine the data quality rule, and reduces the error rate caused by code development work in the generation of data quality rules, thereby improving the efficiency of data quality detection and ensuring the accuracy and reliability of data.
[0134] The present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the data quality detection method provided in any of the above embodiments when executing the program.
[0135] The present application also provides a computer-readable medium having a computer program stored thereon, and when the program is executed by a processor, the data quality detection method provided in any of the above embodiments is implemented.
[0136] Reference below Figure 4 , which shows a structural schematic diagram of an electronic device 400 suitable for implementing the present application. Figure 4 The electronic device shown is only an example and should not bring any limitation to the function and scope of use of the present application.
[0137] like Figure 4As shown, the electronic device 400 includes a central processing unit (CPU) 401, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 402 or a program loaded from a storage part 408 into a random access memory (RAM) 403. In the RAM 403, various programs and data required for the operation of the electronic device 400 are also stored. The CPU 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.
[0138] The following components are connected to the I / O interface 405: an input section 406 including a keyboard, a mouse, etc.; an output section 407 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 408 including a hard disk, etc.; and a communication section 409 including a network interface card such as a LAN card, a modem, etc. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to the I / O interface 405 as needed. A removable medium 411, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 410 as needed, so that a computer program read therefrom is installed into the storage section 408 as needed.
[0139] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 409, and / or installed from the removable medium 411. When the computer program is executed by the central processing unit (CPU) 401, the above-mentioned functions defined in the system of the present application are executed.
[0140] It should be noted that the computer-readable medium shown in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.
[0141] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the above-mentioned module, program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0142] The modules and / or units described in this application may be implemented in software or hardware. The modules and / or units described may also be set in a processor, for example, it may be described as: a processor includes a rule template determination module, an information content determination module, a replacement module and a detection module. The names of these modules do not, in some cases, constitute limitations on the modules themselves.
[0143] As another aspect, the present invention further provides a computer-readable medium, which may be included in the device described in the above embodiment; or may exist independently without being assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device performs the following operations:
[0144] According to the detection requirements of the data to be detected, a target rule template matching the detection requirements is determined from a preset rule template library; wherein the target rule template includes multiple template identification information; the information content of multiple data identification information of the data to be detected is determined; wherein there is a corresponding relationship between the data identification information and the template identification information; each template identification information in the target rule template is replaced with the information content of the corresponding data identification information to obtain a quality detection rule for the data to be detected; the data to be detected is detected according to the quality detection rule to obtain a data quality detection result of the data to be detected.
[0145] According to the technical solution of the present invention, according to the detection requirements of the data to be detected, a target rule template matching the detection requirements is determined from a preset rule template library; wherein the target rule template includes multiple template identification information; the information content of multiple data identification information of the data to be detected is determined; wherein there is a corresponding relationship between the data identification information and the template identification information; each template identification information in the target rule template is replaced with the information content of the corresponding data identification information to obtain the quality detection rule of the data to be detected; the data to be detected is detected according to the quality detection rule to obtain the data quality detection result of the data to be detected. That is, the solution of the present application, on the one hand, determines the target rule template matching the detection requirements from the preset rule template library, does not need to write a large number of rule templates by itself, and improves the efficiency of data quality detection. On the other hand, by replacing information in the target rule template to obtain the quality detection rule, it is avoided to write a large amount of code to determine the data quality rule, and reduces the error rate caused by code development work in the generation of data quality rules, thereby improving the efficiency of data quality detection and ensuring the accuracy and reliability of data.
[0146] An embodiment of the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements a data quality detection method as provided in any embodiment of the present application.
[0147] In the process of implementation, the computer program product can be written in one or more programming languages or a combination thereof to perform the computer program code of the present application, and the programming language includes an object-oriented programming language, such as Java, Smalltalk, C++, and also includes a conventional procedural programming language, such as "C" language or similar programming language. The program code can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on the remote computer, or completely on the remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).
[0148] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.
[0149] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. A data quality detection method, characterized in that: The method comprises: According to the detection requirements of the data to be detected, determine a target rule template matching the detection requirements from a preset rule template library; wherein the target rule template includes a plurality of template identification information; Determine the information content of multiple data identification information of the data to be detected; wherein there is a corresponding relationship between the data identification information and the template identification information; Replacing each template identification information in the target rule template with the information content of the corresponding data identification information to obtain the quality detection rule of the data to be detected; The data to be detected is detected according to the quality detection rule to obtain a data quality detection result of the data to be detected.
2. The method according to claim 1, characterized in that After replacing each template identification information in the target rule template with the information content of the corresponding data identification information to obtain the quality detection rule for the data to be detected, the method further includes: Determining the data quality qualification standard of the data to be tested according to the quality detection rule; The step of detecting the data to be detected according to the quality detection rule to obtain a data quality detection result of the data to be detected includes: According to the quality detection rules and the data quality qualification standard, the data to be detected is detected to obtain a data quality detection result of the data to be detected.
3. The method according to claim 2, characterized in that The data quality detection result includes first information of whether the data to be detected meets the quality detection rule and second information of whether the data to be detected meets the data quality qualification standard; After detecting the data to be detected according to the quality detection rule and obtaining the data quality detection result of the data to be detected, the method further includes: When the second information indicates that the data to be detected does not meet the data quality qualification standard, obtaining the reason why the data quality of the data to be detected is unqualified according to the first information; Generate first alarm information according to the reason for the data quality failure, and send the first alarm information to the target user terminal.
4. The method according to claim 1, characterized in that: The preset rule template library includes an integrity rule template sub-library, a uniqueness rule template sub-library, a timeliness rule template sub-library, a validity rule template sub-library, a consistency rule template sub-library and a stability rule template sub-library; The integrity rule template sub-library includes at least one of the following: a field null value check template and a field empty string check template; The uniqueness rule template sub-library includes at least one of the following: a field uniqueness check template, a field grouping number check template, and a field repeated value number check template; The timeliness rule template sub-library includes at least one of the following: a time function comparison template, a single table time field comparison template, and a two-table time field comparison template; The validity rule template sub-library includes at least one of the following: a field format check template, a field length check template, a field value range check template, a code table reference comparison template, and a data standard code table reference comparison template; The consistency rule template sub-library includes at least one of the following: a single table field value consistency comparison template, a single table field statistical value consistency comparison template, a single table field business logic consistency comparison template, a two-table field value consistency comparison template, a two-table field statistical value consistency comparison template, a two-table field business logic consistency comparison template, and a cross-source two-table field statistical value consistency comparison template; The stability rule template sub-library includes at least one of the following: a table stability check template, a table volatility check template, a field stability check template, and a field volatility check template.
5. The method according to claim 1, characterized in that The number of the target rule templates is multiple, and the step of replacing each template identification information in the target rule template with the information content of the corresponding data identification information to obtain the quality detection rule of the data to be detected includes: Replacing each template identification information in each of the target rule templates with the information content of the corresponding data identification information to obtain a replaced target rule template; The plurality of replaced target rule templates are determined as quality detection rules for the data to be detected.
6. The method according to claim 1, characterized in that Before detecting the data to be detected according to the quality detection rule and obtaining the data quality detection result of the data to be detected, the method further includes: If the same quality detection rule corresponds to N data to be detected, the scheduling start time and the scheduling interval time of the N data to be detected are determined according to the data time information corresponding to each of the N data to be detected; wherein N is an integer greater than 1; The step of detecting the data to be detected according to the quality detection rule to obtain a data quality detection result of the data to be detected includes: At the scheduling start time, detecting the first data to be detected among the N data to be detected according to the quality detection rule, and obtaining a data quality detection result of the first data to be detected; Add i scheduling interval times to the scheduling start time, detect the i+1th data to be detected among the N data to be detected according to the quality detection rule, and obtain the data quality detection result of the i+1th data to be detected; wherein i is an integer greater than 0; The sum of i and 1 is determined as the updated value of i, and the process returns to execute "adding i scheduling intervals to the scheduling start time, detecting the i+1th data to be detected among the N data to be detected according to the quality detection rule, and obtaining the data quality detection result of the i+1th data to be detected", until the updated value of i is greater than N-1.
7. The method according to claim 6, characterized in that After the updated value of i is greater than N-1, the method further includes: A data source quality detection result of a data source of each data to be detected is determined according to the N data quality detection results.
8. The method according to claim 1, characterized in that The step of detecting the data to be detected according to the quality detection rule to obtain a data quality detection result of the data to be detected includes: Detect the data to be detected according to the quality detection rules to obtain detection result information; The data quality detection result of the data to be detected is determined according to the detection result information.
9. The method according to claim 8, characterized in that The step of detecting the data to be detected according to the quality detection rule to obtain detection result information includes: Inputting the quality detection rule and the data to be detected into a task scheduling execution unit, and obtaining an execution result output by the task scheduling execution unit after executing the task of detecting the data to be detected according to the quality detection rule; The execution result is used as the detection result information.
10. The method according to claim 8, characterized in that The detection result information includes detection success information and detection failure information. After the to-be-detected data is detected according to the quality detection rule and the detection result information is obtained, the method further includes: If the detection result information is detection success information, determine to execute the step of "determining the data quality detection result of the to-be-detected data according to the detection result information"; If the detection result information is detection failure information, determining the detection failure cause according to the detection failure information; Generate second alarm information according to the detection failure reason, and send the second alarm information to the target user terminal.
11. A data quality detection device, characterized in that: The device comprises: A rule template determination module is used to determine a target rule template matching the detection requirement from a preset rule template library according to the detection requirement of the data to be detected; wherein the target rule template includes a plurality of template identification information; An information content determination module, used to determine the information content of a plurality of data identification information of the data to be detected; wherein there is a corresponding relationship between the data identification information and the template identification information; A replacement module, used to replace each template identification information in the target rule template with the information content of the corresponding data identification information to obtain the quality detection rule of the data to be detected; The detection module is used to detect the data to be detected according to the quality detection rule to obtain a data quality detection result of the data to be detected.
12. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the data quality detection method according to any one of claims 1 to 10 is implemented.
13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the data quality detection method as described in any one of claims 1 to 10 is implemented.
14. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the computer program implements the data quality detection method according to any one of claims 1 to 10.
Citation Information
Patent Citations
A method for detecting data quality and a device for detecting data quality
CN109491990A
Data quality evaluation method and device, storage medium and electronic equipment
CN113468034A
Data quality detection method and device and computer equipment
CN116894031A
Data quality detection method and device and electronic equipment
CN117668007A
Cited By
Data compliance security detection method and equipment based on adaptive cue word optimization
CN120336492A
Business data inspection method and device, computer equipment, storage medium and product
CN121070940A
Data quality detection method and device, medium and program product
CN121560869A
Data quality detection method, device, medium, and program product
CN121560869B
