Data quality control analysis method, device, equipment, medium and program

Abstract

The invention relates to a data quality control analysis method, device and equipment, a storage medium and a computer program, and the method comprises the steps: collecting standard information of a plurality of preset standard items, and constructing a data standard library according to the standard information; generating a plurality of checking rules by using a pre-constructed checking task, and checking to-be-checked data according to the plurality of checking rules and the data standard library to obtain a checking result corresponding to the checking task; performing task monitoring according to the checking result to obtain a task monitoring result corresponding to the to-be-checked data; and giving an alarm for the check task and the check rule corresponding to the task monitoring result meeting the preset early warning condition.

Description

Technical Field

[0001] The present disclosure relates to the field of data analysis technology, and in particular to a data quality control and analysis method, device, equipment, medium and program. Background Art

[0002] With the rapid development of the Internet and digital technology, the era of digital transformation has arrived. Various digitalizations are actually changing the daily operations of enterprises. Data is a very important asset for enterprises. Perfect data quality can make enterprise data fully realize its value. However, how to have a complete data quality analysis system remains to be discussed.

[0003] The current status of data quality management in existing enterprises is not perfect enough. For example, many Internet companies have more than one set of data, and the various data sets are overlapping and inconsistent. At the same time, the data standards of different business systems are not unified, which makes it difficult to integrate data, resulting in inaccurate data analysis. In addition, relying solely on traditional infrastructure and lacking IT system support, poor data quality leads to a lack of early warning capabilities and irregular and non-standard data governance. Therefore, how to improve early warning capabilities and data quality analysis and control capabilities has become an urgent problem to be solved. Summary of the invention

[0004] The present invention provides a data quality control and analysis method, device, equipment, medium and program, the main purpose of which is to solve the problem of low data quality analysis and control capabilities and poor early warning capabilities.

[0005] In a first aspect, the present disclosure provides a data quality control and analysis method, comprising:

[0006] Collecting standard information of multiple preset standard items, and building a data standard library according to the standard information;

[0007] Generate multiple verification rules using the pre-built verification task, verify the data to be verified according to the multiple verification rules and the data standard library, and obtain the verification result corresponding to the verification task;

[0008] Perform task monitoring according to the verification result to obtain the task monitoring result corresponding to the data to be verified;

[0009] Alerts are issued for the verification tasks and verification rules corresponding to the task monitoring results that meet the preset warning conditions.

[0010] In some embodiments, the step of constructing a data standard library according to the standard information includes:

[0011] Extracting information keywords of each standard item one by one, and generating standard information retrieval of the standard item according to the information keywords;

[0012] The standard information is retrieved and stored in association with each other according to the standard information to obtain a data standard library.

[0013] In some embodiments, extracting information keywords of each standard item one by one includes:

[0014] Selecting one piece of standard information of the standard item one by one as the standard information to be processed;

[0015] Counting the number of local occurrences of the standard information to be processed in the standard item to be processed, and counting the number of global occurrences of the standard information to be processed in all standard items;

[0016] Calculate the criticality of the standard information to be processed according to the local occurrence number and the global occurrence number;

[0017] The to-be-processed standard information whose criticality is greater than a preset number threshold is collected as information keywords of the to-be-processed standard items.

[0018] In some embodiments, performing task monitoring according to the verification result to obtain the task monitoring result corresponding to the data to be verified includes:

[0019] Calculating the standard score and the confidence interval respectively according to the verification results;

[0020] A task monitoring result is generated according to the standard score and the confidence interval.

[0021] In some embodiments, generating the task monitoring result according to the standard score and the confidence interval includes:

[0022] Determine abnormal data to be checked according to the standard score;

[0023] Determine the true value range of the data to be checked according to the confidence interval;

[0024] The standard scores and confidence intervals corresponding to the abnormal data to be checked are taken as the task monitoring results.

[0025] In some embodiments, the alerting of the task monitoring results and the corresponding verification results that meet the preset warning conditions includes:

[0026] When the standard score in the task monitoring result is greater than the preset score threshold, an alarm is issued for the corresponding verification task and verification rule;

[0027] When the true value range of the data to be checked in the task monitoring result is not within the range of the confidence interval, an alarm is issued for the corresponding checking task and the checking rule.

[0028] In a second aspect, the present disclosure provides a data quality control and analysis device, comprising:

[0029] An information collection module is used to collect standard information of multiple preset standard items and build a data standard library according to the standard information;

[0030] A data verification module generates a plurality of verification rules using a pre-built verification task, verifies the data to be verified according to the plurality of verification rules and the data standard library, and obtains a verification result corresponding to the verification task;

[0031] A task monitoring module performs task monitoring according to the verification result to obtain the task monitoring result corresponding to the data to be verified;

[0032] The alarm module issues an alarm for the verification tasks and verification rules corresponding to the task monitoring results that meet the preset warning conditions.

[0033] In a third aspect, the present disclosure provides a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method described in the above aspects.

[0034] In a fourth aspect, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the method described in the above aspects when executed by a processor.

[0035] In a fifth aspect, the present disclosure provides a computer program product, including a computer program, which implements the steps of the method described in the above aspects when executed by a processor.

[0036] The embodiment of the present invention establishes a data standard library by collecting standard information of multiple preset standard items, realizes data sharing, and reduces data redundancy; uses pre-built verification tasks to generate multiple verification rules to perform multi-level verification and audit on the data to be verified, ensuring the security and accuracy of the data; and verifies the data through the verification rules generated corresponding to the pre-built verification tasks, thereby improving the subsequent data quality level. Task monitoring is performed according to the verification results, and the running status of the verification tasks and the verification rules are tracked in real time to obtain the task monitoring results corresponding to the data to be verified, reduce manual intervention, and improve task execution efficiency. Alarms are issued for the verification tasks and verification rules corresponding to the task monitoring results that meet the preset warning conditions, and collaborative processing is prompted, which greatly improves real-time performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The present disclosure will be described in more detail below based on embodiments and with reference to the accompanying drawings:

[0038] Figure 1A flowchart of a data quality control and analysis method provided in an embodiment of the present disclosure.

[0039] Figure 2 A schematic diagram of the structure of a data quality control and analysis device provided in an embodiment of the present disclosure.

[0040] In the drawings, the same reference numerals are used for the same components and the drawings are not drawn to scale. DETAILED DESCRIPTION

[0041] In order to enable those skilled in the art to better understand the technical solution of the present disclosure, and to fully understand and implement how the present disclosure applies technical means to solve technical problems and achieve the corresponding technical effects, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only embodiments of a part of the present disclosure, not all of the embodiments. The embodiments of the present disclosure and the various features in the embodiments can be combined with each other without conflict, and the technical solutions formed are all within the scope of protection of the present disclosure. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative work should fall within the scope of protection of the present disclosure.

[0042] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0043] The embodiment of the present application provides a data quality control and analysis method. The execution subject of the data quality control and analysis method includes but is not limited to at least one of the electronic devices such as a server, a terminal, etc. that can be configured to execute the system provided by the embodiment of the present application. In other words, the data quality control and analysis method can be executed by software or hardware installed on a terminal device or a server device. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc. The server can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms.

[0044] Reference Figure 1 , which is a flow chart of a data quality control and analysis method provided by an embodiment of the present invention.

[0045] In an embodiment of the present invention, the data quality control and analysis method includes:

[0046] S1. Collect standard information of multiple preset standard items and build a data standard library according to the standard information.

[0047] In the embodiment of the present invention, the standard items include data type, data length, data rule, etc., and the type length rule definition is completed through the standard items; the standard information refers to the specific field type, specific field length, etc. For example, Int is an integer type, and the length is fixed to four bytes.

[0048] In an embodiment of the present invention, a standard collection platform can be configured, a collection adapter with Excel as a template can be deployed in the standard collection platform, a data input interface can be set in the Excel worksheet, and data can be imported and adjusted using the acquisition and conversion functions to complete the collection of standard information.

[0049] In the existing technology, when constructing a data standard library, the standard information of the standard items is usually simply stored. When the data volume of the standard information of the standard items is large, it often causes data confusion and redundancy, resulting in low efficiency of data analysis.

[0050] In detail, the building of a data standard library according to the standard information includes:

[0051] Extracting information keywords of each standard item one by one, and generating standard information retrieval of the standard item according to the information keywords;

[0052] The standard information is retrieved and stored in association with each other according to the standard information to obtain a data standard library.

[0053] In an embodiment of the present invention, a data standard library with standard information can be established based on the standard items, so that the standard information of the standard items has a higher structure and can be retrieved through associated storage, thereby improving the efficiency of subsequent analysis of the standard information and improving data scalability.

[0054] In the embodiment of the present invention, since the data volume of the standard items is large, in order to realize the

[0055] Quick processing of standard information can extract information keywords for each standard item one by one, thereby reducing the amount of data and reducing the complexity of the constructed data standard library.

[0056] In detail, extracting information keywords of each standard item one by one includes:

[0057] Selecting one piece of standard information of the standard item one by one as the standard information to be processed;

[0058] Counting the number of local occurrences of the standard information to be processed in the standard item to be processed, and counting the number of global occurrences of the standard information to be processed in all standard items;

[0059] Calculate the criticality of the standard information to be processed according to the local occurrence number and the global occurrence number;

[0060] The to-be-processed standard information whose criticality is greater than a preset number threshold is collected as information keywords of the to-be-processed standard items.

[0061] In detail, the information keywords may be converted into word vectors, and the converted word vectors may be used as information indexes of the standard items.

[0062] In the embodiment of the present invention, by constructing a data standard library through information keywords, the dimension of the standard item data can be reduced, thereby improving the efficiency of constructing the data standard library.

[0063] In the embodiment of the present invention, the standard items are optimized and stored by constructing a data standard library, so that the standard information is more structured, the integrity and consistency of the data are maintained, and the data security is improved.

[0064] In the embodiment of the present invention, by collecting standard information of preset standard items, a data standard library is constructed according to the standard information, data sharing is achieved, data redundancy is reduced, and the efficiency of subsequent data analysis is improved.

[0065] S2. Generate multiple verification rules using the pre-built verification task, and verify the data to be verified according to the multiple verification rules and the data standard library to obtain the verification result corresponding to the verification task.

[0066] In the embodiment of the present invention, the verification task mainly provides an execution carrier for the verification rules, and the verification rules need to be executed through the verification task. The verification rule refers to the rule information for verifying the data standard library. The verification rule may include a unique rule, a non-empty rule, a consistency rule, an approval rule, a threshold rule, etc.

[0067] The verification results include data integrity verification results, data uniqueness verification results, data consistency verification results, data timeliness verification results and data standardization verification results.

[0068] Among them, the unique rule means that the attribute value in the current attribute must be unique and non-repeatable, and is applicable to all types of attributes; the non-empty rule means that the attribute cannot be empty, and is applicable to all types of attributes; the consistency rule is used to verify whether the attribute value of the current attribute is within a certain range, and the range refers to the field value of the comparison field; the approval rule is to verify whether the attribute value is accurate by comparing it with the field of the comparison object. If all the checked field values ​​of the checked data are equal to all the comparison field values ​​of the standard data, then the number is qualified data; the threshold rule is used to check whether the attribute value of the current attribute is within the threshold range. The selected attributes are all date types and numeric fields in the model. The verification rules are established as a verification rule table, as shown below:

[0069]

[0070]

[0071] In the embodiment of the present invention, the logic in the table is used for detection statistics, and the expected value is used for early warning. The verification rule is the basis for data quality assessment, and data instruction management supports user-defined verification rules, through which data is standardized and normalized.

[0072] In an embodiment of the present invention, the data to be checked is checked according to the multiple checking rules and the data standard library to obtain the checking result corresponding to the checking task, and the data to be checked is checked through four aspects: data jitter, data offset, data accumulation, and data comparison.

[0073] Specifically, data jitter refers to filtering the information of the data standard library according to the digital type check field, grouping it, setting the upper and lower threshold ranges, counting the number of times the check field value appears, and calculating the average value of the check field.

[0074] Data deviation refers to inputting a check field in the check task, presetting upper and lower thresholds, and detecting whether the check field deviates from a normal range.

[0075] Data accumulation refers to selecting verification tasks, data sources, and extended conditions on the data accumulation page to identify whether the data to be tested has abnormal accumulation in certain time periods or conditions, thereby avoiding errors in data processing and analysis.

[0076] Data comparison refers to comparing different data to be verified or data of the same data to be verified at different time points through the verification task and relevant parameters of data comparison to detect the consistency and accuracy of the data.

[0077] In the embodiment of the present invention, when performing the check, the data to be checked is automatically checked according to the check rules, and a check list is generated for the user to view, which simplifies the work of going online and modifying, and can display the check list to facilitate the user to correct the data.

[0078] In detail, the checklist refers to a checklist showing the check results after the check is executed. The user can understand the check results of each field through the checklist, and perform data quality assessment and necessary corrections based on the check results to ensure the accuracy and compliance of the data.

[0079] In the embodiment of the present invention, a plurality of verification rules are generated by using a pre-constructed verification task, and the data to be verified is verified according to the verification rules and the data standard library to obtain the verification result corresponding to the verification task, thereby reducing manual intervention and improving work efficiency. The data is verified and audited at multiple levels by using the verification rules generated corresponding to the pre-constructed verification task to ensure the security and accuracy of the data, thereby improving the subsequent data quality level.

[0080] S3. Perform task monitoring according to the verification result to obtain the task monitoring result corresponding to the data to be verified.

[0081] In the embodiment of the present invention, the task monitoring refers to monitoring all the check rules under the check task and displaying the running status of each check rule; the check rule monitoring mainly tracks the status of a single check rule in real time, so as to understand the running status of the check rule.

[0082] In the embodiment of the present invention, the step of performing task monitoring according to the verification result to obtain the task monitoring result corresponding to the data to be verified includes:

[0083] Calculating the standard score and the confidence interval respectively according to the verification results;

[0084] A task monitoring result is generated according to the standard score and the confidence interval.

[0085] The task monitoring result includes the obtained standard score and confidence interval. The abnormal data to be inspected is identified by calculating the standard score, and an alarm notification is subsequently issued for the data to be inspected that is not within the confidence interval, thereby improving the monitoring efficiency and the task execution efficiency.

[0086] In the embodiment of the present invention, generating the task monitoring result according to the standard score and the confidence interval includes:

[0087] Determine abnormal data to be checked according to the standard score;

[0088] Determine the true value range of the data to be checked according to the confidence interval;

[0089] The standard scores and confidence intervals corresponding to the abnormal data to be checked are taken as the task monitoring results.

[0090] In detail, the calculation standard score is used to identify abnormal data to be checked. Specifically, the standard score is calculated by the following formula:

[0091]

[0092] Wherein, Z-score represents the standard score, X represents the vector value of the data to be checked, μ represents the average value of the data to be checked, and σ represents the standard deviation of the data to be checked. The standard score is used to measure the deviation of a single check rule. The larger the standard score, the greater the deviation of the data to be checked from the average value, and vice versa. By comparing the standard score with the preset score threshold, it is determined whether the check task and check rule corresponding to the data to be checked are abnormal.

[0093] The task monitoring refers to setting a reasonable threshold for each check rule so that abnormal situations can be discovered in time. Specifically, the confidence interval is calculated by the following formula to estimate the true value range of the data to be checked:

[0094]

[0095] Here, X is the mean and z is the value of the standard normal distribution, which is determined according to the chosen confidence level.

[0096] For example, for a 95% confidence level, the z value is usually 1.96, which means that in a standard normal distribution, 95% of the data are within plus or minus 1.96 standard deviations of the mean, σ represents the standard deviation of the data points, and n represents the total amount of the data to be checked.

[0097] In detail, the standard deviation represents the dispersion of the data to be checked in the population, and the confidence level indicates how many proportions of confidence intervals will contain the population mean in multiple repeated samplings. For example, a 95% confidence level means that if multiple data are repeatedly extracted from the population and the confidence interval is calculated, approximately 95% of the intervals will contain the population mean. The width of the confidence interval is affected by the amount of data to be checked and the standard deviation. The larger the amount of data to be checked, the smaller the standard deviation, and the narrower the confidence interval is generally, indicating that the uncertainty of the estimate is smaller.

[0098] In the embodiment of the present invention, a confidence interval is calculated to estimate the true value range of the monitoring indicator, thereby ensuring stable operation of data quality, so as to subsequently issue an alarm notification for the monitoring indicator that exceeds the confidence interval.

[0099] In the embodiment of the present invention, the execution status of the check rule, including the time and frequency of the check rule triggering, can be understood through the task monitoring result, which helps to monitor and analyze the execution effect of the check rule and find problems and anomalies in time.

[0100] In an embodiment of the present invention, task monitoring is performed based on the verification result, and the running status of the verification task and the verification rule is tracked in real time, so as to obtain the task monitoring result corresponding to the data to be verified, improve task execution efficiency, and facilitate subsequent data quality warnings.

[0101] S4. Alert the verification tasks and verification rules corresponding to the task monitoring results that meet the preset warning conditions.

[0102] In an embodiment of the present invention, the preset warning condition refers to issuing a warning notification when the task monitoring result and the corresponding verification result exceed the previous range or size, thereby reducing the error rate and maximizing the task execution efficiency. The preset warning condition includes whether the standard score exceeds the preset score threshold and whether the true value range of the data to be verified is within the confidence interval.

[0103] In the embodiment of the present invention, the alarming of the task monitoring results and the corresponding verification results that meet the preset warning conditions includes:

[0104] When the standard score in the task monitoring result is greater than the preset score threshold, an alarm is issued for the corresponding verification task and verification rule;

[0105] When the true value range of the data to be checked in the task monitoring result is not within the range of the confidence interval, an alarm is issued for the corresponding checking task and the checking rule.

[0106] In the embodiment of the present invention, through the preset warning conditions, the task monitoring results and the corresponding verification results are analyzed in real time, and corresponding processing methods are taken in time to improve monitoring efficiency and accuracy, while greatly improving task execution efficiency and task quality.

[0107] In an embodiment of the present invention, an alarm is issued for the task monitoring results and the corresponding verification results that meet the preset warning conditions. By configuring the alarm information, an alarm notification is issued when the task monitoring results and the corresponding verification results trigger the warning conditions to remind collaborative processing, and the real-time performance is greatly improved.

[0108] The embodiment of the present invention establishes a data standard library by collecting standard information of multiple preset standard items, realizes data sharing, and reduces data redundancy; uses pre-built verification tasks to generate multiple verification rules to perform multi-level verification and audit on the data to be verified, ensuring the security and accuracy of the data; and verifies the data through the verification rules generated corresponding to the pre-built verification tasks, thereby improving the subsequent data quality level. Task monitoring is performed according to the verification results, and the running status of the verification tasks and the verification rules are tracked in real time to obtain the task monitoring results corresponding to the data to be verified, reduce manual intervention, and improve task execution efficiency. Alarms are issued for the verification tasks and verification rules corresponding to the task monitoring results that meet the preset warning conditions, and collaborative processing is prompted, which greatly improves real-time performance.

[0109] On the basis of the above embodiments, this embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method described in the above embodiments.

[0110] In some implementations of this embodiment, a computer-readable storage medium is provided, on which a computer program is stored, characterized in that when the computer program is executed by a processor, the steps of the method described in the above embodiment are implemented.

[0111] In some implementations of this embodiment, a computer program product is provided, including a computer program, characterized in that when the computer program is executed by a processor, the steps of the method described in the above embodiment are implemented.

[0112] The processor may include, but is not limited to, one or more processors or microprocessors, etc. Each processor may be an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a controller, a microcontroller, a microprocessor or other electronic components to execute the methods in the above embodiments.

[0113] The computer-readable storage medium may be implemented by any type of volatile or non-volatile storage device or a combination thereof, and the computer-readable storage medium may include but is not limited to, for example, random access memory (RAM), read-only memory (ROM), flash memory, EPROM memory, EEPROM memory, registers, computer storage media (e.g., hard disk, floppy disk, solid-state drive, removable disk, CD-ROM, DVD-ROM, Blu-ray disc, etc.).

[0114] The computer-readable storage medium may also store at least one computer executable program, such as a computer-readable instruction. The computer-readable storage medium includes, but is not limited to, for example, a volatile memory and / or a non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache memory (cache), etc. The computer-readable storage medium may include, for example, a read-only memory (ROM), a hard disk, a flash memory, etc. For example, a non-transitory computer-readable storage medium may be connected to a computing device such as a computer, and then, when the computing device runs the computer-readable instructions stored on the computer-readable storage medium, the various methods described above may be performed.

[0115] In addition, the computer device may also include (but not limited to) a data bus, an input / output (I / O) bus, a display, and input / output devices (eg, keyboard, mouse, speaker, etc.), etc.

[0116] The processor may communicate with external devices via an I / O bus via a wired or wireless network.

[0117] In one embodiment, the at least one computer executable instruction may also be compiled into or constitute a software product / computer program product, wherein one or more computer executable instructions are executed by a processor to perform the various functions and / or method steps in the embodiments described in the present technology.

[0118] In the embodiments provided in the present disclosure, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the above-mentioned module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0119] It should be noted that in the present disclosure, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element limited by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0120] Although the embodiments disclosed in the present disclosure are as above, the above contents are only embodiments adopted for facilitating the understanding of the present disclosure and are not intended to limit the present disclosure. Any technician in the technical field to which the present disclosure belongs can make any modifications and changes in the form and details of the implementation without departing from the spirit and scope disclosed in the present disclosure, but the scope of patent protection of the present disclosure shall still be subject to the scope defined in the attached claims.

Claims

1. A data quality control and analysis method, characterized in that: The method comprises: Collecting standard information of multiple preset standard items, and building a data standard library according to the standard information; Generate multiple verification rules using the pre-built verification task, verify the data to be verified according to the multiple verification rules and the data standard library, and obtain the verification result corresponding to the verification task; Perform task monitoring according to the verification result to obtain the task monitoring result corresponding to the data to be verified; Alerts are issued for the verification tasks and verification rules corresponding to the task monitoring results that meet the preset warning conditions.

2. The data quality control and analysis method according to claim 1, characterized in that: The step of constructing a data standard library according to the standard information includes: Extracting information keywords of each standard item one by one, and generating standard information retrieval of the standard item according to the information keywords; The standard information is retrieved and stored in association with each other according to the standard information to obtain a data standard library.

3. The data quality control and analysis method according to claim 1, characterized in that: The performing of task monitoring according to the verification result to obtain the task monitoring result corresponding to the data to be verified includes: Calculating the standard score and the confidence interval respectively according to the verification results; A task monitoring result is generated according to the standard score and the confidence interval.

4. The data quality control and analysis method according to claim 3 is characterized in that: Generating the task monitoring result according to the standard score and the confidence interval includes: Determine abnormal data to be checked according to the standard score; Determine the true value range of the data to be checked according to the confidence interval; The standard scores and confidence intervals corresponding to the abnormal data to be checked are taken as the task monitoring results.

5. The data quality control and analysis method according to claim 1, characterized in that: The alerting of the verification task and verification rule corresponding to the task monitoring result meeting the preset warning condition includes: When the standard score in the task monitoring result is greater than the preset score threshold, an alarm is issued for the corresponding verification task and verification rule; When the true value range of the data to be checked in the task monitoring result is not within the range of the confidence interval, an alarm is issued for the corresponding checking task and the checking rule.

6. The data quality control and analysis method according to claim 2, characterized in that: The step of extracting information keywords of each standard item one by one includes: Selecting one piece of standard information of the standard item one by one as the standard information to be processed; Counting the number of local occurrences of the standard information to be processed in the standard item to be processed, and counting the number of global occurrences of the standard information to be processed in all standard items; Calculate the criticality of the standard information to be processed according to the local occurrence number and the global occurrence number; The to-be-processed standard information whose criticality is greater than a preset number threshold is collected as information keywords of the to-be-processed standard items.

7. A data quality control and analysis device, characterized in that: include: An information collection module is used to collect standard information of multiple preset standard items and build a data standard library according to the standard information; A data verification module generates a plurality of verification rules using a pre-built verification task, verifies the data to be verified according to the plurality of verification rules and the data standard library, and obtains a verification result corresponding to the verification task; A task monitoring module performs task monitoring according to the verification result to obtain the task monitoring result corresponding to the data to be verified; The alarm module issues an alarm for the verification tasks and verification rules corresponding to the task monitoring results that meet the preset warning conditions.

8. A computer device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images