A data governance system based on knowledge management to check data quality

Abstract

The present invention proposes a data governance system for verifying data quality based on knowledge management, which belongs to the field of data verification and includes: a data governance subsystem, which is configured to supervise the acquisition, identification and standardization of data and is independently deployed in each project; a data quality knowledge base, which is configured to pre-store supplier system tables, quality inspection data tables, and quality inspection rule relationship tables; an implementation workbench, which is configured to interact with the data governance subsystem and the data quality knowledge base for data and knowledge, adjust and supplement quality inspection rules when implementing projects, and verify data quality. This application uses the data quality knowledge base to recommend quality inspection rules, realizes the reuse of empirical quality inspection rules, adjusts quality inspection rules according to specific project requirements through the implementation workbench, improves system adaptability, and stores new quality inspection rules in the data quality knowledge base to update the data quality knowledge base, realizes the reuse of data quality inspection rule knowledge for each project, and improves data quality inspection efficiency.

Description

Technical Field

[0001] The present invention relates to the field of data verification technology, and in particular to a data governance system based on knowledge management to verify data quality. Background Art

[0002] Data quality inspection is a key link in data governance. It mainly inspects data from aspects such as integrity, uniqueness, validity, consistency, and accuracy to identify and solve data quality problems. In the existing data governance system, the data quality management module is responsible for quality inspection of the collected raw data or the data after governance. Implementation engineers need to identify the data meaning, data type, and length of each table and field, and determine the inspection requirements for the field. Based on these requirements, implementation engineers formulate corresponding data quality inspection rules in the data governance system. These rules are the basis for performing data quality inspections and directly affect the effectiveness and efficiency of data governance.

[0003] In the existing technology, implementation engineers need to spend a lot of time collecting and analyzing data quality inspection requirements before formulating data quality inspection rules. In addition, each data governance project requires implementation engineers to redefine data quality inspection rules in the data governance system according to project requirements, even when processing similar business system data. This leads to a lot of duplication of work and reduces project implementation efficiency.

[0004] Therefore, finding a method that can intelligently recommend data quality inspection rules, improve data quality inspection efficiency, and improve the quality of data governance is a technical problem that technical personnel in this field urgently need to solve. Summary of the invention

[0005] In view of this, the present invention proposes a data governance system based on knowledge management to check data quality, which can use the data quality knowledge base to recommend quality inspection rules and realize the reuse of empirical quality inspection rules. By implementing the workbench, the quality inspection rules are adjusted according to specific project requirements, and the new quality inspection rules are stored in the data quality knowledge base to update the data quality knowledge base, thereby realizing the reuse of data quality inspection rule knowledge for each project and improving data quality inspection efficiency.

[0006] The technical solution of the present invention is implemented as follows: The present invention provides a data governance system for checking data quality based on knowledge management, comprising:

[0007] A data governance subsystem, which is configured to oversee the acquisition, identification, and standardization of data and is deployed independently in each project;

[0008] A data quality knowledge base configured to pre-store supplier system tables, quality inspection data tables, and quality inspection data table and quality inspection rule relationship tables;

[0009] The implementation workbench is configured to interact with the data governance subsystem and the data quality knowledge base for data and knowledge, adjust and supplement quality inspection rules when implementing projects, and check data quality.

[0010] Based on the above technical solutions, preferably, the data governance subsystem includes a data acquisition module, a master data management module, a data scheduling module, a data standard management module, a quality inspection rule synchronization module, a quality inspection plan formulation module and a quality inspection plan execution module.

[0011] On the basis of the above technical solution, preferably, the adjustment and supplementation of quality inspection rules specifically include:

[0012] Step 1: The data governance subsystem connects to the data source of the current project and uses the data collection module to collect metadata of the data source;

[0013] Step 2: The master data management module identifies the master data of the metadata to obtain master data information; wherein the master data information includes the schema name, the English name of the table, the Chinese name of the table, the English name of the field, the Chinese name of the field, the field data type and the field length;

[0014] Step 3: The master data management module constructs a standard data table according to the master data information, and the data scheduling module standardizes the master data information according to the standard data table to obtain standardized master data information;

[0015] Step 4: Obtain quality inspection rule recommendation results from the data quality knowledge base according to the first recommendation method, wherein the quality inspection rule recommendation results include data warehouse stratification, database mode, table English name, and table Chinese name;

[0016] Step 5: Add and delete quality inspection rules for each field of the standardized master data information according to the quality inspection rule recommendation results to obtain metadata quality inspection rules;

[0017] Step 6: Save the metadata quality inspection rules to the data quality knowledge base, and the data governance subsystem manages the quality inspection plan for the metadata quality inspection rules.

[0018] Based on the above technical solution, preferably, in step 4, the first recommended method includes:

[0019] The elements of standardized master data information include supplier, system and version; the elements of supplier system table include system number, supplier, system code and version; the elements of quality inspection data table include system number, data warehouse layer, data table code, database mode, table English name and table Chinese name; the elements of quality inspection data table and quality inspection rule relationship table include system number, data table code, field English name, field Chinese name and quality inspection rule code;

[0020] Query the supplier system table based on the supplier, system and version of the standardized master data information to obtain the system number corresponding to the data source;

[0021] According to the system number corresponding to the data source, query in the quality inspection data table to obtain the data table code corresponding to the data source;

[0022] According to the system number corresponding to the data source and the data table code corresponding to the data source, a query is performed in the quality inspection data table and the quality inspection rule relationship table to obtain the field-level quality inspection rule recommendation results for each data table in the data source.

[0023] Based on the above technical solution, preferably, step five specifically includes:

[0024] Get the field quality inspection rules for each field based on the standardized master data information;

[0025] Match the recommended quality inspection rule results with the field quality inspection rules of each field, and check whether there are any rules in the recommended quality inspection rule results that are not applicable to the quality inspection rules of each field:

[0026] If yes, delete the field quality inspection rules in the standardized master data information that are not applicable to the quality inspection rule recommendation results;

[0027] If not, check whether the field quality inspection rules can be directly applied to the quality inspection rule recommendation results:

[0028] If yes, directly select and apply the recommended result of the quality inspection rule;

[0029] If not, add field quality inspection rules for standardized master data information.

[0030] Based on the above technical solution, preferably, the quality inspection rule recommendation result is matched with the field quality inspection rule of each field, specifically including:

[0031] The field quality inspection rules of each field are queried with the supplier system table, quality inspection data table, and the quality inspection data table and quality inspection rule relationship table in the data quality knowledge base to obtain the current field quality inspection rule information, wherein the current field quality inspection rule information includes the system number, data table code, and field Chinese name;

[0032] Find the records in the quality inspection rule recommendation results that match the quality inspection rule information of the current field, and extract the quality inspection rule code corresponding to the record;

[0033] Determine specific quality inspection rules according to the quality inspection rule code and apply the specific quality inspection rules to the corresponding fields.

[0034] Based on the above technical solution, preferably, in step 5, field quality inspection rules for the standardized master data information are added, specifically including:

[0035] Create new field quality inspection rules in the data governance subsystem and set inspection logic and parameters based on the new field quality inspection rules;

[0036] Synchronize new quality inspection rules to the implementation workbench.

[0037] More preferably, in step 6, the quality inspection plan management specifically includes:

[0038] The quality inspection rule synchronization module synchronizes metadata quality inspection rules to the data governance subsystem;

[0039] The quality inspection plan formulation module formulates quality inspection plans based on metadata quality inspection rules;

[0040] The quality inspection plan execution module executes the quality inspection plan according to the execution time.

[0041] On the basis of the above technical solutions, preferably, the quality inspection plan is formulated to specifically include:

[0042] Group metadata quality inspection rules according to data warehouse layers and data source entities to form several quality inspection plans;

[0043] Set a scheduled execution time for each quality inspection plan.

[0044] Based on the above technical solutions, preferably, the quality inspection rule synchronization module synchronizes the metadata quality inspection rules to the data governance subsystem, specifically including:

[0045] The data governance subsystem includes a first API interface and a second API interface;

[0046] Call the first API interface to store the data warehouse layer, data table code, database mode, table code, table English name, and table Chinese name of the metadata quality inspection rule in the data quality management module;

[0047] Call the second API interface to store the table code, field English name, field Chinese name, and quality inspection rule code of the metadata quality inspection rule in the data quality management module;

[0048] Synchronously verify whether the API call is successful. If successful, the metadata quality inspection rules are synchronized to the data governance subsystem.

[0049] The data governance system of the present invention has the following beneficial effects compared with the prior art:

[0050] (1) Metadata is collected through the data governance subsystem, and quality inspection rules are recommended using the data quality knowledge base, so that the quality inspection rules based on experience can be reused, duplication of work can be reduced, and the efficiency of starting new projects can be improved. Users can adjust the quality inspection rules according to specific project requirements through the implementation workbench to improve system adaptability, and store the new quality inspection rules in the data quality knowledge base to update the data quality knowledge base, thus realizing the reuse of data quality inspection rules for each project and improving data quality inspection efficiency.

[0051] (2) By adopting the first recommended method to obtain quality inspection rules, the consistency of data governance between different projects is ensured, and the new quality inspection rules are saved in the data quality knowledge base to achieve continuous accumulation of knowledge, which can adapt to different data sources and improve the applicability of quality inspection rules. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0053] Figure 1 A schematic diagram of the process of a data governance system for checking data quality based on knowledge management according to the present invention;

[0054] Figure 2 This is a system structure diagram of a data governance system for checking data quality based on knowledge management according to the present invention. DETAILED DESCRIPTION

[0055] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0056] like Figure 1 and Figure 2 As shown, the present invention provides a data governance system for checking data quality based on knowledge management, comprising:

[0057] A data governance subsystem, which is configured to oversee the acquisition, identification, and standardization of data and is deployed independently in each project;

[0058] A data quality knowledge base configured to pre-store supplier system tables, quality inspection data tables, and quality inspection data table and quality inspection rule relationship tables;

[0059] The implementation workbench is configured to interact with the data governance subsystem and the data quality knowledge base for data and knowledge, adjust and supplement quality inspection rules when implementing projects, and check data quality.

[0060] It can be understood that the data quality knowledge base stores historical project experience and standardized quality inspection rules. The implementation workbench obtains initial quality inspection rule recommendations from the data quality knowledge base. The implementation engineer reviews and adjusts the rules on the implementation workbench. The adjusted rules are sent to the data governance subsystem for execution. The data governance subsystem performs quality inspection and feeds the results back to the implementation workbench to supervise the data, ensure data consistency and standardization, and synchronize the new rules back to the data quality knowledge base for future projects, realizing the accumulation and reuse of knowledge and improving the accuracy and efficiency of data verification.

[0061] Specifically, in one embodiment of the present application, the data governance subsystem includes a data acquisition module, a master data management module, a data scheduling module, a data standard management module, a quality inspection rule synchronization module, a quality inspection plan formulation module and a quality inspection plan execution module.

[0062] In the data governance subsystem, the data acquisition module acquires raw data, the master data management module identifies and processes master data, the data standard management module provides standardized knowledge, the quality inspection rule synchronization module acquires the latest rules from the data quality knowledge base, the quality inspection plan execution module formulates the quality inspection plan according to the synchronized rules, the data scheduling module coordinates the execution order of each step, and the quality inspection plan execution module finally performs data quality inspection.

[0063] Specifically, adjust and supplement the quality inspection rules, including:

[0064] Step 1: The data governance subsystem connects to the data source of the current project and uses the data collection module to collect metadata of the data source;

[0065] Step 2: The master data management module identifies the master data of the metadata to obtain master data information; wherein the master data information includes the schema name, the English name of the table, the Chinese name of the table, the English name of the field, the Chinese name of the field, the field data type and the field length;

[0066] Master data identification includes:

[0067] Get metadata data source reports;

[0068] Obtain master data recommendation results from the data dictionary knowledge base and the master data knowledge base. The master data recommendation results include the schema name, the table English name, and the table Chinese name;

[0069] According to the data source report and the master data recommendation results, the master data information of the data source is obtained according to the screening method.

[0070] Step 3: The master data management module constructs a standard data table according to the master data information, and the data scheduling module standardizes the master data information according to the standard data table to obtain standardized master data information;

[0071] Step 4: Obtain quality inspection rule recommendation results from the data quality knowledge base according to the first recommendation method, wherein the quality inspection rule recommendation results include data warehouse stratification, database mode, table English name, and table Chinese name;

[0072] The first recommended method includes:

[0073] The elements of standardized master data information include supplier, system and version; the elements of supplier system table include system number, supplier, system code and version; the elements of quality inspection data table include system number, data warehouse layer, data table code, database mode, table English name and table Chinese name; the elements of quality inspection data table and quality inspection rule relationship table include system number, data table code, field English name, field Chinese name and quality inspection rule code;

[0074] Query the supplier system table based on the supplier, system and version of the standardized master data information to obtain the system number corresponding to the data source;

[0075] According to the system number corresponding to the data source, query in the quality inspection data table to obtain the data table code corresponding to the data source;

[0076] According to the system number corresponding to the data source and the data table code corresponding to the data source, a query is performed in the quality inspection data table and the quality inspection rule relationship table to obtain the field-level quality inspection rule recommendation results for each data table in the data source.

[0077] Step 5: Add and delete quality inspection rules for each field of the standardized master data information according to the quality inspection rule recommendation results to obtain metadata quality inspection rules;

[0078] Step 5 specifically includes:

[0079] Get the field quality inspection rules for each field based on the standardized master data information;

[0080] Match the recommended quality inspection rule results with the field quality inspection rules of each field, and check whether there are any rules in the recommended quality inspection rule results that are not applicable to the quality inspection rules of each field:

[0081] If yes, delete the field quality inspection rules in the standardized master data information that are not applicable to the quality inspection rule recommendation results;

[0082] If not, check whether the field quality inspection rules can be directly applied to the quality inspection rule recommendation results:

[0083] If yes, directly select and apply the recommended result of the quality inspection rule;

[0084] If not, add field quality inspection rules for standardized master data information.

[0085] Step 6: Save the metadata quality inspection rules to the data quality knowledge base, and the data governance subsystem manages the quality inspection plan for the metadata quality inspection rules.

[0086] In step six, quality inspection plan management specifically includes:

[0087] The quality inspection rule synchronization module synchronizes metadata quality inspection rules to the data governance subsystem;

[0088] The quality inspection plan formulation module formulates quality inspection plans based on metadata quality inspection rules;

[0089] The quality inspection plan execution module executes the quality inspection plan according to the execution time.

[0090] By adopting the recommended method to obtain quality inspection rules, the consistency of data governance between different projects is ensured, and the new quality inspection rules are saved in the data quality knowledge base to achieve continuous accumulation of knowledge and adapt to different data sources.

[0091] Specifically, a specific embodiment is used to illustrate:

[0092] (1) Collect metadata. Call the metadata collection capability in the metadata management of the data governance subsystem. The data governance subsystem manages the vendor, system, version, data source type, IP, port, user, and password of the data source. After connecting to the data source, collect the metadata of the database, including the schema name, table English name, table comment, field English name, field comment, data type, whether it is required, and whether it is a primary key.

[0093] (2) Collect data. Call the offline data synchronization tool of the data governance subsystem to synchronize data from the data source to the ODS layer of the data governance subsystem.

[0094] (3) Identify master data. Conduct data research and identify master data from the data source. Master data information includes: schema name, table English name, table Chinese name, field English name, field Chinese name, field data type, and field length. Obtain standardized relationships of master data from the data quality knowledge base.

[0095] (4) Constructing the standard layer data model. The master data management module creates physical tables at the standard layer of the data warehouse based on the English name of the table, the Chinese name of the table, the English name of the field, the Chinese name of the field, the English name of the data item field, the Chinese name of the data item field, the data type of the data item, and the length of the data item to achieve data standardization.

[0096] (5) Data standardization. The master data management module establishes a standardized conversion relationship between the English names of the master data tables and fields at the ODS layer of the data warehouse and the English names of the physical tables and fields at the standard layer of the data warehouse based on the master data standardization relationship. This relationship is maintained in the offline data synchronization tool, forming a data conversion task from the ODS layer data to the standard layer data. In the data scheduling module, a timed task for this data standardization relationship is set to execute the data standardization task within the specified time.

[0097] (6) Obtain the recommended results of quality inspection rules from the data quality knowledge base. The data quality knowledge base presets the supplier system table, quality inspection data table, and the relationship table between the quality inspection data table and the quality inspection rules.

[0098] The supplier system table is as follows:

[0099] SYSID PROVIDER_CODE SYSTEM_CODE VERSION 100001 Yunqi Datamiddleplatform V1.0.0 100002 Yunqi Smartform V2.0.0

[0100] Among them, SYSID is the system number, PROVIDER_CODE is the provider, SYSTEM_CODE is the system code, and VERSION is the version.

[0101] The quality inspection data table example is as follows:

[0102]

[0103]

[0104] Among them, SYSID is the system number, DATAWAREHOUSE is the data warehouse layer (ODS represents the source layer, DWD represents the standard layer), TABLE_ID is the data table code, SCHEMA is the database mode, TABLENAME_EN is the English name of the table, and TABLENAME_CHN is the Chinese name of the table.

[0105] An example of the relationship between the quality inspection data table and the quality inspection rules is as follows:

[0106] SYSID TABLE_ID COLUMN_EN COLUMN_CHN RULE_ID 100001 800001 NAME Name 6020101 100001 800001 ENGLISHNAME English name 6020102 100001 800001 PINYIN Name Pinyin 6020103

[0107] Wherein, SYSID is the system number, TABLE_ID is the data table code, COLUMN_EN is the English name of the field, COLUMN_CHN is the Chinese name of the field, and RULE_ID is the quality inspection rule code. The implementation example of the quality inspection rule may include the table object to be inspected, the quality inspection field, the quality inspection query result column, the quality inspection rule condition, and the query filter condition.

[0108] According to the supplier, system, and version of the current project data source, the system number SYSID is queried in the knowledge base supplier system table, and the quality inspection rule recommendation results of the system are queried according to SYSID in the knowledge base quality inspection data table, quality inspection data table, and quality inspection rule relationship table. The recommended results include data warehouse stratification, database mode, table English name, and table Chinese name.

[0109] (7) Adjust and supplement quality inspection rules. By executing the second step of data collection, the ODS layer already has data tables that can be used for quality inspection. By executing the fifth step of data standardization, the standard layer already has data tables that can be used for quality inspection. According to the quality inspection rule recommendation results obtained in the sixth step. The implementation engineer observes the quality inspection rules recommended for each data table in the ODS layer and the standard layer of each system in this project on the implementation workbench, and adds and deletes quality inspection rules for each field in the ODS layer and the standard layer data table based on project requirements. If quality inspection rules are missing, first create quality inspection rules in the data governance subsystem, and then synchronize the quality inspection rules to the implementation workbench.

[0110] (8) The data quality knowledge base saves the quality inspection rules. The quality inspection rules of this project are saved and merged into the supplier system table, the knowledge base quality inspection data table, and the quality inspection data table and quality inspection rule relationship table. The quality inspection rule knowledge data includes: system number, supplier, system code, version, data warehouse layer, data table code, database model, table English name, table Chinese name, field English name, field Chinese name, quality inspection rule code. The above knowledge is used to recommend quality inspection rules for the project next time.

[0111] (9) Synchronize the quality inspection rules to the data governance subsystem. Save the quality inspection rules of this project to the data quality management module of the data governance subsystem. The quality inspection rule data includes: data warehouse layering, data table coding, database model, table English name, table Chinese name, field English name, field Chinese name, and quality inspection rule coding.

[0112] (10) Develop a quality inspection plan. Group the quality inspection rules for the data table according to the data warehouse layer and data theme to form several quality inspection plans. Set a scheduled execution time for the quality inspection plan.

[0113] The following is an example of the quality inspection scheme after grouping:

[0114]

[0115] (11) Execute the quality inspection plan. The task scheduling module in the data governance subsystem schedules the execution of the quality inspection plan according to the set execution time. All quality inspection rules in the quality inspection plan are scheduled for execution and a quality inspection report is generated.

[0116] Specifically, in this embodiment, the system collects metadata through the data governance subsystem, and uses the data quality knowledge base to recommend quality inspection rules, thereby realizing the reuse of empirical quality inspection rules, reducing duplication of work, and thereby improving the efficiency of starting new projects. Users adjust quality inspection rules according to specific project requirements through the implementation workbench to improve system adaptability, and store new quality inspection rules in the data quality knowledge base to update the data quality knowledge base, thereby realizing the reuse of data quality inspection rule knowledge for each project.

[0117] In an embodiment of the present application, matching the quality inspection rule recommendation result with the field quality inspection rule of each field specifically includes:

[0118] The field quality inspection rules of each field are queried with the supplier system table, quality inspection data table, and the quality inspection data table and quality inspection rule relationship table in the data quality knowledge base to obtain the current field quality inspection rule information, wherein the current field quality inspection rule information includes the system number, data table code, and field Chinese name;

[0119] Find the records in the quality inspection rule recommendation results that match the quality inspection rule information of the current field, and extract the quality inspection rule code corresponding to the record;

[0120] Determine specific quality inspection rules according to the quality inspection rule code and apply the specific quality inspection rules to the corresponding fields.

[0121] Through multi-dimensional matching of system numbers, data table codes and field Chinese names, the accuracy of quality inspection rule application is improved, ensuring that each field can obtain the most suitable quality inspection rules. This not only improves the efficiency and accuracy of data quality management, but also enables the flexibly application of different quality inspection rules according to the characteristics of different fields, thereby improving the adaptability of the system.

[0122] In an embodiment of the present application, in step 5, field quality inspection rules for standardized master data information are added, specifically including:

[0123] Create new field quality inspection rules in the data governance subsystem and set inspection logic and parameters based on the new field quality inspection rules;

[0124] Synchronize new quality inspection rules to the implementation workbench.

[0125] By dynamically creating new field quality inspection rules, we can quickly adapt to new data quality requirements, improve the flexibility and adaptability of the system, set inspection logic and parameters according to specific needs, and synchronize new quality inspection rules to the implementation workbench to ensure consistency in quality inspection between the data governance subsystem and the implementation workbench, thereby improving the overall system collaboration efficiency.

[0126] In an embodiment of the present application, formulating a quality inspection plan specifically includes:

[0127] Group metadata quality inspection rules according to data warehouse layers and data source entities to form several quality inspection plans;

[0128] Set a scheduled execution time for each quality inspection plan.

[0129] In one embodiment of the present application, the quality inspection rule synchronization module synchronizes the metadata quality inspection rules to the data governance subsystem, specifically including:

[0130] The data governance subsystem includes a first API interface and a second API interface;

[0131] Call the first API interface to store the data warehouse layer, data table code, database mode, table code, table English name, and table Chinese name of the metadata quality inspection rule in the data quality management module;

[0132] Call the second API interface to store the table code, field English name, field Chinese name, and quality inspection rule code of the metadata quality inspection rule in the data quality management module;

[0133] Synchronously verify whether the API call is successful. If successful, the metadata quality inspection rules are synchronized to the data governance subsystem.

[0134] By storing data at different levels (such as data warehouse layers, table information, and field information) separately, the organization and queryability of the data are improved, and the success of the API call is verified simultaneously, which increases the reliability of the data synchronization process and ensures that the data is transmitted and stored correctly.

[0135] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A data governance system for checking data quality based on knowledge management, characterized in that: include: A data governance subsystem, which is configured to oversee the acquisition, identification, and standardization of data and is deployed independently in each project; A data quality knowledge base configured to pre-store supplier system tables, quality inspection data tables, and quality inspection data table and quality inspection rule relationship tables; The implementation workbench is configured to interact with the data governance subsystem and the data quality knowledge base for data and knowledge, adjust and supplement quality inspection rules when implementing projects, and check data quality; The data governance subsystem includes a data acquisition module, a master data management module, a data scheduling module, a data standard management module, a quality inspection rule synchronization module, a quality inspection plan formulation module and a quality inspection plan execution module; The said adjustment and supplementation of quality inspection rules specifically include: Step 1: The data governance subsystem connects to the data source of the current project and uses the data collection module to collect metadata of the data source; Step 2: The master data management module identifies the master data of the metadata to obtain master data information; wherein the master data information includes the schema name, the English name of the table, the Chinese name of the table, the English name of the field, the Chinese name of the field, the field data type and the field length; Step 3: The master data management module constructs a standard data table according to the master data information, and the data scheduling module standardizes the master data information according to the standard data table to obtain standardized master data information; Step 4: Obtain quality inspection rule recommendation results from the data quality knowledge base according to the first recommendation method, wherein the quality inspection rule recommendation results include data warehouse stratification, database mode, table English name, and table Chinese name; Step 5: Add and delete quality inspection rules for each field of the standardized master data information according to the quality inspection rule recommendation results to obtain metadata quality inspection rules; Step 6: Save the metadata quality inspection rules to the data quality knowledge base, and the data governance subsystem manages the quality inspection plan for the metadata quality inspection rules; In step 4, the first recommended method includes: The elements of standardized master data information include supplier, system and version; the elements of supplier system table include system number, supplier, system code and version; the elements of quality inspection data table include system number, data warehouse layer, data table code, database mode, table English name and table Chinese name; the elements of quality inspection data table and quality inspection rule relationship table include system number, data table code, field English name, field Chinese name and quality inspection rule code; Query the supplier system table based on the supplier, system and version of the standardized master data information to obtain the system number corresponding to the data source; According to the system number corresponding to the data source, query in the quality inspection data table to obtain the data table code corresponding to the data source; According to the system number corresponding to the data source and the data table code corresponding to the data source, query in the quality inspection data table and quality inspection rule relationship table to obtain the field-level quality inspection rule recommendation results for each data table in the data source; Step 5 specifically includes: Get the field quality inspection rules for each field based on the standardized master data information; Match the recommended quality inspection rule results with the field quality inspection rules of each field, and check whether there are any rules in the recommended quality inspection rule results that are not applicable to the quality inspection rules of each field: If yes, delete the field quality inspection rules in the standardized master data information that are not applicable to the quality inspection rule recommendation results; If not, check whether the field quality inspection rules can be directly applied to the quality inspection rule recommendation results: If yes, directly select and apply the recommended result of the quality inspection rule; If not, add field quality inspection rules for standardized master data information; In step six, quality inspection plan management specifically includes: The quality inspection rule synchronization module synchronizes metadata quality inspection rules to the data governance subsystem; The quality inspection plan formulation module formulates quality inspection plans based on metadata quality inspection rules; The quality inspection plan execution module executes the quality inspection plan according to the execution time.

2. A data governance system for checking data quality based on knowledge management as claimed in claim 1, characterized in that: Match the quality inspection rule recommendation results with the field quality inspection rules of each field, including: The field quality inspection rules of each field are queried with the supplier system table, quality inspection data table, and the quality inspection data table and quality inspection rule relationship table in the data quality knowledge base to obtain the current field quality inspection rule information, wherein the current field quality inspection rule information includes the system number, data table code, and field Chinese name; Find the records in the quality inspection rule recommendation results that match the quality inspection rule information of the current field, and extract the quality inspection rule code corresponding to the record; Determine specific quality inspection rules according to the quality inspection rule code and apply the specific quality inspection rules to the corresponding fields.

3. A data governance system for checking data quality based on knowledge management as claimed in claim 2, characterized in that: In step 5, add field quality inspection rules for standardized master data information, including: Create new field quality inspection rules in the data governance subsystem and set inspection logic and parameters based on the new field quality inspection rules; Synchronize new quality inspection rules to the implementation workbench.

4. A data governance system for checking data quality based on knowledge management as claimed in claim 1, characterized in that: The quality inspection plan includes: Group metadata quality inspection rules according to data warehouse layers and data source entities to form several quality inspection plans; Set a scheduled execution time for each quality inspection plan.

5. A data governance system for checking data quality based on knowledge management as claimed in claim 3, characterized in that: The quality inspection rule synchronization module synchronizes metadata quality inspection rules to the data governance subsystem, including: The data governance subsystem includes a first API interface and a second API interface; Call the first API interface to store the data warehouse layer, data table code, database mode, table code, table English name, and table Chinese name of the metadata quality inspection rule in the data quality management module; Call the second API interface to store the table code, field English name, field Chinese name, and quality inspection rule code of the metadata quality inspection rule in the data quality management module; Synchronously verify whether the API call is successful. If successful, the metadata quality inspection rules are synchronized to the data governance subsystem.

Images