A data governance method and system

Through data source cleaning and fusion and meta-model acquisition, a data standard database is established and data quality management is implemented, data fragmentation and quality problems are solved, efficient identification and value utilization of data are achieved, and unified standards and high quality of data are ensured.

CN111190881BActive Publication Date: 2024-12-06SHENZHEN AUDAQUE DATA TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN201911104160.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2019-11-13
Publication Date
2024-12-06
Estimated Expiration
2039-11-13

AI Technical Summary

Technical Problem

Data fragmentation and data quality problems lead to data waste, data cannot be quickly identified and positioned, business attributes are difficult to understand, resulting in data value being buried, data standards are not unified, and quality is low, making decisions difficult to base on.

Method used

Through data source cleaning and fusion, a data standard database is established, metadata is collected and managed using metamodels, standardized data, and data quality management and regulatory protection are performed, including data hierarchical management, authorization management and desensitization management.

Benefits of technology

It realizes the rapid identification and positioning of data, improves the understanding and sharing of data's business attributes, solves data quality problems, improves the overall value utilization of data, and ensures unified standards and high quality of data.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN111190881B_ABST
    Figure CN111190881B_ABST
Patent Text Reader

Abstract

The present invention discloses a data governance method and system, including cleaning and fusing data sources to establish a data standard library; using a meta-model to collect metadata of the data standard library, governing the collected metadata, and obtaining governed metadata; standardizing the data of the data standard library; performing data quality management and regulatory protection on the generated data, and ultimately solving the problem of data waste.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of data governance technology, and more specifically, to a data governance method and system. Background Art

[0002] Government departments, Internet companies, and large group companies have accumulated a large amount of data resources. my country has become one of the countries with the largest amount of data and the richest data types. From the perspective of national strategy and urban strategy, data has become the first resource. However, in the process of informatization construction, enterprises and governments are facing the same problem, the waste of data. The main reasons for data waste are as follows:

[0003] Horizontal data fragmentation

[0004] Data fragmentation is serious within the same organization. With the development of informatization, data silos are becoming more and more prominent. Data cannot be quickly identified and located, and the business attributes of data are difficult to understand. Sharing and exchanging data is also becoming increasingly difficult, which leads to the value of data being buried.

[0005] Vertical data silos

[0006] In the early stages of information construction, each organization built a large number of current systems based on business needs. There was a lack of unified planning and management, inconsistent data standards, different descriptions of the same data by different businesses, and different data formats, which resulted in a large amount of data not being able to be used for decision-making and having low quality.

[0007] Poor data quality

[0008] With the development of informatization, more and more data quality problems such as data duplication, inaccurate data, unreliable data, missing data, unlinkable data, inconsistent data, slow data updates and even interruptions have been exposed. Low quality has become a core issue for government and corporate data. Summary of the invention

[0009] In view of the above problems, the present invention proposes a data governance method and system to solve the problem of data waste.

[0010] In order to achieve the above object, an embodiment of the present invention provides a data governance method, including:

[0011] Step a: Clean and integrate the data sources and establish a data standard library;

[0012] Step b: using the meta-model to collect metadata of the data standard library, managing the collected metadata, and obtaining managed metadata;

[0013] Step c: standardize the data in the data standard library;

[0014] Step d: Perform data quality management and regulatory protection on the data generated in steps a-c.

[0015] Furthermore, the data standard library in step a includes a relational database, a file-based database, a large database and an interface-based database.

[0016] Furthermore, the governance of the collected metadata in step b includes managing, analyzing and applying the metadata;

[0017] The management of metadata includes the management of basic information of business metadata, technical metadata and management metadata;

[0018] The analysis of metadata includes lineage analysis, impact analysis and generation of metadata maps; the application of metadata includes using the basic information of managed metadata and analyzed metadata to trace quality issues, analyze the impact of metadata quality issues and manage assets.

[0019] Further, step c includes: creating a data standard system, maintaining and managing the created data standard system, and mapping the data standard system with business objects;

[0020] Conduct standard system conformity testing and standard system assessment according to standard rules;

[0021] Intelligent standardization: standardize and convert data according to the results of standard compliance detection.

[0022] In a further step d, data quality management is performed on the data generated in step a to step c, including: formulating a data quality monitoring and verification plan to monitor and verify data quality; formulating a data quality rule base; regularly performing data quality control according to the data quality rule base to obtain data quality problems; managing data quality problems; and evaluating data quality;

[0023] The supervisory protection performed on the data generated in steps a to c includes data classification management, data authorization management and data desensitization management.

[0024] A data governance system includes a server and a client, wherein the server includes a memory and a processor, wherein the memory stores a plurality of instructions, and wherein the instructions are suitable for being loaded and executed by the processor of the server:

[0025] Step a: Clean and integrate the data sources and establish a data standard library;

[0026] Step b: using the meta-model to collect metadata of the data standard library, managing the collected metadata, and obtaining managed metadata;

[0027] Step c: standardize the data in the data standard library;

[0028] Step d: Perform data quality management and regulatory protection on the data generated in steps a to c;

[0029] The client includes a display for displaying data in the data standard library, displaying metadata after governance, displaying data in the data standard library after standardization, and displaying content after data quality management.

[0030] Furthermore, the governance of the collected metadata in step b includes managing, analyzing and applying the metadata;

[0031] The management of metadata includes the management of basic information of business metadata, technical metadata and management metadata;

[0032] The analysis of metadata includes lineage analysis, impact analysis and generation of metadata maps; the application of metadata includes using the basic information of managed metadata and analyzed metadata to trace quality issues, analyze the impact of metadata quality issues and manage assets.

[0033] Further, step c includes: creating a data standard system, maintaining and managing the created data standard system, and mapping the data standard system with business objects;

[0034] Conduct standard system conformity testing and standard system assessment according to standard rules;

[0035] Intelligent standardization: standardize and convert data according to the results of standard compliance detection.

[0036] Furthermore, in step d, performing data quality management on the data generated in steps a to c includes:

[0037] Formulate data quality monitoring and verification plans to monitor and verify data quality; formulate a data quality rule base; regularly perform data quality control according to the data quality rule base to identify data quality issues; manage data quality issues; and evaluate data quality;

[0038] The supervisory protection performed on the data generated in steps a to c includes data classification management, data authorization management and data desensitization management.

[0039] The embodiments of the present invention provide a data governance method and system, including cleaning and fusing data sources to establish a data standard library; using a meta-model to collect metadata of the data standard library, governing the collected metadata, and obtaining governed metadata; standardizing the data of the data standard library; performing data quality management and regulatory protection on the generated data, and ultimately solving the problem of data waste. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0041] Figure 1 This is the overall architecture diagram of data governance;

[0042] Figure 2 A standardized schematic diagram of data application scenarios in the data cleaning and fusion process. DETAILED DESCRIPTION

[0043] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0044] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0045] The establishment of a data governance system is a key factor in achieving data management goals, solving data problems, and satisfying data application results. The system needs to establish an effective data governance organization, efficient data governance processes, and a complete data governance architecture. Data, as an asset, is an organic component necessary for government data to achieve greater value, and the city data governance system is a value system that gives a sense of responsibility to individuals or organizations that provide data products and services to customers. Therefore, the data governance environment in the data governance system will cover all aspects of government data, thereby forming a humanistic environment such as the organizational structure and management process of data governance. In addition to the humanistic environment, the basic platform and auxiliary tool environment are also components of the data governance support environment.

[0046] First, create a data governance humanistic environment that supports the construction of a data fusion governance system - a data governance organization. The implementation of a data fusion governance system is the responsibility of every role in government data (data producers, data users, data governors, etc.). Implementation must include the growth and development of all organizations, from senior managers, middle-level managers to executors of specific tasks, from data producers to data users, and from data governors to application developers. The process is long, and the key is to know where you are at the moment, what the intermediate process is like, what the ultimate goal is, and to actively take the next step without trying to avoid the necessary steps. If you want to create value in the work you do, you must pay attention to and give the resources needed to complete the next task. The goal of urban data governance is not to establish an organization or department, but to establish a system and atmosphere culture for data fusion governance.

[0047] Secondly, it is necessary to establish a complete management process, policies, regulations and systems in the data governance system. These are some summaries and regulations of the important steps, actions and tasks in the implementation of the data governance element system, and are the guiding principles, methods and programs for the implementation of the data governance element system. It is to solidify and standardize the organizational structure, job responsibilities, work processes and other contents of the data fusion governance design in the form of rules and regulations. It is both related to and different from many data governance-related policies, regulations and systems issued and implemented in previous business activities. The connection is that their forms are consistent, and the difference is that the policies, regulations and systems in the data fusion governance elements are more unified, coordinated and complete. The policies, regulations and systems in the data fusion governance elements are the relevant data governance systems formulated around the implementation of the data governance system, including data governance planning, data problem exploration, data model design, data cleaning and integration, information resource library construction, data resource management, data service support, data quality monitoring, and data security management. The purpose is to elevate the construction of the government data fusion governance system to the level of government strategy and form a guiding principle with value-oriented nature. The basic elements of the policies, regulations and systems in the data fusion governance elements include goals, prerequisites and related policy content.

[0048] Thirdly, based on the requirements of the organizational structure and management process structure, we build a data governance basic platform and auxiliary tool environment to form an implementation framework for data governance, solidify management processes and specifications, and support the construction of a data governance system. The basic platform and auxiliary tool environment provide a good set of platforms, technologies and management methods for the implementation of the data governance system. They play an important role in different stages of the entire data life cycle, and are used to establish, analyze and evaluate different activities in different stages of the data life cycle of the data governance model. Based on the principle of goal-oriented and problem-oriented government data, in the process of data governance implementation, we use platforms and tools to achieve data cleaning and integration through a six-layer data cleaning pipeline, establish a standard library, and thus form a basic library and a theme library and special library of each department and bureau. At the same time, the quality of the above-mentioned databases and the traceability of problems are ensured through the four-layer "filter" of metadata, data standards, data security and data quality audit. On the built high-quality database, we use data labels to achieve five-level quality classification of data, and provide data to three types of application scenarios: credit-based, analytical, and operational through data service support.

[0049] When designing the processing flow in the data governance system, we should flexibly apply the successful experience of leading practices to build an effective, hierarchical and detailed process system:

[0050] Level 1 (L1 for short), steps: the highest level of actions used in the data element system. Each step includes multiple actions, and all steps constitute the entire processing flow of urban government data elements.

[0051] The design of the data governance process is a step-by-step refinement process. According to the major steps of the data life cycle (planning, problem exploration, model design, cleaning and integration, resource library construction, resource management and service support), the second-level and lower-level actions and tasks are gradually designed, and a logical process at the conceptual level is gradually refined into an operational level process.

[0052] Level 2 (L2), Actions: Generally includes all actions of this step that are completed before the next step. These actions may occur at the same time as another action, but all actions within a step must be completed before processing the next step. At the same time, each action includes multiple tasks.

[0053] Level 3 (L3), Tasks: Specific items that must be completed in order to process an action. Tasks can be completed simultaneously in the process flow, but all tasks within an action must be completed before processing the next action. In the design, if necessary, a more detailed process description can also be given in the process flow, that is, Level 4 (L4): Subtasks. In the subtasks, the detailed description contains the roles and responsibilities of each functional group action.

[0054] Combination Figure 1 , a data governance method, comprising:

[0055] Step a: Clean and integrate the data sources and establish a data standard library; the data standard library includes relational databases, file databases, large databases and interface databases;

[0056] Data source cleaning and fusion methods include: standardization of data sources and data application scenarios, standardization of data problems and data governance effects, and standardization of data cleaning processes.

[0057] The standardization of data sources and data application scenarios includes: standardizing data sources through data research and data cataloging; abstracting data application scenarios into business models and data models to standardize data application scenarios.

[0058] The data research includes research on information resources, data requirements, data standards, data quality and data security.

[0059] The data application scenarios include operational application scenarios, analytical application scenarios and credit application scenarios.

[0060] Standardize data sources and application scenarios with a goal-oriented approach:

[0061] The data source is the premise of good data cleaning governance. Data cleaning governance should be connected to the government information resource directory management system and the data sharing and exchange platform upstream. When cataloging the information resource directory, it is necessary not only to automatically crawl the metadata of the data source and conduct data research, but also to conduct business research. The combination of data research and business research is to achieve data integration and business integration.

[0062] In the work of dual adjustment and dual integration, there are two major types of work: business research and data research. In business research, the focus is on the research of departments (three-determination plan), business (power and responsibility list), system (system list), process, etc.

[0063] In data research, the focus is on information resources, data needs, data standards, data quality and data security.

[0064] Through double adjustment, double integration and data cataloging, we can clarify the data source, clarify the source data model, and standardize the data source cataloging work.

[0065] Application scenario abstraction and application scenario standardization:

[0066] The application scenarios of data are ever-changing. Over the past half century, the application scenarios of data in enterprises have gradually been summarized into two categories. One is operational applications (OLTP, Online Transaction Processing), which are mainly based on individual data applications. They often only need to access one or several individuals in massive data. They involve less data and relatively simple calculations, but have very high requirements for real-time performance and accuracy. The other type of analytical applications (OLAP, Online Analysis Processing) are mainly based on group data applications. They often need to access a considerable portion of the group in massive data. They involve more data and relatively complex calculations, but the requirements for real-time performance and accuracy are generally slightly lower than those for operational applications.

[0067] In government applications, operational applications and analytical applications are also two important types of applications. This type of application exists in large numbers due to the highly fragmented nature of government data. Therefore, in government data application scenarios, in addition to operational applications and analytical applications, there is also a large category of credit applications. We have carried out systematic standardization work on these three types of application scenarios, thereby abstracting the goal of data cleaning.

[0068] like Figure 2 As shown in the figure, analytical application scenarios are divided into subcategories such as planning, budget, policy, population structure, population mobility, and decision-making. The statistical subcategory corresponds to the statistical indicators in the statistical yearbook, business environment index, and other statistical reports, and indicates which statistical indicator in the statistical yearbook and index the data is used for calculation. The planning subcategory indicates that the data is used for analytical indicators such as urban planning, land planning, economic and social development planning, transportation planning, degree and school planning, bed and medical and health institution planning, and general planning revision.

[0069] The population structure subcategory includes gender structure, age structure, occupation structure, education structure, etc. The population mobility subcategory includes internal population mobility, external population mobility, etc. For example, the number of school-age children in the next few years can be predicted through the population age structure and population mobility, so as to plan the school construction and the number of degrees in advance.

[0070] Operational applications are divided into two subcategories: power application scenarios and responsibility application scenarios. The power scenario subcategories are divided into 10 items: administrative license, administrative confirmation, administrative reward, administrative payment, administrative collection, administrative inspection, administrative ruling, administrative punishment, administrative enforcement, and other powers. The responsibility scenario subcategories are also divided into 9 items: administrative license, administrative confirmation, administrative reward, administrative payment, administrative collection, administrative inspection, administrative ruling, administrative punishment, and administrative enforcement.

[0071] Among them, each authority type is an application scenario item, each implementing agency is an application scenario detail, and each authority name is an application scenario item.

[0072] Operational application scenarios include providing accurate data to government service items in operational applications, so as to realize automatic form filling, automatic certification, and automatic approval of government service items: through high-quality and timely data supply, the form filling process can be manually supplemented and automatically filled in seconds, the certification process can achieve "instant certification" without human intervention and automatic certification, and the approval process can achieve "instant approval" without human intervention and automatic approval, so as to realize "instant processing". Operational application scenarios also include proactive services, precise services, and personalized services for government service items. For example: through age statistics, it is possible to predict the elderly who will soon meet the conditions of the "elderly preferential card", pre-prepare certificates for the elderly in advance, and provide door-to-door proactive services on the elderly's birthday or when they meet the conditions.

[0073] Credit application scenarios are divided into two subcategories: natural person verification scenarios and legal person verification scenarios. Natural person verification scenarios are divided into 11 categories, including public education, employment and entrepreneurship, social insurance, medical care, social services, housing security, public culture and sports, the disabled, talent support, individual tax payment, and poverty alleviation funding. Legal person verification scenarios are divided into fund declaration, project undertaking, tax payment, social insurance payment, etc.

[0074] The credit application scenario class can be used to guide the verification of population information, legal person information, resident household economic status and other information of natural persons and legal persons in credit applications.

[0075] By abstracting the three application scenarios of operation, analysis and credit, we can establish a business model of five cross-data and then establish a data model of five cross-data.

[0076] GLDM methodology is a goal-oriented data cleaning, in which LDM (Logical Data Model) is the goal of data cleaning - data model. LDM is derived from the business model, which is abstracted from three major application scenarios. The G in GLDM refers to Government, which refers to the entire government, that is, the five-cross data model of cross-region, cross-level, cross-department, cross-system, and cross-business. The goal-oriented five-cross data cleaning system described in this article is the data cleaning guided by the GLDM methodology.

[0077] The standardization of data problems and data governance effects includes: abstraction of data problems, abstraction of data cleaning governance methods and abstraction of cleaning governance effects.

[0078] The abstraction of data problems includes abstracting data quality problems into data repeatability problems, data completeness problems, data standardization problems, data consistency problems and data timeliness problems;

[0079] Standardize issues and governance effects with a problem-oriented approach:

[0080] Abstract data problems: "Five characteristics", "Five no's", "Two mistakes"

[0081] There are many types of problems in data, and the most important one is quality problems. There are two main types of quality problems in data: formal errors and substantive errors. These two types of errors are mainly divided into five aspects: repetitive problems, completeness problems, standardization problems, consistency problems and timeliness problems. Solving these five problems can make the data neither too much nor too little, not messy, not wrong, and not old.

[0082] In order to solve the “five characteristics” and “two errors” problems in the above data, two types of operations need to be performed on the data: error checking: data quality assessment and detection; error correction: correction of data errors and resolution of data problems.

[0083] These two types of operations match the two types of errors in the data, and there are eight operations as follows:

[0084] 1. Check for incorrect format:

[0085] 2 Correction of formal errors

[0086] 3. Checking for substantive errors;

[0087] 4. Correction of substantial errors;

[0088] 4.1 According to the principle of "one number, one source", among the multiple data providers, the data provided by the responsible unit of one number and one source is used as the benchmark data, and substantial errors are corrected;

[0089] 4.2 According to the principle of "one number, one source", the suggested correction value of the substantial error is confirmed with the responsible unit of the one number, one source, and the substantial error is corrected using the confirmed value as the benchmark data;

[0090] 4.3 Based on the principle of “multi-variable verification”, the suggested correction values ​​of substantial errors were confirmed with all data providers, and substantial errors were corrected using the confirmed values ​​as the benchmark data;

[0091] 4.4 Based on the principle of “multi-verification”, the suggested correction values ​​of substantial errors are confirmed with all data providers, users and managers, and substantial errors are corrected using the confirmed values ​​as benchmark data;

[0092] 4.5 A complete “one number, one source, multiple verification” system has not been formed. Substantive error corrections are made among multiple data providers based on empirical principles such as the “majority principle, freshness principle, authority principle, and self-consistency principle”.

[0093] Abstracting the data cleaning governance method includes abstracting the data cleaning governance method into error checking and error correction;

[0094] Abstract of cleaning management methods: Eight cleaning operations and two cleaning categories Eight operations: Check and correct formal errors based on the principle of one number, one source, and multiple verification; check and correct substantive errors. Further classification of the above eight operations can form two cleaning categories: pre-cleaning and post-cleaning. Pre-cleaning: cleaning operations that are not confirmed by business units, post-cleaning: cleaning operations that require confirmation by business units

[0095] Abstracting the cleaning governance effect includes abstracting the cleaning governance effect into data quality grading and data quality grading coloring.

[0096] According to the execution results of the above eight operations, after cleaning, the data quality can be divided into 5+1 levels from low to high:

[0097] (I) Poor Grade 5 quality: None of the above seven operations are performed, and the data has not been checked for errors or corrected. This quality is “untested quality”;

[0098] (II) Level 5 quality: The data has been checked and corrected for formal errors and checked for substantive errors (operations 1, 2, and 3). There are still conflicts of substantive errors in this data, and no "Single Truth" has been formed. Data consistency cannot be ensured, but problems such as duplication, completeness, and standardization have been solved. The data is "not too much, not too little, and not chaotic." This quality is "multi-source conflict data quality."

[0099] (III) Level 4 quality: The data has been checked and corrected for formal errors and checked for substantive errors (operations 1, 2, and 3). However, unlike level 5 quality, these data have only a single source and do not have data conflicts, which ensures data consistency and has formed a "Single Truth". These data have solved the "five properties" of repeatability, completeness, standardization, consistency, and timeliness. The data is "not too much, not too little, not messy, not wrong, and not old", but the data values ​​have not been confirmed by "one number, one source, and multiple verification". This quality is "single evidence data quality".

[0100] (IV) Level 3 quality: The data has been checked and corrected for formal errors, checked for substantive errors, and corrected in 4.5 (operations 1, 2, 3, 4.5). The data has formed a "single truth" and can ensure the five core quality requirements (five qualities) of uniqueness (relevance), completeness, standardization, consistency, and timeliness. However, the data values ​​have not been confirmed by "one number, one source, and multiple verification". This quality is "multi-source consistent data quality".

[0101] (V) Secondary quality: The data has been checked for formal errors, corrected, checked for substantive errors, and corrected in 4.1 or 4.2 (operations 1, 2, 3, 4.1, 4.2). The data has formed a "Single Truth" and can ensure the five core quality requirements (five qualities) of uniqueness (relevance), completeness, standardization, consistency, and timeliness. The data value has been confirmed by "one number and one source" but not confirmed by "multiple verification". This quality is "one number and one source confirmed quality".

[0102] (VI) Level 1 quality: The data has been checked and corrected for formal errors, checked for substantive errors, and corrected in 4.3 or 4.4 (operations 1, 2, 3, 4.3, 4.4). The data has formed a "Single Truth" and can ensure the five core quality requirements (five qualities) of uniqueness (relevance), completeness, standardization, consistency, and timeliness. The data values ​​have been confirmed by "one number, one source, and multiple verifications". This quality is "multiple verification and confirmation quality".

[0103] The results of data cleaning can be colored according to the above five levels of quality, such as:

[0104] a) Green: Level 1 and 2 quality, can be used in various application scenarios;

[0105] b) Yellow: Level 3 quality, can be used in analytical and credit application scenarios;

[0106] c) Gray: Level 4 quality, which can be used in one scenario of analytical applications;

[0107] d) Red: Level 5 quality, can be used in credit-based application scenarios.

[0108] The standardization of the data cleaning process includes: performing pre-cleaning and post-cleaning on the data; dividing the pre-cleaning into cleaning the historical layer, standard layer and basic layer according to the standard, and dividing the post-cleaning into cleaning the integration layer, theme layer and application layer according to the standard.

[0109] Standardize cleaning management principles and processes by dividing and conquering:

[0110] Through the eight cleaning operations, two types of cleaning can be formed: pre-cleaning and post-cleaning. After pre-cleaning, the innate quality of the data can be formed, that is, the quality without substantial error correction. Since substantial errors have not been corrected, there is no third-level quality in the innate quality, only first, second, fourth, and fifth levels of quality. Pre-cleaning is the process of formal error checking, error correction, and substantial error checking.

[0111] The data that has been pre-cleaned can continue to be cleaned, and after post-cleaning - substantial errors have been corrected, it can form data quality levels one to five.

[0112] Six-layer data cleaning pipeline:

[0113] In addition to being divided into two stages, pre-cleaning and post-cleaning, the data cleaning process still needs to be further refined and standardized to ensure that the previous and subsequent steps are connected and that the cleaning results do not vary from time to time, from case to case, or from person to person.

[0114] The data cleaning process is as follows:

[0115] 1. Data collection

[0116] a) Directory layer

[0117] b) Aggregation layer

[0118] 2. Pre-cleaning

[0119] a) Historical layer: data versioning, historical data archiving, and incremental data processing;

[0120] b) Standard layer: check and correct formal errors;

[0121] c) Basic layer: checking of substantial errors, data granularization, data association and comparison;

[0122] 3. Post-cleaning

[0123] a) Integration layer: correction of substantial errors, logical integration of data, modeling and building of basic libraries;

[0124] b) Theme layer: data pre-calculation, modeling and database construction of theme database;

[0125] c) Application layer: customized processing of data, design and development of data interfaces.

[0126] Pre-cleaning and post-cleaning are further subdivided into six cleaning layers: historical layer, standard layer, basic layer, integration layer, theme layer, and application layer. These six layers form a data cleaning pipeline. Each layer focuses on cleaning a certain aspect of the data. Each layer cooperates with each other to form a data processing pipeline to complete the data cleaning work. Among these six layers of work, some are completed by machines, some require manual intervention, some are completed automatically, and some are completed semi-automatically.

[0127] Standardization of data cleaning rules:

[0128] In the six layers of processing, each layer has corresponding standards and rules. There are four types of standards that are often involved in each layer:

[0129] 1) Field naming standards: Unify the naming of fields that represent the same entity and the same attribute in each database and table so that automatic field mapping can be performed in subsequent processing.

[0130] a) Core fields: directly give the suggested field name, for example, the field name of gender can be "Gender".

[0131] b) Non-core fields: Provide suggested naming rules, such as naming fields with English words with the first letter capitalized, naming fields with Chinese words with the first letter capitalized, naming fields with Chinese characters, etc.

[0132] 2) Data element standard: Give data element attributes such as the data type of each field.

[0133] 3) Data encoding standards

[0134] a) For enumeration type fields, a code set consisting of all possible values ​​of the field and their encoding is given.

[0135] 4) Data model standard: mainly refers to the logical data model (LDM) standard.

[0136] a) Logical Data Model (LDM): The logical data model of normalized data.

[0137] b) Conceptual Data Model (CDM): A conceptual data model of normalized data, such as an entity relationship diagram.

[0138] c) Physical Data Model (PDM): A physical data model that normalizes actual data storage, such as a database's DDL script for an Oracle environment.

[0139] At the integration layer and thematic layer, the above four standards need to be given for each basic database and thematic database. At the data source layer, data standardization can be gradually implemented, and the above four standards in the business databases of various commissions and bureaus can be gradually specified for incremental data. At the principle layer, the focus should be on data element standards and data encoding standards. In addition, at the application layer, data interface standards should be formulated.

[0140] In addition, in data cleaning, the rules of great significance include:

[0141] 5) Data quality evaluation standards: Develop evaluation standards for the cleaning of formal errors such as uniqueness (entity identity, relevance), completeness, and standardization in data, as well as checking for substantive errors. This includes how to deduplicate and associate entities, how to complete fields, how to convert formats and models, and how to associate and compare data.

[0142] 6) Data cleaning rule standards: What kind of cleaning rules are used for formal errors and substantive errors. In particular, for conflicting values ​​of substantive errors, standardize the use scenarios, order and priority of cleaning rules such as one number one source rule, multi-check rule, majority rule, freshness rule, authority rule and self-consistency rule.

[0143] 7) One-number-one-source standard: For each field, the responsible unit of one-number-one-source under different conditions is given, which is a conditional one-number-one-source specification.

[0144] 8) Algorithm package standards: Develop the design, development, and interface specifications of algorithm packages for the detection and correction of formal errors.

[0145] Among them, the one-number-one-source standard is the prerequisite for achieving the second-level data quality, and the data cleaning rule standard is the prerequisite for achieving the third-level data quality.

[0146] Step b: using the meta-model to collect metadata of the data standard library, managing the collected metadata, and obtaining managed metadata;

[0147] Metamodels include relational metamodels, document metamodels, large database metamodels, and interface metamodels; metadata collected for governance includes management, analysis, and application of metadata;

[0148] Through the content defined by the metamodel, metadata is collected, compared, stored, and classified for related data sources through scheduled tasks. Metadata management includes the management of business metadata, technical metadata, and administrator data, such as basic metadata maintenance, metadata changes, metadata versions, metadata comparison, and other management. Through the information collected by metadata, metadata is analyzed to generate a global metadata map, lineage analysis, and impact analysis. Since metadata naturally has the property of being shareable after aggregation, combining metadata governance with data quality governance can quickly identify the root causes of data quality and impact analysis, helping users realize the value of metadata governance.

[0149] The basic information management of business metadata, technical metadata and management metadata includes automatic comparison of metadata of meta-models collected in real time to achieve incremental metadata updates; automatic discovery of metadata change information; automatic generation of metadata catalogs based on attributes; automatic control of metadata versions; and discovery of metadata gaps through version comparison;

[0150] The analysis of metadata includes lineage analysis, impact analysis and the generation of metadata maps; the application of metadata includes the use of the basic information of the managed metadata and the analyzed metadata to trace quality issues, analyze the impact of metadata quality issues and manage assets. The metadata map provides a data map display of global metadata, discovers the full-link relationship of data and can be transferred down in layers

[0151] Step c: standardize the data in the data standard library;

[0152] Create a data standard system, maintain and manage the created data standard system, and map the data standard system with business objects;

[0153] Conduct standard system conformity testing and standard system assessment according to standard rules;

[0154] Intelligent standardization: standardize and convert data according to the results of standard compliance detection.

[0155] The maintenance and management of the created data standard system includes managing the name of the standard package in the data standard or the description of the standard package in the data standard, configuring the jar package for the created data standard, importing the reference data, and also including previewing, editing and updating the jar package and reference data.

[0156] Business objects are obtained according to the data source definition.

[0157] The maintenance and management of the created data standard system also includes rule management, which unifies the management of data rules.

[0158] Unified management of data rules includes value range analysis, dictionary rules, function dependency rules, SQL rules, regular expressions, and inclusion dependencies.

[0159] The maintenance and management of the created data standard system also includes the versioning management of standard files in the data standards, and the classification of standard files in the data standards according to national standards, local standards, international standards and industry standards.

[0160] The maintenance and management of the created data standard system also includes unified management of data elements and associating data elements with standard packages in data standards.

[0161] Mapping the data standard system with the business object includes determining the data standard subject to which it belongs according to the meaning of the field of the business object, searching for the data standard in the corresponding data standard subject for association, and forming a mapping result table.

[0162] Standard system management provides data standard documents, data element standards, standard terms, standard terminology and other functions to help customers establish global standard management. Among them, standard package management + reference data provides data standard creation functions, which can dynamically expand various types of data standards. It realizes the maintenance and management of specific data standards, including the editing, modification and deletion functions of the name and description of the standard package, and can realize the configuration jar package of custom data standards, the import function of reference data, and can preview, edit and update the jar package and reference data.

[0163] The rule management realizes the unified management of data rules. The main management rules include: value range analysis, dictionary rules, function dependency rules, SQL rules, regular expression books, including dependencies, etc.

[0164] Standard management is to manage the versions of standard documents and classify them according to national standards, local standards, international standards and industry standards. Standard version management is provided to help customers check and compare the contents of standards in different periods.

[0165] Data element management is to manage data elements in a unified manner according to standards, and to associate data elements with standard packages in order to map data elements and business objects.

[0166] Business object management defines corresponding business objects according to the data source, and groups and categorizes the defined business objects; it can also generate substrings for the fields of business objects (i.e., customize data table attributes).

[0167] Step d: Perform data quality management and regulatory protection on the data generated in steps a-c.

[0168] In step d, performing data quality management on the data generated in steps a to c includes:

[0169] Formulate data quality monitoring and verification plans to monitor and verify data quality; formulate a data quality rule base; regularly perform data quality control according to the data quality rule base to identify data quality issues; manage data quality issues; and evaluate data quality;

[0170] Develop data quality monitoring and verification plans, including data quality control in business processes and data quality control in information systems;

[0171] The data quality control in the business process includes data quality control in the data generation stage, data quality control in the data integration stage and data quality control in the data use stage;

[0172] The data quality control in the information system includes the control of data quality problems generated by personnel, processes, business system front-ends, business system databases, extraction processes and loading processes in the information system.

[0173] The formulation of the data quality rule base includes:

[0174] Data quality requirements collection: collect data quality requirements, collect and organize quality issues, data usage quality issues, data process quality issues and overall data quality issues, discover the overall requirements for data quality, integrate the overall quality requirements, and confirm the overall data quality goals;

[0175] Data quality inspection data sorting, confirming the data quality inspection object, inspection data scope and indicator composition, sorting out the inspection data scope, discovering the core objects within the inspection data scope, and preliminarily deriving data quality inspection indicators, inspection rules, inspection methods, inspection cycles, inspection targets, scoring standards and data quality responsible persons and other information according to the object standard definition and business scenarios, and forming documents, and confirming and revising the contents according to the document contents;

[0176] Formulate data quality verification rules, and formulate data quality verification rules based on the data quality measurement indicators;

[0177] Data quality check rule management, including management of the public rule base, and reuse of public rules through SQL rules, regular rules, value range rules, and algorithm packages;

[0178] Rule configuration management, based on data detection indicators, the built-in rule engine implements quality detection;

[0179] And changes to data quality verification rules.

[0180] The data quality control is performed regularly according to the data quality rule base, including data quality rule analysis, data quality verification frequency, data quality monitoring scope formulation, data quality monitoring report generation, data quality assessment report generation and data quality comprehensive report generation.

[0181] The management of data quality issues includes data quality cycle management, data quality issue feedback, data quality issue cause analysis, data quality issue correction, data quality issue correction, and data quality issue rectification.

[0182] Data quality assessment includes evaluation of the core indicators of data quality assessment, data quality assessment model and data quality assessment management process.

[0183] The data quality assessment management process includes data quality demand analysis, determining the evaluation object and scope, selecting data quality dimensions and evaluation indicators, determining quality measurement and its evaluation method, applying the method for evaluation, result analysis and evaluation, and quality results and reports.

[0184] The supervisory protection performed on the data generated in steps a to c includes data classification management, data authorization management and data desensitization management.

[0185] Data hierarchical management is to allocate data according to data permissions, data authorization management is to allocate data permissions, and data desensitization management is to desensitize data.

[0186] Data desensitization management includes dividing data according to the data service objects or according to the data service users; formulating desensitization rules for the divided data; formulating desensitization tasks to discover sensitive data and formulating desensitization strategies.

[0187] Developing a desensitization strategy includes implementing data masking, encryption, security auditing, and log auditing for sensitive data.

[0188] like Figure 1 As shown, a data governance system includes a server and a client, wherein the server includes a memory and a processor, wherein the memory stores a plurality of instructions, and wherein the instructions are suitable for being loaded and executed by the processor of the server:

[0189] Step a: Clean and integrate the data sources and establish a data standard library;

[0190] Step b: using the meta-model to collect metadata of the data standard library, managing the collected metadata, and obtaining managed metadata;

[0191] Step c: standardize the data in the data standard library;

[0192] Step d: Perform data quality management and regulatory protection on the data generated in steps a to c; the client includes a display for displaying data in the data standard library, displaying metadata after governance, displaying data in the data standard library after standardization, and displaying content after data quality management.

[0193] The data visualization process includes extracting data source data, data standard content, data quality content and metadata according to indicators, visualizing the extracted data source data, data standard data content, data quality data content and metadata through interfaces, and providing third-party service interfaces, which include data query interfaces and data editing interfaces. A series of operations on data through interfaces are all implemented using code.

[0194] The present invention provides a data governance method and system, including cleaning and fusing data sources to establish a data standard library; using a meta-model to collect metadata of the data standard library, governing the collected metadata, and obtaining governed metadata; standardizing the data of the data standard library; performing data quality management and regulatory protection on the generated data, and ultimately solving the problem of data waste.

[0195] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and the storage medium may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0196] A person skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. The above-mentioned storage medium can be a read-only memory, a disk or an optical disk, etc.

[0197] The above is a detailed introduction to a data analysis method and system provided by the present invention. For a person skilled in the art, according to the idea of ​​the embodiments of the present invention, there may be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A data governance method, characterized in that: include: Step a: Clean and integrate the data sources and establish a data standard library, which includes relational databases, file databases, large databases and interface databases; Data source cleaning and integration methods, including: standardization of data sources and data application scenarios, standardization of data problems and data governance effects, and standardization of data cleaning processes; The standardization of the data cleaning process includes: performing pre-cleaning and post-cleaning on the data; the pre-cleaning is divided into cleaning the historical layer, the standard layer and the basic layer according to the standard, and the post-cleaning is divided into cleaning the integration layer, the theme layer and the application layer according to the standard; The data cleaning process is as follows:

1. Data collection: a) Directory layer; b) aggregation layer; 2. Pre-cleaning: a) Historical layer: data versioning, historical data archiving, and incremental data processing; b) Standard layer: check and correct formal errors; c) Basic layer: checking of substantial errors, data granularization, data association and comparison; 3. Post-cleaning: a) Integration layer: correction of substantial errors, logical integration of data, modeling and building of basic libraries; b) Theme layer: data pre-calculation, modeling and database construction of theme database; c) Application layer: customized processing of data, design and development of data interfaces; Standardization of data cleaning rules: 1) Field naming standards: Unify the naming of fields that represent the same entity and the same attribute in each database and table so that automatic field mapping can be performed in subsequent processing: a) Core fields: directly give suggested field names; non-core fields: give suggested naming rules; 2) Data element standard: Give the data type of each field; 3) Data encoding standards: a) For enumeration fields, give all possible values ​​of the field and the code set formed by their encoding; 4) Data model standard: Logical Data Model (LDM) standard: a) Logical Data Model (LDM): The logical data model of normalized data; b) Conceptual Data Model (CDM): conceptual data model of normalized data; c) Physical Data Model (PDM): A physical data model that normalizes actual data storage; 5) Data quality assessment standards; 6) Data cleaning rules and standards; The standardization of data sources and data application scenarios includes: standardizing data sources through data research and data cataloging; Abstract data application scenarios into business models and data models to standardize data application scenarios; The data research includes research on information resources, data requirements, data standards, data quality and data security; The data application scenarios include operational application scenarios, analytical application scenarios and credit application scenarios; Step b: using the meta-model to collect metadata of the data standard library, managing the collected metadata, and obtaining managed metadata; Step c: standardize the data in the data standard library; Step c includes: Create a data standard system, maintain and manage the created data standard system, and map the data standard system with business objects; Maintenance and management of the created data standard system includes management of the name of the standard package in the data standard or the description of the standard package in the data standard, configuration of the jar package for the created data standard, import of reference data, and preview, editing and updating of the jar package and reference data; The maintenance and management of the created data standard system also includes rule management, which is the unified management of data rules; Unified management of data rules includes value range analysis, dictionary rules, function dependency rules, SQL rules, regular expressions, and inclusion dependencies; The maintenance and management of the created data standard system also includes the versioning management of standard files in the data standard, and the classification of standard files in the data standard according to national standards, local standards, international standards and industry standards; Maintenance and management of the created data standard system also includes unified management of data elements and associating data elements with standard packages in data standards; Mapping the data standard system with the business object includes determining the data standard subject to which it belongs according to the meaning of the field of the business object, searching for the data standard in the corresponding data standard subject for association, and forming a mapping result table; Conduct standard system conformity testing and standard system assessment according to standard rules; Intelligent standardization: data is converted and standardized according to the standard compliance test results; Step d: Performing data quality management and regulatory protection on the data generated in steps a to c, wherein performing data quality management on the data generated in steps a to c in step d includes: Formulate a data quality monitoring and verification plan to monitor and verify data quality; the formulation of a data quality monitoring and verification plan includes data quality control in business processes and data quality control in information systems; The data quality control in the business process includes data quality control in the data generation stage, data quality control in the data integration stage and data quality control in the data use stage; The data quality control in the information system includes the control of data quality issues generated by personnel, processes, business system front end, business system database, extraction process and loading process in the information system; Formulate a data quality rule base; the formulating a data quality rule base includes: Data quality requirements collection: collect data quality requirements, collect and organize quality issues, data usage quality issues, data process quality issues and overall data quality issues, discover the overall requirements for data quality, integrate the overall quality requirements, and confirm the overall data quality goals; Data quality inspection data sorting, confirming the data quality inspection object, inspection data scope and indicator composition, sorting out the inspection data scope, discovering the core objects within the inspection data scope, and preliminarily deriving data quality inspection indicators, inspection rules, inspection methods, inspection cycles, inspection targets, scoring standards and data quality responsible person information according to the object standard definition and business scenarios, and forming documents, and confirming and revising the contents according to the document contents; Formulate data quality verification rules, and formulate data quality verification rules based on the data quality measurement indicators; Data quality check rule management, including management of the public rule base, and reuse of public rules through SQL rules, regular rules, value range rules, and algorithm packages; Rule configuration management: Based on data detection indicators, the built-in rule engine implements quality detection and changes to data quality inspection rules; Performing data quality control regularly according to the data quality rule base to obtain data quality problems, wherein the regularly performing data quality control according to the data quality rule base includes analyzing data quality rules, frequency of data quality verification, formulating data quality monitoring scope, generating data quality monitoring report, generating data quality assessment report and generating data quality comprehensive report; Data quality problem management includes data quality cycle management, data quality problem feedback, data quality problem cause analysis, data quality problem correction, and data quality problem rectification; Data quality assessment includes the core indicators of data quality assessment, data quality assessment model and data quality assessment management process. The data quality assessment management process includes data quality demand analysis, determination of evaluation objects and scope, selection of data quality dimensions and evaluation indicators, determination of quality measurement and evaluation methods, application of methods for evaluation, result analysis and evaluation, quality results and reports; The supervision and protection of the data generated in steps a to c includes data classification management, data authorization management, and data desensitization management; Data classification management is to allocate data according to data permissions, data authorization management is to allocate data permissions, and data desensitization management is to desensitize data; data desensitization management includes dividing data according to the service object of the data or according to the data service user; Formulate desensitization rules for the divided data; Formulate desensitization tasks to discover sensitive data and formulate desensitization strategies; Developing a desensitization strategy includes implementing data masking, encryption, security auditing, and log auditing for sensitive data.

2. The data governance method according to claim 1, characterized in that: Governing the collected metadata in step b includes managing, analyzing and applying the metadata; The management of metadata includes the management of basic information of business metadata, technical metadata and management metadata; Analysis of metadata includes lineage analysis, impact analysis, and generation of metadata maps; The application of metadata includes using the basic information of managed metadata and analyzed metadata to trace quality issues, analyze the impact of metadata quality issues, and manage assets.

3. A data governance system, characterized in that: The invention comprises a server and a client, wherein the server comprises a memory and a processor, wherein the memory stores a plurality of instructions, and the instructions are suitable for being loaded and executed by the processor of the server: Step a: Clean and integrate the data sources and establish a data standard library; the data standard library includes relational databases, file databases, large databases and interface databases; Data source cleaning and integration methods, including: standardization of data sources and data application scenarios, standardization of data problems and data governance effects, and standardization of data cleaning processes; The standardization of the data cleaning process includes: performing pre-cleaning and post-cleaning on the data; the pre-cleaning is divided into cleaning the historical layer, the standard layer and the basic layer according to the standard, and the post-cleaning is divided into cleaning the integration layer, the theme layer and the application layer according to the standard; The data cleaning process is as follows:

1. Data collection: a) Directory layer; b) aggregation layer; 2. Pre-cleaning: a) Historical layer: data versioning, historical data archiving, and incremental data processing; b) Standard layer: check and correct formal errors; c) Basic layer: checking of substantial errors, data granularization, data association and comparison; 3. Post-cleaning: a) Integration layer: correction of substantial errors, logical integration of data, modeling and building of basic libraries; b) Theme layer: data pre-calculation, modeling and database construction of theme database; c) Application layer: customized processing of data, design and development of data interfaces; Standardization of data cleaning rules: 1) Field naming standards: Unify the naming of fields that represent the same entity and the same attribute in each database and table so that automatic field mapping can be performed in subsequent processing: a) Core fields: directly give suggested field names; non-core fields: give suggested naming rules; 2) Data element standard: Give the data type of each field; 3) Data encoding standards: a) For enumeration fields, give all possible values ​​of the field and the code set formed by their encoding; 4) Data model standard: Logical Data Model (LDM) standard: a) Logical Data Model (LDM): The logical data model of normalized data; b) Conceptual Data Model (CDM): conceptual data model of normalized data; c) Physical Data Model (PDM): A physical data model that normalizes actual data storage; 5) Data quality assessment standards; 6) Data cleaning rules and standards; The standardization of data sources and data application scenarios includes: standardizing data sources through data research and data cataloging; Abstract data application scenarios into business models and data models to standardize data application scenarios; The data research includes research on information resources, data requirements, data standards, data quality and data security; The data application scenarios include operational application scenarios, analytical application scenarios and credit application scenarios; Step b: using the meta-model to collect metadata of the data standard library, managing the collected metadata, and obtaining managed metadata; Step c: standardize the data in the data standard library; Step c includes: Create a data standard system, maintain and manage the created data standard system, and map the data standard system with business objects; Maintenance and management of the created data standard system includes management of the name of the standard package in the data standard or the description of the standard package in the data standard, configuration of the jar package for the created data standard, import of reference data, and preview, editing and updating of the jar package and reference data; The maintenance and management of the created data standard system also includes rule management, which is the unified management of data rules; Unified management of data rules includes value range analysis, dictionary rules, function dependency rules, SQL rules, regular expressions, and inclusion dependencies; The maintenance and management of the created data standard system also includes the versioning management of standard files in the data standard, and the classification of standard files in the data standard according to national standards, local standards, international standards and industry standards; Maintenance and management of the created data standard system also includes unified management of data elements and associating data elements with standard packages in data standards; Mapping the data standard system with the business object includes determining the data standard subject to which it belongs according to the meaning of the field of the business object, searching for the data standard in the corresponding data standard subject for association, and forming a mapping result table; Conduct standard system conformity testing and standard system assessment according to standard rules; Intelligent standardization: data is converted and standardized according to the standard compliance test results; Step d: Performing data quality management and regulatory protection on the data generated in steps a to c, wherein performing data quality management on the data generated in steps a to c in step d includes: Formulate a data quality monitoring and verification plan to monitor and verify data quality; the formulation of a data quality monitoring and verification plan includes data quality control in business processes and data quality control in information systems; The data quality control in the business process includes data quality control in the data generation stage, data quality control in the data integration stage and data quality control in the data use stage; The data quality control in the information system includes the control of data quality issues generated by personnel, processes, business system front end, business system database, extraction process and loading process in the information system; Formulate a data quality rule base; the formulating a data quality rule base includes: Data quality requirements collection: collect data quality requirements, collect and organize quality issues, data usage quality issues, data process quality issues and overall data quality issues, discover the overall requirements for data quality, integrate the overall quality requirements, and confirm the overall data quality goals; Data quality inspection data sorting, confirming the data quality inspection object, inspection data scope and indicator composition, sorting out the inspection data scope, discovering the core objects within the inspection data scope, and preliminarily deriving data quality inspection indicators, inspection rules, inspection methods, inspection cycles, inspection targets, scoring standards and data quality responsible person information according to the object standard definition and business scenarios, and forming documents, and confirming and revising the contents according to the document contents; Formulate data quality verification rules, and formulate data quality verification rules based on the data quality measurement indicators; Data quality check rule management, including management of the public rule base, and reuse of public rules through SQL rules, regular rules, value range rules, and algorithm packages; Rule configuration management: Based on data detection indicators, the built-in rule engine implements quality detection and changes to data quality inspection rules; Performing data quality control regularly according to the data quality rule base to obtain data quality problems, wherein the regularly performing data quality control according to the data quality rule base includes analyzing data quality rules, frequency of data quality verification, formulating data quality monitoring scope, generating data quality monitoring report, generating data quality assessment report and generating data quality comprehensive report; Data quality problem management includes data quality cycle management, data quality problem feedback, data quality problem cause analysis, data quality problem correction, and data quality problem rectification; Data quality assessment includes the core indicators of data quality assessment, data quality assessment model and data quality assessment management process. The data quality assessment management process includes data quality demand analysis, determination of evaluation objects and scope, selection of data quality dimensions and evaluation indicators, determination of quality measurement and evaluation methods, application of methods for evaluation, result analysis and evaluation, quality results and reports; The supervision and protection of the data generated in steps a to c includes data classification management, data authorization management, and data desensitization management; Data classification management is to allocate data according to data permissions, data authorization management is to allocate data permissions, and data desensitization management is to desensitize data; data desensitization management includes dividing data according to the service object of the data or according to the data service user; Formulate desensitization rules for the divided data; Formulate desensitization tasks to discover sensitive data and formulate desensitization strategies; Develop desensitization strategies including data masking, encryption, security audits and log audits for sensitive data; The client includes a display for displaying data in the data standard library, displaying metadata after governance, displaying data in the data standard library after standardization, and displaying content after data quality management.

4. A data management system according to claim 3, characterized in that: Governing the collected metadata in step b includes managing, analyzing and applying the metadata; The management of metadata includes the management of basic information of business metadata, technical metadata and management metadata; Analysis of metadata includes lineage analysis, impact analysis, and generation of metadata maps; The application of metadata includes using the basic information of managed metadata and analyzed metadata to trace quality issues, analyze the impact of metadata quality issues, and manage assets.