Data governance analysis method and system

Abstract

The invention provides a data governance analysis method and system, and relates to the technical field of data analysis, and the method comprises the steps: monitoring various types of data generated in a business operation process in real time, and carrying out the processing and classification, and obtaining a first processing data set; obtaining a data type corresponding to each first processing data subset, screening a corresponding data quality evaluation standard from the evaluation database, and constructing a comprehensive quality evaluation system, thereby performing data evaluation; determining a data quality problem of each piece of first processing data based on the data evaluation result, and performing inspection and tracking to obtain a first analysis result of each piece of first processing data; and synthesizing each first analysis result in the first processing data set, thereby performing corresponding data visualization on a data synthesis result. According to the method, the data of each data source is classified, so that classification analysis is performed, personalized data visualization is realized, data analysis is more accurate and deep, and value judgment can be more effectively performed.

Description

Technical Field

[0001] The present invention relates to the field of data analysis technology, and in particular to a data governance analysis method and system. Background Art

[0002] At present, with the rapid development of information technology, data has become an important asset of enterprises.

[0003] However, the data governance process faces problems such as low data quality, poor data security, and insufficient data value mining. Traditional data governance methods often focus on data storage and management, while ignoring in-depth data analysis and value mining.

[0004] Therefore, the present invention provides a data governance analysis method and system. Summary of the invention

[0005] The present invention provides a data governance analysis method and system to solve the problems in the prior art such as low data quality and inaccurate and unclear value judgment of data.

[0006] The present invention provides a data governance analysis method, comprising: Step 1: monitor various types of data generated during the business operation process in real time, and process and classify the various types of data obtained through monitoring to obtain a first processed data set; Step 2: Obtain the data type corresponding to each first processed data subset in the first processed data set, thereby screening the corresponding data quality assessment standard from the assessment database, and constructing a comprehensive quality assessment system, thereby performing data assessment on the first processed data in the first processed data set; Step 3: determining the data quality problem of each first processed data in the first processed data set based on the data evaluation result, and performing inspection and tracking, thereby obtaining a first analysis result of each first processed data; Step 4: Integrate each first analysis result in the first processed data set, and then perform corresponding data visualization on the data integration results According to the present invention, various types of data generated during the real-time monitoring of business operations are processed and classified to obtain a first processed data set, including: Step 11: Monitor the data sources involved in the business operation process in real time, and collect data in real time to obtain initial collected data; Step 12: Perform data cleaning and data standardization on the initial collected data to obtain initial processed data; Step 13: Classify the initial processed data based on the data properties and usage, and sort the classified data subsets to obtain a first processed data set.

[0007] According to the present invention, the data type corresponding to each first processed data subset in the first processed data set is obtained, so as to screen the corresponding data quality assessment standard from the assessment database, and construct a comprehensive quality assessment system, so as to perform data assessment on the first processed data in the first processed data set, including: Step 21: Obtain the classification type corresponding to each first processed data subset in the first processed data set and the data type of each first processed data in the current first processed data subset, thereby comprehensively determining the first data type of the first processed data; Step 22: based on the first data type, a corresponding data quality assessment standard is screened from the assessment database to obtain a first quality assessment standard for the current first processed data; Step 23: obtaining a first quality assessment standard set based on each first quality assessment standard of the same first processed data subset; Step 24: Classify each first quality assessment standard in the first quality assessment standard set according to the assessment type to obtain a second quality assessment standard set; Step 25: Determine whether each second quality assessment standard in the second quality assessment standard set conflicts; If there is a conflict in the second quality assessment standard, the current second quality assessment standard set is divided into two assessment standard subsets, so as to perform quality assessment standard judgments respectively; If there is no second quality assessment standard conflict, extracting the second quality assessment standard with the highest assessment standard in each second quality assessment standard subset in the second quality assessment standard set as the first benchmark quality assessment standard; Step 26: Integrate each first reference quality assessment standard in the second quality assessment standard set to obtain the first quality assessment system of the current second quality assessment standard set, thereby obtaining an initial integrated quality assessment system of the current first processed data set; Step 27: Evaluate each first processed data in the first processed data set based on the comprehensive quality evaluation system to obtain a data evaluation result.

[0008] According to the present invention, the current second quality evaluation standard set is divided into two evaluation standard subsets, so as to perform quality evaluation standard judgment respectively, including: Step 251: acquiring two second quality assessment criteria in conflict with each other in the second quality assessment criteria set, and obtaining a third quality assessment criteria and a fourth quality assessment criteria; Step 252: determining whether the remaining second quality assessment criteria in the second quality assessment criteria set conflict with the third quality assessment criteria; If not, combining the remaining second quality assessment criteria in the second quality assessment criteria set with the third quality assessment criteria to obtain a third quality assessment criteria set; Extracting the third quality assessment standard with the highest assessment standard in each third quality assessment standard subset in the third quality assessment standard set as the first benchmark quality assessment standard; At the same time, the fourth quality assessment standard is used as the first benchmark instruction assessment standard.

[0009] The comprehensive quality evaluation system provided by the present invention is used to evaluate each first processed data in the first processed data set to obtain a data evaluation result, including: Performing data evaluation on the current first processed data to obtain a first data evaluation result S of the current first processed data; ; Wherein, S is the first data evaluation result of the current first processed data, is the historical data evaluation result of the i-th historical processing data of the intelligent terminal that has the same data type as the current first processing data, is the influence factor of the historical data evaluation result of the i-th historical processed data of the intelligent terminal that has the same data type as the current first processed data on the current first data evaluation result, is the influence weight of the historical data evaluation result on the first data evaluation result, is the criticality of the data type corresponding to the current first processed data, is the data integrity indicator of the current first processed data, is the weight of the impact of the criticality of the data type on the first data evaluation result, is the influence weight of the data integrity index on the first data evaluation result, where n is the number of historical processed data of the smart terminal, is the natural logarithm, .

[0010] According to the present invention, determining the data quality problem of each first processed data in the first processed data set based on the data evaluation result, and performing inspection and tracking to obtain the first analysis result of each first processed data includes: Step 31: determining the data quality problem of each first processed data in the first processed data set based on the data evaluation result, and obtaining a first data quality set; Step 32: Classify each data quality problem in the first data quality set according to different quality problem types, thereby obtaining a first classified quality set; Step 33: obtaining the quality problem type of each first classification quality subset in the first classification quality set, and selecting the corresponding initial quality solution from the quality-solution database; Step 34: Optimizing the corresponding initial quality solution based on the data value of the first processed data in each classified quality subset to obtain a first quality solution for each first processed data; Step 35: performing data processing on the corresponding first processed data based on the first quality solution, thereby obtaining second processed data; Step 36: Perform data evaluation on the second processed data based on the comprehensive quality evaluation system to obtain second evaluation data, and obtain a first analysis result of the first processed data based on the data processing evaluation process from the first processed data to the second evaluation data.

[0011] According to the present invention, each first analysis result in the first processed data set is integrated, thereby performing corresponding data visualization on the data integration result, including: Step 41: Integrate each first analysis result in the first processed data set to obtain a comprehensive data quality report; Step 42: Obtain the data characteristics of the first processed data corresponding to the first analysis result and the visualization requirements of the intelligent terminal to determine a visualization tool for visualizing the first processed data; Step 43: Based on the visualization tool, corresponding data visualization is performed on the content of the comprehensive data quality report to obtain a first visualization result, thereby achieving data visualization.

[0012] The present invention provides a data governance analysis system, comprising: Data processing module: used for real-time monitoring of various types of data generated during business operations, and processing and classifying various types of data obtained through monitoring to obtain a first processed data set; Data evaluation module: used to obtain the data type corresponding to each first processed data subset in the first processed data set, thereby screening the corresponding data quality evaluation standard from the evaluation database, building a comprehensive quality evaluation system, and thus performing data evaluation on the first processed data in the first processed data set; Data analysis module: used to determine the data quality problem of each first processed data in the first processed data set based on the data evaluation result, and perform inspection and tracking to obtain a first analysis result of each first processed data; Data visualization module: used to integrate each first analysis result in the first processed data set, so as to perform corresponding data visualization on the data integration result.

[0013] Compared with the prior art, the beneficial effects of the present invention are as follows: the data governance analysis method and system provided by the present invention classifies the data of each data source, thereby performing classification analysis and realizing personalized data visualization, making the analysis of the data more accurate and in-depth, thereby enabling more effective value judgment. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0015] Figure 1 is a flow chart of a data governance analysis method provided by an embodiment of the present invention; Figure 2 It is a structural diagram of a data governance analysis system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0016] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0017] Embodiment 1: The embodiment of the present invention provides a data governance analysis method, such as Figure 1 As shown, including: Step 1: monitor various types of data generated during the business operation process in real time, and process and classify the various types of data obtained through monitoring to obtain a first processed data set; Step 2: Obtain the data type corresponding to each first processed data subset in the first processed data set, thereby screening the corresponding data quality assessment standard from the assessment database, and constructing a comprehensive quality assessment system, thereby performing data assessment on the first processed data in the first processed data set; Step 3: determining the data quality problem of each first processed data in the first processed data set based on the data evaluation result, and performing inspection and tracking, thereby obtaining a first analysis result of each first processed data; Step 4: Integrate each first analysis result in the first processed data set, and perform corresponding data visualization on the data integration result.

[0018] In this embodiment, real-time monitoring refers to the continuous and uninterrupted monitoring and recording of various types of data generated during business operations, which usually involves the use of specific software tools or systems to capture data for subsequent analysis and processing.

[0019] In this embodiment, the first processed data set refers to a data set that has undergone preliminary processing (such as cleaning, sorting, etc.). These data are extracted from the original data obtained from real-time monitoring and have undergone some form of preprocessing to facilitate subsequent analysis and evaluation.

[0020] In this embodiment, the data type refers to the nature and classification of data, such as number, text, date, etc.

[0021] In this embodiment, the evaluation database is a database storing various data quality evaluation standards, which are used to measure quality indicators such as accuracy, completeness, and consistency of data.

[0022] In this embodiment, the comprehensive quality assessment system is a system constructed based on the data quality assessment standard in the assessment database, and is used to perform a comprehensive and integrated assessment on the data in the first processed data set.

[0023] In this embodiment, data evaluation refers to a process of evaluating the data in the first processed data set according to a comprehensive quality evaluation system.

[0024] In this embodiment, data quality issues refer to data quality issues discovered during the data evaluation process, such as data errors, missing data, inconsistencies, etc.

[0025] In this embodiment, verification tracking refers to further inspection and analysis of data quality issues to determine the cause of the problem and possible solutions.

[0026] In this embodiment, the first analysis result refers to the analysis result obtained after performing data quality assessment on each first processed data, determining data quality problems, and performing verification and tracking.

[0027] In this embodiment, data visualization refers to the process of displaying data in the form of graphics, images, animations, etc.

[0028] In this embodiment, for example, in a certain retail enterprise, the data governance analysis method proposed in the present invention is applied to collect and analyze customer purchase records, preferences, behaviors and other data. Through data analysis, it is found that the sales volume of a certain type of product shows a significant growth trend in a specific season. Based on this finding, the enterprise adjusted its marketing strategy, increased the promotion of such products, and prepared stocks in advance before the season, ultimately achieving a substantial increase in sales.

[0029] The beneficial effect of the above technical solution is: by classifying and processing the data of each data source, classification analysis is carried out to achieve personalized data visualization, making the analysis of the data more accurate and in-depth, thereby enabling more effective value judgment.

[0030] Embodiment 2: Based on Example 1, various types of data generated during the business operation process are monitored in real time, and various types of data obtained by monitoring are processed and classified to obtain a first processed data set, including: Step 11: Monitor the data sources involved in the business operation process in real time, and collect data in real time to obtain initial collected data; Step 12: Perform data cleaning and data standardization on the initial collected data to obtain initial processed data; Step 13: Classify the initial processed data based on the data properties and usage, and sort the classified data subsets to obtain a first processed data set.

[0031] In this embodiment, the data source includes a database, a log file, sensor data, etc.

[0032] In this embodiment, the initial collected data refers to the collected data obtained after real-time monitoring of various data sources involved in the business operation process and data collection.

[0033] In this embodiment, data cleaning refers to the process of identifying and correcting (or deleting) erroneous, duplicate, incomplete or abnormal data in a data set.

[0034] In this embodiment, data standardization is a process of scaling data so that it falls into a small specific interval (usually 0 to 1 or -1 to 1). The purpose of standardization is to eliminate the dimensional effects between different types of data (variables) so that they have the same scale, thereby facilitating comparison and weighting. For example, the method of data standardization is generally Min-Max standardization or Z-score standardization.

[0035] In this embodiment, data classification refers to classifying the initial processing data according to factors such as the nature and purpose of the data. For example, the initial processing data is classified into user behavior data, transaction data, system log data, etc.

[0036] In this embodiment, the first processed data set refers to a processed data set obtained by classifying the initial processed data based on the data properties and usage, and sorting the classified data subsets.

[0037] The beneficial effect of the above technical solution is: by performing data cleaning and data standardization on the data of each data source, and classifying the processed data, it is possible to perform classification analysis more accurately, making the analysis of the data more accurate and in-depth.

[0038] Embodiment 3: Based on Example 2, the data type corresponding to each first processed data subset in the first processed data set is obtained, so as to screen the corresponding data quality assessment standard from the assessment database, and construct a comprehensive quality assessment system, so as to perform data assessment on the first processed data in the first processed data set, including: Step 21: Obtain the classification type corresponding to each first processed data subset in the first processed data set and the data type of each first processed data in the current first processed data subset, thereby comprehensively determining the first data type of the first processed data; Step 22: based on the first data type, a corresponding data quality assessment standard is screened from the assessment database to obtain a first quality assessment standard for the current first processed data; Step 23: obtaining a first quality assessment standard set based on each first quality assessment standard of the same first processed data subset; Step 24: Classify each first quality assessment standard in the first quality assessment standard set according to the assessment type to obtain a second quality assessment standard set; Step 25: Determine whether each second quality assessment standard in the second quality assessment standard set conflicts; If there is a conflict in the second quality assessment standard, the current second quality assessment standard set is divided into two assessment standard subsets, so as to perform quality assessment standard judgments respectively; If there is no second quality assessment standard conflict, extracting the second quality assessment standard with the highest assessment standard in each second quality assessment standard subset in the second quality assessment standard set as the first benchmark quality assessment standard; Step 26: Integrate each first reference quality assessment standard in the second quality assessment standard set to obtain the first quality assessment system of the current second quality assessment standard set, thereby obtaining an initial integrated quality assessment system of the current first processed data set; Step 27: Evaluate each first processed data in the first processed data set based on the comprehensive quality evaluation system to obtain a data evaluation result.

[0039] In this embodiment, the data type refers to the nature and classification of data, such as number, text, date, etc.

[0040] In this embodiment, the evaluation database is a database storing various data quality evaluation standards, which are used to measure quality indicators such as accuracy, completeness, and consistency of data.

[0041] In this embodiment, the first quality evaluation standard is a quality evaluation standard selected from an evaluation database and corresponding to the first data type.

[0042] In this embodiment, the first quality evaluation standard set refers to a set of multiple first quality evaluation standards screened based on different aspects or dimensions for the same first processed data subset.

[0043] In this embodiment, the evaluation type refers to the classification method of the quality evaluation standard, such as accuracy, completeness, consistency, etc.

[0044] In this embodiment, the second quality assessment standard set is a set obtained by classifying the standards in the first quality assessment standard set according to the assessment type.

[0045] In this embodiment, the evaluation standard subset refers to two subsets into which the current second quality evaluation standard set is divided in order to resolve the conflict of the second quality evaluation standards.

[0046] In this embodiment, quality assessment standard determination refers to evaluating and comparing the standards in the assessment standard subset to determine which standard is more appropriate or more accurate.

[0047] In this embodiment, the first reference quality evaluation standard refers to the second quality evaluation standard with the highest evaluation standard under the same evaluation type.

[0048] In this embodiment, the first quality assessment system refers to an assessment system obtained by integrating a plurality of first reference quality assessment standards, and is used to perform overall quality assessment on the first processed data set.

[0049] In this embodiment, the initial comprehensive quality assessment system refers to an initial quality assessment system obtained based on the first quality assessment system for the current first processed data set.

[0050] In this embodiment, the comprehensive quality assessment system is a system constructed based on the data quality assessment standard in the assessment database, and is used to perform a comprehensive and integrated assessment on the data in the first processed data set.

[0051] In this embodiment, data evaluation refers to a process of evaluating the data in the first processed data set according to a comprehensive quality evaluation system.

[0052] The beneficial effect of the above technical solution is: different processed data are classified and evaluated through different quality assessment standards, so as to more accurately judge the data quality problems of the first processed data, thereby achieving more accurate data analysis of the first processed data, which can make the analysis of the data more accurate and in-depth, so as to make value judgments more effective.

[0053] Embodiment 4: Based on Example 3, the current second quality assessment standard set is divided into two assessment standard subsets, so as to perform quality assessment standard judgments respectively, including: Step 251: acquiring two second quality assessment criteria in conflict with each other in the second quality assessment criteria set, and obtaining a third quality assessment criteria and a fourth quality assessment criteria; Step 252: determining whether the remaining second quality assessment criteria in the second quality assessment criteria set conflict with the third quality assessment criteria; If not, combining the remaining second quality assessment criteria in the second quality assessment criteria set with the third quality assessment criteria to obtain a third quality assessment criteria set; Extracting the third quality assessment standard with the highest assessment standard in each third quality assessment standard subset in the third quality assessment standard set as the first benchmark quality assessment standard; At the same time, the fourth quality assessment standard is used as the first benchmark instruction assessment standard.

[0054] In this embodiment, the third quality evaluation standard and the fourth quality evaluation standard refer to two second quality evaluation standards in the second quality evaluation standard set that conflict with the second quality evaluation standard.

[0055] The beneficial effect of the above technical solution is: different processed data are classified and evaluated through different quality evaluation standards, so as to more accurately judge the data quality problems of the first processed data, thereby achieving more accurate data analysis of the first processed data.

[0056] Embodiment 5: Based on Example 3, each first processed data in the first processed data set is evaluated based on the comprehensive quality evaluation system to obtain a data evaluation result, including: Performing data evaluation on the current first processed data to obtain a first data evaluation result S of the current first processed data; ; Wherein, S is the first data evaluation result of the current first processed data, is the historical data evaluation result of the i-th historical processing data of the intelligent terminal that has the same data type as the current first processing data, is the influence factor of the historical data evaluation result of the i-th historical processed data of the intelligent terminal that has the same data type as the current first processed data on the current first data evaluation result, is the influence weight of the historical data evaluation result on the first data evaluation result, is the criticality of the data type corresponding to the current first processed data, is the data integrity indicator of the current first processed data, is the weight of the impact of the criticality of the data type on the first data evaluation result, is the influence weight of the data integrity index on the first data evaluation result, where n is the number of historical processed data of the smart terminal, is the natural logarithm, .

[0057] The beneficial effect of the above technical solution is: by determining the data evaluation result of the first processed data, the data quality problem of the first processed data can be judged more accurately, thereby achieving more accurate data analysis of the first processed data, which can make the analysis of the data more accurate and in-depth.

[0058] Embodiment 6: Based on the third embodiment, the data quality problem of each first processed data in the first processed data set is determined based on the data evaluation result, and verification tracking is performed to obtain the first analysis result of each first processed data, including: Step 31: determining the data quality problem of each first processed data in the first processed data set based on the data evaluation result, and obtaining a first data quality set; Step 32: Classify each data quality problem in the first data quality set according to different quality problem types, thereby obtaining a first classified quality set; Step 33: obtaining the quality problem type of each first classification quality subset in the first classification quality set, and selecting the corresponding initial quality solution from the quality-solution database; Step 34: Optimizing the corresponding initial quality solution based on the data value of the first processed data in each classified quality subset to obtain a first quality solution for each first processed data; Step 35: performing data processing on the corresponding first processed data based on the first quality solution, thereby obtaining second processed data; Step 36: Perform data evaluation on the second processed data based on the comprehensive quality evaluation system to obtain second evaluation data, and obtain a first analysis result of the first processed data based on the data processing evaluation process from the first processed data to the second evaluation data.

[0059] In this embodiment, data quality issues refer to data quality issues discovered during the data evaluation process, such as data errors, missing data, inconsistencies, etc.

[0060] In this embodiment, the first data quality set refers to a set consisting of data quality issues of each first processed data determined according to the data evaluation result.

[0061] In this embodiment, the types of quality issues include: integrity issues, security issues, redundancy issues, accuracy issues, consistency issues, etc.

[0062] In this embodiment, the first classified quality set refers to classifying the first data quality set according to different quality problem types.

[0063] In this embodiment, the initial quality solution refers to a quality solution corresponding to each subset in the first classification quality set screened from a quality-solution database.

[0064] In this embodiment, the first quality solution refers to a solution obtained by optimizing the corresponding initial quality solution based on the data value of the first processed data in each classified quality subset.

[0065] In this embodiment, the second processed data refers to processed data obtained after the first processed data is processed according to the first quality solution.

[0066] In this embodiment, the second evaluation data refers to evaluation data obtained by performing data evaluation on the second processed data according to the comprehensive quality evaluation system.

[0067] In this embodiment, the first analysis result refers to the analysis result obtained after performing data quality assessment on each first processed data, determining data quality problems, and performing verification and tracking.

[0068] The beneficial effect of the above technical solution is: by analyzing the data quality issues, the analysis results of each first processed data are obtained, and then classification analysis is performed to achieve personalized data visualization, making the analysis of the data more accurate and in-depth, so that value judgments can be made more effectively.

[0069] Embodiment 7: Based on Example 6, each first analysis result in the first processed data set is integrated, so as to perform corresponding data visualization on the data integration result, including: Step 41: Integrate each first analysis result in the first processed data set to obtain a comprehensive data quality report; Step 42: Obtain the data characteristics of the first processed data corresponding to the first analysis result and the visualization requirements of the intelligent terminal to determine a visualization tool for visualizing the first processed data; Step 43: Based on the visualization tool, corresponding data visualization is performed on the content of the comprehensive data quality report to obtain a first visualization result, thereby achieving data visualization.

[0070] In this embodiment, the comprehensive data quality report refers to rearranging and integrating each first analysis result corresponding to the first processed data set to obtain a comprehensive data quality judgment report of the data.

[0071] In this embodiment, the visualization tools include Tableau, Power BI, etc.

[0072] In this embodiment, performing corresponding visualization refers to designing appropriate visualization charts for the comprehensive data quality report based on visualization tools to visualize the data quality situation intuitively, wherein the visualization charts include: bar charts, line charts, pie charts, scatter charts, etc., wherein different visualization charts focus on different visualization angles and visualization requirements.

[0073] The beneficial effect of the above technical solution is: by comprehensively analyzing the data characteristics of the first processed data and the visualization requirements of the intelligent terminal, the data visualization tool for each data is determined, and personalized data visualization is performed, so that the analysis of the data is more accurate and in-depth, thereby enabling more effective value judgments.

[0074] Embodiment 8: The embodiment of the present invention provides a data governance analysis system, such as Figure 2 As shown, including: Data processing module: used for real-time monitoring of various types of data generated during business operations, and processing and classifying various types of data obtained through monitoring to obtain a first processed data set; Data evaluation module: used to obtain the data type corresponding to each first processed data subset in the first processed data set, thereby screening the corresponding data quality evaluation standard from the evaluation database, building a comprehensive quality evaluation system, and thus performing data evaluation on the first processed data in the first processed data set; Data analysis module: used to determine the data quality problem of each first processed data in the first processed data set based on the data evaluation result, and perform inspection and tracking to obtain a first analysis result of each first processed data; Data visualization module: used to integrate each first analysis result in the first processed data set, so as to perform corresponding data visualization on the data integration result.

[0075] The beneficial effect of the above technical solution is: by classifying and processing the data of each data source, classification analysis is carried out to achieve personalized data visualization, making the analysis of the data more accurate and in-depth, thereby enabling more effective value judgment.

[0076] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data governance analysis method, characterized in that: include: Step 1: monitor various types of data generated during the business operation process in real time, and process and classify the various types of data obtained through monitoring to obtain a first processed data set; Step 2: Obtain the data type corresponding to each first processed data subset in the first processed data set, thereby screening the corresponding data quality assessment standard from the assessment database, and constructing a comprehensive quality assessment system, thereby performing data assessment on the first processed data in the first processed data set; Step 3: determining the data quality problem of each first processed data in the first processed data set based on the data evaluation result, and performing inspection and tracking, thereby obtaining a first analysis result of each first processed data; Step 4: Integrate each first analysis result in the first processed data set, and perform corresponding data visualization on the data integration result.

2. A data governance analysis method according to claim 1, characterized in that: Monitor various types of data generated during the business operation process in real time, and process and classify the various types of data obtained through monitoring to obtain a first processed data set, including: Step 11: Monitor the data sources involved in the business operation process in real time, and collect data in real time to obtain initial collected data; Step 12: Perform data cleaning and data standardization on the initial collected data to obtain initial processed data; Step 13: Classify the initial processed data based on the data properties and usage, and sort the classified data subsets to obtain a first processed data set.

3. A data governance analysis method according to claim 2, characterized in that: Obtaining a data type corresponding to each first processed data subset in the first processed data set, thereby screening a corresponding data quality assessment standard from an assessment database, and constructing a comprehensive quality assessment system, thereby performing data assessment on the first processed data in the first processed data set, including: Step 21: Obtain the classification type corresponding to each first processed data subset in the first processed data set and the data type of each first processed data in the current first processed data subset, thereby comprehensively determining the first data type of the first processed data; Step 22: Filtering a corresponding data quality assessment standard from an assessment database based on the first data type to obtain a first quality assessment standard for the current first processed data; Step 23: obtaining a first quality assessment standard set based on each first quality assessment standard of the same first processed data subset; Step 24: Classify each first quality assessment standard in the first quality assessment standard set according to the assessment type to obtain a second quality assessment standard set; Step 25: Determine whether each second quality assessment standard in the second quality assessment standard set conflicts; If there is a conflict in the second quality assessment standard, the current second quality assessment standard set is divided into two assessment standard subsets, so as to perform quality assessment standard judgments respectively; If there is no second quality assessment standard conflict, extracting the second quality assessment standard with the highest assessment standard in each second quality assessment standard subset in the second quality assessment standard set as the first benchmark quality assessment standard; Step 26: Integrate each first reference quality assessment standard in the second quality assessment standard set to obtain the first quality assessment system of the current second quality assessment standard set, thereby obtaining an initial integrated quality assessment system of the current first processed data set; Step 27: Evaluate each first processed data in the first processed data set based on the comprehensive quality evaluation system to obtain a data evaluation result.

4. A data governance analysis method according to claim 3, characterized in that: The current second quality assessment standard set is divided into two assessment standard subsets, so as to perform quality assessment standard judgments respectively, including: Step 251: acquiring two second quality assessment criteria in conflict with each other in the second quality assessment criteria set, and obtaining a third quality assessment criteria and a fourth quality assessment criteria; Step 252: determining whether the remaining second quality assessment criteria in the second quality assessment criteria set conflict with the third quality assessment criteria; If not, combining the remaining second quality assessment criteria in the second quality assessment criteria set with the third quality assessment criteria to obtain a third quality assessment criteria set; Extracting the third quality assessment standard with the highest assessment standard in each third quality assessment standard subset in the third quality assessment standard set as the first benchmark quality assessment standard; At the same time, the fourth quality assessment standard is used as the first benchmark instruction assessment standard.

5. A data governance analysis method according to claim 3, characterized in that: Each first processed data in the first processed data set is evaluated based on the comprehensive quality evaluation system to obtain a data evaluation result, including: Performing data evaluation on the current first processed data to obtain a first data evaluation result S of the current first processed data; ; Wherein, S is the first data evaluation result of the current first processed data, is the historical data evaluation result of the i-th historical processing data of the intelligent terminal that has the same data type as the current first processing data, is the influence factor of the historical data evaluation result of the i-th historical processed data whose data type is consistent with the current first processed data on the current first data evaluation result, is the influence weight of the historical data evaluation result on the first data evaluation result, is the criticality of the data type corresponding to the current first processed data, is the data integrity indicator of the current first processed data, is the weight of the impact of the criticality of the data type on the first data evaluation result, is the influence weight of the data integrity index on the first data evaluation result, where n is the number of historical processed data of the smart terminal, is the natural logarithm, .

6. A data governance analysis method according to claim 3, characterized in that: Determining data quality issues of each first processed data in the first processed data set based on the data evaluation result, and performing verification tracking to obtain a first analysis result of each first processed data, including: Step 31: determining the data quality problem of each first processed data in the first processed data set based on the data evaluation result, and obtaining a first data quality set; Step 32: Classify each data quality problem in the first data quality set according to different quality problem types, thereby obtaining a first classified quality set; Step 33: obtaining the quality problem type of each first classification quality subset in the first classification quality set, and selecting the corresponding initial quality solution from the quality-solution database; Step 34: Optimizing the corresponding initial quality solution based on the data value of the first processed data in each classified quality subset to obtain a first quality solution for each first processed data; Step 35: performing data processing on the corresponding first processed data based on the first quality solution, thereby obtaining second processed data; Step 36: Perform data evaluation on the second processed data based on the comprehensive quality evaluation system to obtain second evaluation data, and obtain a first analysis result of the first processed data based on the data processing evaluation process from the first processed data to the second evaluation data.

7. A data governance analysis method according to claim 6, characterized in that: Each first analysis result in the first processed data set is integrated, thereby performing corresponding data visualization on the data integration result, including: Step 41: Integrate each first analysis result in the first processed data set to obtain a comprehensive data quality report; Step 42: Obtain the data characteristics of the first processed data corresponding to the first analysis result and the visualization requirements of the intelligent terminal to determine a visualization tool for visualizing the first processed data; Step 43: Based on the visualization tool, corresponding data visualization is performed on the content of the comprehensive data quality report to obtain a first visualization result, thereby achieving data visualization.

8. A data governance analysis system, characterized in that: include: Data processing module: used for real-time monitoring of various types of data generated during business operations, and processing and classifying various types of data obtained through monitoring to obtain a first processed data set; Data evaluation module: used to obtain the data type corresponding to each first processed data subset in the first processed data set, thereby screening the corresponding data quality evaluation standard from the evaluation database, building a comprehensive quality evaluation system, and thus performing data evaluation on the first processed data in the first processed data set; Data analysis module: used to determine the data quality problem of each first processed data in the first processed data set based on the data evaluation result, and perform inspection and tracking to obtain a first analysis result of each first processed data; Data visualization module: used to integrate each first analysis result in the first processed data set, so as to perform corresponding data visualization on the data integration result.

Images