Data asset quality assessment method and system

By designing a complete set of data asset quality assessment processes and functional frameworks, we solve the problems of incomplete, unsafe, and inflexible data asset quality assessment in existing technologies, and achieve efficient and professional data asset quality assessment and continuous improvement suggestions. .

CN119336740AInactive Publication Date: 2025-01-21北京鸿鹄元数科技有限公司

Patent Information

Application Number
CN202411432615.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-14
Publication Date
2025-01-21
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The existing data asset quality assessment plan lacks comprehensive and complete assessment capabilities, cannot provide full-process data quality assessment, and has security, flexibility and objectivity problems.

Method used

Designed a complete set of data asset quality evaluation process and function framework, including creating an independent evaluation user and data space, providing flexible assessment index system configuration capabilities, realizing data pre -processing, generating professional evaluation reports, and providing continuous improvement suggestions Essence

Benefits of technology

A more objective, comprehensive, and secure data asset quality assessment has been achieved, the security of the assessment data is guaranteed, the ability to evaluate the index configuration has been provided, and the objectivity and professionalism of the evaluation results are improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119336740A_ABST
    Figure CN119336740A_ABST
Patent Text Reader

Abstract

The invention provides a data asset quality evaluation method and system. The method comprises the following steps: creating an evaluation user; configuring a data source and acquiring metadata; managing the data table; configuring an evaluation rule; configuring an evaluation task; configuring an evaluation report template; generating an evaluation report; exporting an evaluation log; and recovering and evaluating the user permission and destroying the data space. According to the invention, a set of complete data asset quality evaluation process and function framework is created, flexible evaluation index system configuration capability is provided, and dynamic adjustment of an index system is supported; a data preprocessing capability is provided; evaluation rules are flexibly configured, and objectivity and specialty of evaluation results are guaranteed; template contents can be dynamically managed; professional quality improvement suggestions are given, so that the evaluation object can conveniently and continuously improve the data quality; all data operations are traced, so that the safety of evaluated data is effectively guaranteed while data asset quality evaluation is completed; effective evaluation of data asset quality is realized, and a scientific basis is provided for transaction, use and management of data assets.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of data asset quality assessment; in particular, to a data asset quality assessment method and system. Background Art

[0002] With the advent of the big data era, data assets have become one of the important assets of enterprises. At present, due to the uneven quality of data assets, the actual application effect and transaction value of data are seriously affected. Therefore, it is necessary to scientifically and reasonably evaluate the quality of data assets.

[0003] However, in the field of data asset quality assessment, the existing data asset quality assessment solutions have the following shortcomings:

[0004] 1. Lack of comprehensive and complete evaluation solutions, and inability to provide full-process data quality evaluation capabilities;

[0005] 2. Lack of a secure and controllable data quality assessment environment, and inability to effectively guarantee the security of confidential assessment data;

[0006] 3. Lack of flexible evaluation indicator configuration capabilities. Often only fixed evaluation indicators are provided, and evaluation indicators cannot be flexibly and dynamically configured according to scenarios and industries;

[0007] 4. Most quality assessment rules are based on preprocessed assessment data. However, there is a lack of professional and secure preprocessing capabilities and tools at this stage.

[0008] 5. The evaluation results lack objectivity and professionalism, are easily modified, and lose their authority;

[0009] 6. Lack of comprehensive and flexible assessment report template management capabilities, unable to provide comprehensive and integrated assessment reports;

[0010] 7. The assessment behavior lacks security audit capabilities and cannot provide detailed and reliable audit logs;

[0011] 8. The evaluation report only provides evaluation results and lacks effective suggestions for continuous improvement. Summary of the invention

[0012] In view of this, the purpose of the present invention is to propose a data asset quality assessment method and system, which can assess the quality of data assets more objectively, comprehensively and safely by designing a complete set of data asset quality assessment processes and functional frameworks. A new assessment user is created for each assessment activity, and an independent and secure data space is allocated. After the assessment activity is completed, the data space and the assessment user are recovered and destroyed to ensure the security of the assessment data; a flexible assessment indicator system configuration capability is provided, support is provided for dynamic adjustment of the indicator system, and data preprocessing capabilities are provided to achieve the processing of assessment data in an independent, controllable and secure data space. While ensuring data security, convenience is provided for data quality assessment, professional quality improvement suggestions are given, and it is convenient for the assessment object to continuously improve data quality, thereby achieving effective assessment of data asset quality, so as to meet the needs of corporate decision-making, data management and market transactions, promote the improvement of data asset quality, and promote data application and transactions.

[0013] The present invention provides a data asset quality assessment method, comprising the following steps:

[0014] Step 1, create an evaluation user, the attributes of which include user name, user Chinese name, and user login password; create an independent user data space for the evaluation user; and assign database operation permissions and evaluation function operation permissions to the evaluation user;

[0015] Preferably, the data space is a newly created database of the type Mysql, Oracle, SQL Server, etc., and the newly created database uses the user name of the evaluation user as the database name; and a database user is created for the newly created database with the user name of the evaluation user as the name and the obtained 10-digit random string as the password.

[0016] Step 2: Configure the evaluation data source for which data quality evaluation is required, including: data source IP address, port number, connection user, password information; collect metadata information of the evaluation data source;

[0017] Step 3. Complete the basic information of the evaluation data table; for the metadata information collected in Step 2, set whether to perform data asset quality evaluation;

[0018] Specifically, for the metadata information collected in Step 2, if the Chinese name of the table is empty, it is supplemented and improved by manual modification to ensure that all tables have clear Chinese names, so that the information of the data table is complete and accurate when the evaluation report is generated in Step 7;

[0019] When collecting metadata information in Step 2, it is set by default that all data tables need to undergo quality assessment. If data asset quality assessment is not required, the corresponding status of the data table can be changed to "Not assessed". For data tables set to "Not assessed", the corresponding status of the data table can also be changed to "Evaluated".

[0020] Step 4: Establish an evaluation indicator system and configure evaluation rules;

[0021] Step 5. Create a new evaluation task for executing the evaluation rules. The attributes of the evaluation task configuration include: task name and execution rules. Execute each evaluation rule configured in the evaluation task in turn to generate an evaluation result.

[0022] Specifically, the task name is the Chinese description of the task, and the execution rule is one, multiple or all evaluation rules selected from the evaluation rules configured in step Step4; one or more evaluation tasks can be configured to facilitate the execution of different quality evaluation rules in different task modes.

[0023] Step 6. Configure the document template in a standardized format for generating the evaluation report, and configure the basic information of the template generated into the evaluation report; configure improvement suggestions for all the second-order indicators in the evaluation indicator system;

[0024] The basic information of the template includes: the name of the entrusting party, the unified social credit code of the entrusting party, the name of the data asset, the name of the evaluation agency, the unified social credit code of the evaluation agency, the report number, the validity period of the quality evaluation report, the data asset agency profile, the data asset content profile, the data asset application scenario profile, etc.;

[0025] The basic information of the template is pre-configured and generated directly into the assessment report; if an assessment task generates assessment reports multiple times or the same assessment user has multiple assessment tasks, the configured basic information of the template can be used repeatedly to avoid the tedious manual filling each time the assessment report is generated.

[0026] Preferably, the content of the document template includes: report homepage, statement, basic information table of data asset quality assessment, macro background of data asset quality assessment, data asset organization introduction, data asset content introduction, data asset application scenario introduction, data asset quality assessment objectives, data asset quality assessment methods, data asset quality assessment process, data asset quality assessment results, analysis and suggestions;

[0027] The report homepage includes: the name of the client, the name of the asset appraisal unit, and the asset appraisal time. All information is dynamically generated;

[0028] The statement is a formal description of the data asset quality assessment;

[0029] The basic information table of data asset quality assessment includes the information of the data asset quality assessment client, the data asset quality assessment agency, the basis for data asset quality assessment, the data asset quality assessment results, and the data asset quality assessment report information; the information of the data asset quality assessment client includes: the name of the agency, the unified social credit code, which is dynamic content; the information of the data asset quality assessment agency includes: the name of the agency, the unified social credit code, which is dynamic content; the basis for data asset quality assessment includes the enumeration of various laws and regulations related to data quality; the data asset quality assessment results include the name of the data asset and the total score of this assessment, which is dynamic content; the data asset quality assessment report information includes the quality assessment report number and the validity period of the quality assessment report, which is dynamic content;

[0030] The macro background of data asset quality assessment is fixed content, which is the relevant background description of national laws, regulations, policy documents, etc. on data asset quality;

[0031] The data asset organization profile is dynamic content, which is the profile information of the data asset organization filled in by the evaluation user;

[0032] The data asset content brief is dynamic content, which is the data asset content brief information filled in by the evaluation user;

[0033] The data asset application scenario brief is a dynamic content, which is filled in by the evaluation user and corresponds to the application scenario brief of the evaluated data;

[0034] The data asset quality assessment objectives are fixed content, which explains why data asset quality assessment is necessary;

[0035] The data asset quality assessment method is to explain the quality assessment basis and the quality assessment indicator system, wherein the quality assessment basis is various national quality assessment laws and regulations, such as "Information Technology Data Quality Evaluation Indicators", etc., and the quality assessment indicator system is the assessment indicator system established in the dynamically acquired step Step 4;

[0036] The data asset quality assessment process includes data asset details and the assessment results of each first-order indicator, wherein the content of the data asset details includes the number of data sources connected to the data asset quality assessment, the number of tables, the number of fields, the total number of data items, the graphical display of the number of data items in the data table, and the data asset overview, wherein the data asset overview lists in detail the English name, Chinese name, and data volume of the data tables involved in the data asset quality assessment; each first-order indicator assessment result is a first-order indicator in the assessment indicator system configured in step 4, and the assessment result of each first-order indicator is displayed, including the name, weight, score, and score rate of all corresponding second-order indicators, and at the same time, which second-order indicators have a score rate above 90% and which second-order indicators have a score rate below 90%, and a radar chart displays the total score and score of all second-order indicators under this first-order indicator;

[0037] The data asset quality assessment results describe the calculation formula for the assessment results and the total score of this data asset quality assessment;

[0038] The analysis and suggestions are a paragraph describing the score analysis of this assessment and the improvement suggestions based on the scores. The score analysis includes: which first-order indicators have a higher score rate and which second-order indicators have room for improvement; wherein the score rate refers to the proportion of the actual score to the indicator weight, and the second-order indicators having room for improvement means that the second-order indicators affect the scores of the first-order indicators; the improvement suggestions are improvement suggestions for the second-order indicators with room for improvement, and the improvement suggestions come from the quality improvement suggestions for the corresponding assessment indicators configured in step Step6; quality improvement suggestions are given for each second-order indicator with room for improvement; the judgment rules for a higher score rate and room for improvement are uniformly configured thresholds, which default to 80%. If the score rate is ≥80%, it is considered as a higher score rate; if the score rate is lower than 80%, it is considered as having room for improvement.

[0039] The configuration improvement suggestions for all second-order indicators in the evaluation indicator system include: a textual description of matters that need to be paid attention to in order to continue to improve the score of the second-order indicator, such as the quality improvement suggestion for the second-order indicator "data standard" can be "establish and improve data standards, strictly implement data standards, strictly follow standard specifications for data table naming, and improve the standardization score of data table naming"; the quality improvement suggestions for all second-order indicators are dynamically configured and can be modified and improved according to industry and actual needs; when the score of a second-order indicator of the evaluation indicator is lower than a certain threshold, such as the score is lower than the weight of the second-order indicator multiplied by 80%, when the evaluation report is generated, the low score of the second-order indicator will be clearly displayed in the evaluation report, and a quality improvement suggestion for improving the score of the second-order indicator will be given.

[0040] Step 7, select the assessment task for which the assessment result is generated in step Step 5, and generate an assessment report according to the document template and template basic information of the assessment report configured in step Step 6; download the assessment report (preferably in pdf file format) as the final report document of this assessment;

[0041] Specifically, the basic information is replaced into the corresponding position of the template basic information, and the dynamically calculated information is calculated and replaced into the corresponding position of the document template; the evaluation report after replacing the basic information and the dynamically calculated information is displayed at the same time, and the replaced basic information content parts are marked with colors. The evaluation user can preview the entire evaluation report, and at the same time display the editing input areas of all basic information and the basic information content configured in the corresponding step Step 6. The evaluation user can continue to modify the basic information content as needed and directly take effect in the evaluation report.

[0042] Step 8. During the data quality assessment process, record all assessment log information related to the access and operation of the assessment data to ensure that all assessment data operation behaviors are log-able; export all recorded assessment log information for security auditing.

[0043] The log information includes: operator, operation time, access IP, access address, operation function, database connection information, executed SQL statement, and number of execution results;

[0044] Step 9. Reclaim the evaluation user rights; reclaim and destroy the user data space created in Step 1, delete the corresponding database user and database, and ensure that the data is cleared; delete the evaluation user and all data created in Step 1.

[0045] Exemplarily, the method of reclaiming the evaluation user rights includes: the evaluation user actively applies to reclaim the evaluation user rights assigned in step 1, or the system administrator passively reclaims the evaluation user rights assigned in step 1; the system administrator is the system operation account with the highest authority to manage the evaluation system, and manages the system through this account. All data related to the evaluation of the evaluation user is completely cleared, and the evaluation user cannot conduct the evaluation again. The security of the evaluation data is guaranteed while the evaluation report is obtained after the evaluation is completed.

[0046] Further, the method for establishing an evaluation index system in Step 4 includes: constructing an index set for quality evaluation, wherein the index set includes multiple first-order indicators, each first-order indicator includes multiple second-order indicators, and each second-order indicator includes multiple third-order indicators; wherein the attributes of the first-order indicators include: indicator code, indicator name, and indicator weight; the attributes of the second-order indicators include: indicator code, indicator name, and indicator weight; the attributes of the third-order indicators include: indicator code, indicator name, indicator weight, third-order indicator detection description, A SQL template for indicator calculation, and BSQL template for indicator calculation;

[0047] The indicator code is a unique ID value defined for the indicator, and different indicators are distinguished by the unique ID value; the indicator name is the name of a parameter or standard used to measure or describe a specific goal or phenomenon, such as: table naming standardization; the indicator weight is the proportion of the indicator in the total score, and the proportion is a score value between 0 and 100; the third-order indicator detection description is a detailed explanatory text describing the definition, calculation method, purpose of use and related precautions of the third-order indicator; the indicator calculation in the A SQL template of the indicator calculation means that each third-order indicator will eventually calculate a score, which is: indicator weight * (A / B), where A and B are both data values ​​calculated by SQL statements, the A data value is the number of data tables that do not meet the table naming specifications, the B data value is the total number of data tables participating in the quality inspection, and A / B obtains a percentage; the A SQL template refers to a general SQL statement for calculating the A data value provided for each third-order indicator, and the BSQL template refers to a general SQL statement for calculating the B data value provided for each third-order indicator; the A SQL template and the BSQL template are used to configure evaluation rules.

[0048] The indicators in the indicator set meet the following rules: the weight of each first-order indicator is equal to the sum of the weights of all second-order indicators contained in the first-order indicator, and the weight of each second-order indicator is equal to the sum of the weights of all third-order indicators contained in the second-order indicator. The evaluation indicator system adopts a percentage system, and the sum of the weights of all first-order indicators is equal to 100.

[0049] For example, the evaluation indicator system is provided with a global default configuration by the system and adopted by each evaluation user. The evaluation indicator system can be centrally managed by the system administrator, who can modify, delete, and add any indicator. When modifying, deleting, and adding an indicator causes a change in the indicator weight, the indicator weight can be adjusted or manually adjusted so that the indicator weight meets the above rules;

[0050] When adjusting indicator weights, the third-order indicators equally divide the weights of the corresponding second-order indicators; the corresponding weights of the second-order and first-order indicators can only be modified manually and must meet the above rules; an industry data assessment indicator system can be constructed based on the industry data characteristics of the industries involved in the data quality assessment to meet the actual needs of data asset quality assessment in different industries.

[0051] Furthermore, when establishing the evaluation index system in Step 4, if the A data value or the B data value cannot be directly calculated by SQL statements, the evaluation data is preprocessed, and the data involved in the calculation is merged, split, and extracted, so that the A data value or the B data value in the index can be quickly calculated using the SQL template when calculating the index;

[0052] The data preprocessing can only extract evaluation data from the evaluation data source configured in step Step2 to the user data space created in step Step1, and complete the merging, splitting, and extraction processing of one or more data tables in the user data space. The evaluation user can only read data from the evaluation data source configured in step Step2, and cannot add, modify, or delete data from the evaluation data source, nor can the data in the user data space created in Step1 be written back to the evaluation data source, thereby ensuring the security of the evaluation data source.

[0053] Specifically, data preprocessing allows one or more evaluation data tables to be preprocessed to generate new transition data tables; and allows unlimited data preprocessing and generation of unlimited transition data tables in the user data space.

[0054] Furthermore, the method for configuring the evaluation rules in step 4 includes:

[0055] Configure the inspection rules for data asset quality, the attributes of the inspection rules include: rule code, rule name, evaluation index, database, data table, A inspection SQL statement, B inspection SQL statement, wherein the rule code is a generated unique string code, the rule name is a Chinese description of the inspection content of the inspection rule, the evaluation index is a third-order index selected from the evaluation index system, the database is the evaluation data source configured in step Step2, or if the evaluation data is pre-processed by step Step4, the database is the user data space created in step Step1; the data table refers to the data table involved in the A inspection SQL statement and the B inspection SQL statement, which may be a data table collected from the evaluation data source, or a transitional data table generated after the evaluation data is pre-processed in step Step4. If the selected evaluation indicator is to perform quality assessment on all data tables without limiting the specific data table, the data table may not be selected; the A inspection SQL statement is A calculated based on the indicator recommended by the selected evaluation indicator. The corresponding SQL statement in the SQL template is modified appropriately based on the SQL statement to be a SQL statement that is more in line with the current inspection scenario as needed; the B inspection SQL statement is the corresponding SQL statement in the B SQL template calculated according to the indicators recommended by the selected evaluation indicators, and is modified appropriately based on the SQL statement to be a SQL statement that is more in line with the current inspection scenario as needed; the configuration evaluation rules are configured as multiple rules, covering all evaluation data tables, and covering all third-order indicators in the evaluation indicator system.

[0056] Furthermore, the method of collecting metadata information of the evaluation data source in step 2 includes:

[0057] Collect information on all tables and all fields of the evaluation data source. The table information includes: English name of the table, Chinese name of the table, number of data entries in the table (and other information). The field information includes: English name of the field, Chinese name of the field, field type, length, precision, whether to allow null, default value (field definition information, etc.); use the collected metadata information as the evaluation object of data quality evaluation (i.e., data table).

[0058] Furthermore, the method of sequentially executing each evaluation rule configured in the evaluation task in step 5 includes:

[0059] When executing each evaluation rule, connect the process to the database selected by the evaluation rule, execute the A check SQL statement and the B check SQL statement respectively, and divide the number of execution results of the A check SQL statement by the number of execution results of the B check SQL statement to obtain a percentage value with two decimal places after the decimal point, and multiply the percentage value by the weight of the evaluation indicator corresponding to the evaluation rule to obtain a score with two decimal places after the decimal point. The score is the evaluation score of the third-order indicator. After all the evaluation rules of the evaluation task are executed, the final evaluation result of the evaluation task is generated;

[0060] The final evaluation result is based on the evaluation index system to generate scores for all indicators in the evaluation index system. The scores of all third-order indicators in the evaluation index system are the scores of the evaluation indicators of the evaluation rules selected in the evaluation task; the sum of all third-order indicator scores forms the score of the second-order indicator corresponding to the third-order indicator, the sum of all second-order indicator scores forms the score of the first-order indicator corresponding to the second-order indicator, and the sum of all first-order indicator scores forms the final total score of this evaluation task.

[0061] If a third-order indicator in the evaluation indicator system is not included in all the evaluation rules selected in the evaluation task, then the third-order indicator is not included in the evaluation indicator of this evaluation task, and the score of the third-order indicator defaults to the full score, that is, the weight.

[0062] Furthermore, the method for assigning database operation permissions in step 1 includes: granting the operation permissions of the newly created database only to a set evaluation user, so that only the set evaluation user has the permission to access the newly created database, and all other evaluation users cannot know and cannot access the newly created database;

[0063] The evaluation function operation authority is the access and operation authority for the evaluation functions in step 2 to step 9. Only the evaluation users who are granted the authority can access and operate the evaluation functions in step 2 to step 9.

[0064] The present invention also provides a data asset quality assessment system, which executes the data asset quality assessment method as described above, including:

[0065] Create an evaluation user module: used to create an evaluation user, the attributes of which include user name, user Chinese name, and user login password; create an independent user data space for the evaluation user; and assign database operation permissions and evaluation function operation permissions to the evaluation user;

[0066] Module for configuring data source and collecting metadata: used to configure the evaluation data source for which data quality evaluation is required, the evaluation data source includes: data source IP address, port number, connection user, password information; collect metadata information of the evaluation data source;

[0067] Data table management module: used to complete the basic information of the evaluation data table; set whether to perform data asset quality evaluation for the metadata information collected in Step 2;

[0068] Configuration evaluation rule module: used to establish the evaluation indicator system and configure evaluation rules;

[0069] Configuration evaluation task module: used to create an evaluation task for executing evaluation rules, the attributes of the evaluation task configuration include: task name, execution rules; execute each evaluation rule configured in the evaluation task in sequence to generate evaluation results;

[0070] Configure the evaluation report template module: used to configure the document template in a standardized format for generating evaluation reports, configure the basic information of the template generated into the evaluation report; configure improvement suggestions for all second-order indicators in the evaluation indicator system;

[0071] Generate evaluation report module: used to select the evaluation task for which the evaluation result is generated in step 5, and generate an evaluation report according to the document template and template basic information of the evaluation report configured in step 6; download the evaluation report as the final report document of this evaluation;

[0072] Export evaluation log module: used to record all evaluation log information related to the access and operation of evaluation data during the data quality evaluation process, to ensure that all evaluation data operation behaviors have logs available for inspection; export all recorded evaluation log information for security auditing;

[0073] Module for reclaiming evaluation user rights and destroying data space: used to reclaim evaluation user rights; reclaim and destroy the user data space created in step 1, delete the corresponding database user and database, and delete the evaluation user and all data created in step 1.

[0074] The present invention also provides a computer-readable storage medium having a computer program stored thereon, and when the program is executed by a processor, the steps of the data asset quality assessment method as described above are implemented.

[0075] The present invention also provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the data asset quality assessment method as described above when executing the program.

[0076] Compared with the prior art, the present invention has the following beneficial effects:

[0077] The data asset quality assessment method and system provided by the present invention create a complete set of data asset quality assessment processes and functional frameworks. A new assessment user is created for each assessment activity, and an independent and secure data space is allocated. After the assessment activity is completed, the data space and the assessment user are recovered and destroyed, thereby ensuring the security of the assessment data; a flexible assessment indicator system configuration capability is provided, and the dynamic adjustment of the indicator system is supported, which can adapt to various application scenarios and industries; data preprocessing capabilities are provided to realize the processing of assessment data in an independent, controllable and secure data space, providing convenience for data quality assessment while ensuring data security; based on the assessment indicator system, assessment rules are flexibly configured, and the assessment indicator scores are strictly calculated by the assessment rules, ensuring the objectivity and professionalism of the assessment results; a flexible, comprehensive and content-rich assessment report template is provided, and the template content can be dynamically managed; while generating an assessment report and giving the overall assessment results, professional quality improvement suggestions are given, so that the assessment object can continuously improve the data quality; all data operations leave traces, which effectively ensure the security of the assessed data while ensuring the completion of the data asset quality assessment; through the complete and comprehensive assessment process provided by the present invention, effective assessment of data asset quality can be achieved, providing a scientific basis for the transaction, use and management of data assets. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the following detailed description of the preferred embodiment.The drawings are only for the purpose of illustrating the preferred embodiments and are not to be construed as limiting the invention.

[0079] In the attached picture:

[0080] Figure 1 A basic process architecture diagram of data asset quality assessment according to an embodiment of the present invention;

[0081] Figure 2 A flow chart of a data asset quality assessment method of the present invention;

[0082] Figure 3 The figure is a schematic diagram of the structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0083] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present disclosure. Instead, they are merely examples of systems and products consistent with some aspects of the present disclosure as detailed in the appended claims.

[0084] The terms used in this disclosure are for the purpose of describing specific embodiments only and are not intended to limit the disclosure. The singular forms of "a", "said" and "the" used in this disclosure and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0085] It should be understood that although the terms first, second, third, etc. may be used in the present disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present disclosure, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0086] The embodiments of the present invention are described in further detail below.

[0087] The present invention provides a data asset quality assessment method. Figure 2 As shown, the following steps are included:

[0088] Step 1, create an evaluation user, the attributes of which include user name, user Chinese name, and user login password; create an independent user data space for the evaluation user; and assign database operation permissions and evaluation function operation permissions to the evaluation user;

[0089] The data space is a newly created database of the Mysql type, and the newly created database uses the user name of the evaluation user as the database name; and a database user is created for the newly created database with the user name of the evaluation user as the name and the obtained 10-digit random string as the password.

[0090] The method for assigning database operation permissions includes: granting the operation permissions of the newly created database only to a set evaluation user, so that only the set evaluation user has the permission to access the newly created database, and all other evaluation users cannot know and cannot access the newly created database;

[0091] The evaluation function operation authority is the access and operation authority for the evaluation functions in Step 2 to Step 9. Only the evaluation users who are granted the authority can access and operate the evaluation functions in Step 2 to Step 9.

[0092] Step 2: Configure the evaluation data source for which data quality evaluation is required, including: data source IP address, port number, connection user, password information; collect metadata information of the evaluation data source;

[0093] Collect information on all tables and all fields of the evaluation data source. The table information includes: English name of the table, Chinese name of the table, number of data entries in the table (and other information). The field information includes: English name of the field, Chinese name of the field, field type, length, precision, whether to allow null, default value (field definition information, etc.); use the collected metadata information as the evaluation object of data quality evaluation (i.e., data table).

[0094] Step 3. Complete the basic information of the evaluation data table; for the metadata information collected in Step 2, set whether to perform data asset quality evaluation;

[0095] For the metadata information collected in Step 2, if the Chinese name of the table is empty, it is supplemented and improved by manual modification, so that the information of the data table is complete and accurate when the evaluation report is generated in Step 7;

[0096] When collecting metadata information in Step 2, it is set by default that all data tables need to undergo quality assessment; for data assets that do not require quality assessment, change the corresponding status of the data table to "Not assessed".

[0097] Step 4: Establish an evaluation indicator system and configure evaluation rules;

[0098] Construct an indicator set for quality assessment, wherein the indicator set includes multiple first-order indicators, each first-order indicator includes multiple second-order indicators, and each second-order indicator includes multiple third-order indicators; wherein the attributes of the first-order indicator include: indicator code, indicator name, indicator weight; the attributes of the second-order indicator include: indicator code, indicator name, indicator weight; the attributes of the third-order indicator include: indicator code, indicator name, indicator weight, third-order indicator detection description, ASQL template for indicator calculation, and B SQL template for indicator calculation;

[0099] The indicator code is a unique ID value defined for the indicator, and different indicators are distinguished by this unique ID value; the indicator name is the name of the parameter or standard used to measure or describe a specific goal or phenomenon; the indicator weight is the proportion of the indicator in the total score, which is a score value between 0 and 100; the third-order indicator detection description is a detailed explanatory text describing the definition, calculation method, purpose of use and related precautions of the third-order indicator; the indicator calculation in the A SQL template of the indicator calculation means that each third-order indicator will eventually calculate a score, which is: indicator weight * (A / B), where A and B are both data values ​​calculated by SQL statements, the A data value is the number of data tables that do not meet the table naming specifications, and the B data value is the total number of data tables involved in the quality inspection, and A / B obtains a percentage; the A SQL template refers to the general SQL statement for calculating the A data value provided for each third-order indicator, and the B SQL template refers to the general SQL statement for calculating the B data value provided for each third-order indicator; the A SQL template and the B SQL template are used to configure the evaluation rules.

[0100] The indicators in the indicator set meet the following rules: the weight of each first-order indicator is equal to the sum of the weights of all second-order indicators contained in the first-order indicator, and the weight of each second-order indicator is equal to the sum of the weights of all third-order indicators contained in the second-order indicator. The evaluation indicator system adopts a percentage system, and the sum of the weights of all first-order indicators is equal to 100.

[0101] When adjusting indicator weights, the third-order indicators equally divide the weights of the corresponding second-order indicators; the corresponding weights of the second-order and first-order indicators can only be modified manually and must meet the above rules; an industry data assessment indicator system can be constructed based on the industry data characteristics of the industries involved in the data quality assessment to meet the actual needs of data asset quality assessment in different industries.

[0102] When establishing an evaluation index system, if the A data value or the B data value cannot be directly calculated through SQL statements, the data involved in the calculation is merged, split, and extracted to facilitate the use of SQL templates to quickly calculate the A data value or the B data value in the indicator when calculating the indicator;

[0103] Data preprocessing can only extract evaluation data from the evaluation data source configured in step Step2 to the user data space created in step Step1, and complete the merging, splitting, and extraction of one or more data tables in the user data space. Evaluation users can only read data from the evaluation data source configured in step Step2, and cannot add, modify, or delete data from the evaluation data source, nor can they write data in the user data space created in Step1 back to the evaluation data source, thus ensuring the security of the evaluation data source.

[0104] The inspection rules for data asset quality are configured. The attributes of the inspection rules include: rule code, rule name, evaluation index, database, data table, A inspection SQL statement, B inspection SQL statement, wherein the rule code is a generated unique string code, the rule name is a Chinese description of the inspection content of the inspection rule, the evaluation index is a third-order index selected from the evaluation index system, the database is the evaluation data source configured in step Step2 or if the evaluation data is pre-processed by step Step4, the database is the user data space created in step Step1; the data table refers to the data table involved in the A inspection SQL statement and the B inspection SQL statement; the A inspection SQL statement is the corresponding SQL statement in the A SQL template calculated according to the indicator recommended by the selected evaluation index, and is appropriately modified to a SQL statement that is more in line with the current inspection scenario based on the SQL statement; the B inspection SQL statement is the corresponding SQL statement in the B SQL template calculated according to the indicator recommended by the selected evaluation index, and is appropriately modified to a SQL statement that is more in line with the current inspection scenario based on the SQL statement; the configuration evaluation rules are configured as multiple items, covering all evaluation data tables and all third-order indicators in the evaluation index system.

[0105] Step 5. Create a new evaluation task for executing the evaluation rules. The attributes of the evaluation task configuration include: task name and execution rules. Execute each evaluation rule configured in the evaluation task in turn to generate an evaluation result.

[0106] The task name is the Chinese description of the task, and the execution rules are all the evaluation rules selected from the evaluation rules configured in step Step4; multiple evaluation tasks are configured to facilitate the execution of different quality evaluation rules in different task modes.

[0107] When executing each evaluation rule, connect the process to the database selected by the evaluation rule, execute the A check SQL statement and the B check SQL statement respectively, and divide the number of execution results of the A check SQL statement by the number of execution results of the B check SQL statement to obtain a percentage value with two decimal places after the decimal point, and multiply the percentage value by the weight of the evaluation indicator corresponding to the evaluation rule to obtain a score with two decimal places after the decimal point. The score is the evaluation score of the third-order indicator. After all the evaluation rules of the evaluation task are executed, the final evaluation result of the evaluation task is generated;

[0108] The scores of all third-order indicators in the evaluation index system are the scores of the evaluation indicators of the evaluation rules selected in the evaluation task; the sum of all third-order indicator scores forms the score of the second-order indicator corresponding to the third-order indicator, the sum of all second-order indicator scores forms the score of the first-order indicator corresponding to the second-order indicator, and the sum of all first-order indicator scores forms the final total score of this evaluation task.

[0109] If a third-order indicator in the evaluation indicator system is not included in all the evaluation rules selected in the evaluation task, the third-order indicator is not included in the evaluation indicator of this evaluation task, and the score of the third-order indicator defaults to the full score, that is, the weight.

[0110] Step 6. Configure the document template in a standardized format for generating the evaluation report, and configure the basic information of the template generated into the evaluation report; configure improvement suggestions for all the second-order indicators in the evaluation indicator system;

[0111] The basic information of the template includes: the name of the client's organization, the client's unified social credit code, the name of the data asset, the name of the assessment agency, the unified social credit code of the assessment agency, the report number, the validity period of the quality assessment report, an introduction to the data asset organization, an introduction to the data asset content, an introduction to the data asset application scenarios, etc.; the basic information of the template is pre-configured and directly generated into the assessment report; for an assessment task that generates assessment reports multiple times or the same assessment user has multiple assessment tasks, the configured basic information of the template is repeatedly used to avoid the tedious manual filling in each time an assessment report is generated.

[0112] The content of the document template includes: report homepage, statement, basic information table of data asset quality assessment, macro background of data asset quality assessment, data asset organization introduction, data asset content introduction, data asset application scenario introduction, data asset quality assessment objectives, data asset quality assessment methods, data asset quality assessment process, data asset quality assessment results, analysis and suggestions;

[0113] Step 7, select the assessment task for which the assessment result is generated in step Step 5, and generate an assessment report according to the document template and template basic information of the assessment report configured in step Step 6; download the assessment report as the final report document of this assessment;

[0114] The basic information is replaced into the corresponding position of the template basic information, and the dynamically calculated information is calculated and replaced into the corresponding position of the document template; the evaluation report after the basic information and the dynamically calculated information are replaced is displayed at the same time, and the replaced basic information content parts are marked with colors. The evaluation user can preview the entire evaluation report, and the editing input area of ​​all basic information and the basic information content configured in the corresponding step Step 6 are displayed at the same time. The evaluation user can continue to modify the basic information content as needed and directly take effect in the evaluation report. In this embodiment, the evaluation report is downloaded in pdf file format.

[0115] Step 8. During the data quality assessment process, record all assessment log information related to the access and operation of the assessment data to ensure that all assessment data operation behaviors are log-able; export all recorded assessment log information for security auditing.

[0116] The recorded log information includes: operator, operation time, access IP, access address, operation function, database connection information, executed SQL statement, and number of execution results;

[0117] Step 9. Reclaim the evaluation user rights; reclaim and destroy the user data space created in Step 1, delete the corresponding database user and database; delete the evaluation user and all data created in Step 1.

[0118] All data related to this assessment by the assessment user has been completely cleared, and the assessment user cannot conduct the assessment again. The security of the assessment data is guaranteed while the assessment report is obtained after the assessment is completed.

[0119] Figure 1 The basic process architecture of data asset quality assessment in this embodiment is shown.

[0120] The embodiment of the present invention further provides a data asset quality assessment system, which executes the data asset quality assessment method as described above, including:

[0121] Create an evaluation user module: used to create an evaluation user, the attributes of which include user name, user Chinese name, and user login password; create an independent user data space for the evaluation user; and assign database operation permissions and evaluation function operation permissions to the evaluation user;

[0122] Module for configuring data source and collecting metadata: used to configure the evaluation data source for which data quality evaluation is required, the evaluation data source includes: data source IP address, port number, connection user, password information; collect metadata information of the evaluation data source;

[0123] Data table management module: used to complete the basic information of the evaluation data table; set whether to perform data asset quality evaluation for the metadata information collected in Step 2;

[0124] Configuration evaluation rule module: used to establish the evaluation indicator system and configure evaluation rules;

[0125] Configuration evaluation task module: used to create an evaluation task for executing evaluation rules, the attributes of the evaluation task configuration include: task name, execution rules; execute each evaluation rule configured in the evaluation task in sequence to generate evaluation results;

[0126] Configure the evaluation report template module: used to configure the document template in a standardized format for generating evaluation reports, configure the basic information of the template generated into the evaluation report; configure improvement suggestions for all second-order indicators in the evaluation indicator system;

[0127] Generate evaluation report module: used to select the evaluation task for which the evaluation result is generated in step 5, and generate an evaluation report according to the document template and template basic information of the evaluation report configured in step 6; download the evaluation report as the final report document of this evaluation;

[0128] Export evaluation log module: used to record all evaluation log information related to the access and operation of evaluation data during the data quality evaluation process, to ensure that all evaluation data operation behaviors have logs available for inspection; export all recorded evaluation log information for security auditing;

[0129] Module for reclaiming evaluation user rights and destroying data space: used to reclaim evaluation user rights; reclaim and destroy the user data space created in step 1, delete the corresponding database user and database, and delete the evaluation user and all data created in step 1.

[0130] The embodiment of the present invention creates a complete data asset quality assessment process and functional framework; each assessment activity creates a new assessment user and allocates an independent and secure data space; after the assessment activity is completed, the data space and assessment users are recovered and destroyed to ensure the security of the assessment data; it has a flexible assessment indicator system configuration capability, supports dynamic adjustment of the indicator system, and adapts to various application scenarios and industries; provides data preprocessing capabilities to achieve data processing in an independent, controllable, and secure data space, providing convenience for data quality assessment while ensuring data security; based on the assessment indicator system, flexible configuration of assessment rules, assessment indicator scores are strictly calculated by the assessment rules, ensuring the objectivity and professionalism of the assessment results; provides a flexible, comprehensive, and content-rich assessment report template, and can dynamically manage the template content; generates a report giving the overall assessment results and at the same time gives professional quality improvement suggestions, so that the assessment object can continuously improve data quality; all data operations leave traces, effectively ensuring the security of the assessed data while completing the data asset quality assessment.

[0131] An embodiment of the present invention further provides a computer device, Figure 3 is a schematic diagram of the structure of a computer device provided by an embodiment of the present invention; see the accompanying drawings Figure 3 As shown, the computer device includes: an input system 23, an output system 24, a memory 22 and a processor 21; the memory 22 is used to store one or more programs; when the one or more programs are executed by the one or more processors 21, the one or more processors 21 implement the data asset quality assessment method provided in the above embodiment; wherein the input system 23, the output system 24, the memory 22 and the processor 21 can be connected by a bus or other means, Figure 3 The example of connecting through bus is taken in the following.

[0132] The memory 22 is a readable and writable storage medium of a computing device, which can be used to store software programs, computer executable programs, such as program instructions corresponding to the data asset quality assessment method described in the embodiment of the present invention; the memory 22 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system and at least one application required for a function; the data storage area can store data created according to the use of the device, etc.; in addition, the memory 22 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device; in some instances, the memory 22 can further include a memory remotely arranged relative to the processor 21, and these remote memories can be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0133] The input system 23 may be used to receive input digital or character information, and to generate key signal input related to user settings and function control of the device; the output system 24 may include display devices such as display screens.

[0134] The processor 21 executes various functional applications and data processing of the device by running the software programs, instructions and modules stored in the memory 22, that is, implements the above-mentioned data asset quality assessment method.

[0135] The computer device provided above can be used to execute the data asset quality assessment method provided in the above embodiment, and has corresponding functions and beneficial effects.

[0136] The embodiment of the present invention also provides a storage medium containing computer executable instructions, which are used to execute the data asset quality assessment method provided in the above embodiment when executed by a computer processor. The storage medium is any of various types of memory devices or storage devices, and the storage medium includes: installation media, such as CD-ROM, floppy disk or tape system; computer system memory or random access memory, such as DRAM, DDR RAM, SRAM, EDORAM, Rambus RAM, etc.; non-volatile memory, such as flash memory, magnetic media (such as hard disk or optical storage); registers or other similar types of memory elements, etc.; the storage medium may also include other types of memory or a combination thereof; in addition, the storage medium may be located in a first computer system in which the program is executed, or may be located in a different second computer system, which is connected to the first computer system via a network (such as the Internet); the second computer system may provide program instructions to the first computer for execution. The storage medium includes two or more storage media that may reside in different locations (for example, in different computer systems connected via a network). The storage medium may store program instructions (for example, specifically implemented as a computer program) that can be executed by one or more processors.

[0137] Of course, the storage medium containing computer executable instructions provided in an embodiment of the present invention is not limited to the data asset quality assessment method described in the above embodiment, and can also execute related operations in the data asset quality assessment method provided in any embodiment of the present invention.

[0138] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

[0139] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A data asset quality assessment method, characterized in that: The following steps are involved: Step 1, create an evaluation user, the attributes of which include user name, user Chinese name, and user login password; create an independent user data space for the evaluation user; and assign database operation permissions and evaluation function operation permissions to the evaluation user; Step 2: Configure the evaluation data source for which data quality evaluation is required, including: data source IP address, port number, connection user, password information; collect metadata information of the evaluation data source; Step 3. Complete the basic information of the evaluation data table; for the metadata information collected in Step 2, set whether to perform data asset quality evaluation; Step 4: Establish an evaluation indicator system and configure evaluation rules; Step 5. Create a new evaluation task for executing the evaluation rules. The attributes of the evaluation task configuration include: task name and execution rules. Execute each evaluation rule configured in the evaluation task in turn to generate an evaluation result. Step 6. Configure the document template in a standardized format for generating the evaluation report, and configure the basic information of the template generated into the evaluation report; configure improvement suggestions for all the second-order indicators in the evaluation indicator system; Step 7, select the assessment task for which the assessment result is generated in step Step 5, and generate an assessment report according to the document template and template basic information of the assessment report configured in step Step 6; download the assessment report as the final report document of this assessment; Step 8. During the data quality assessment process, record all assessment log information related to the access and operation of the assessment data to ensure that all assessment data operation behaviors are log-able; export all recorded assessment log information for security auditing; The log information includes: operator, operation time, access IP, access address, operation function, database connection information, executed SQL statement, and number of execution results; Step 9. Reclaim the evaluation user rights; reclaim and destroy the user data space created in Step 1, delete the corresponding database user and database, and delete the evaluation user and all data created in Step 1.

2. The data asset quality assessment method according to claim 1, characterized in that: The method for establishing an evaluation index system in Step 4 includes: constructing an index set for quality evaluation, wherein the index set includes multiple first-order indicators, each first-order indicator includes multiple second-order indicators, and each second-order indicator includes multiple third-order indicators; wherein the attributes of the first-order indicators include: indicator code, indicator name, and indicator weight; the attributes of the second-order indicators include: indicator code, indicator name, and indicator weight; the attributes of the third-order indicators include: indicator code, indicator name, indicator weight, third-order indicator detection description, A SQL template for indicator calculation, and B SQL template for indicator calculation; The indicator code is a unique ID value defined for the indicator, and different indicators are distinguished by the unique ID value; the indicator name is the name of the parameter or standard used to measure or describe a specific goal or phenomenon; the indicator weight is the proportion of the indicator in the total score, and the proportion is a score value between 0 and 100; the third-order indicator detection description is a detailed explanatory text describing the definition, calculation method, purpose of use and related precautions of the third-order indicator; the indicator calculation in the ASQL template of the indicator calculation means that each third-order indicator will eventually calculate a score, which is: indicator weight * (A / B), where A and B are both data values ​​calculated by SQL statements, the A data value is the number of data tables that do not meet the table naming specifications, the B data value is the total number of data tables involved in the quality inspection, and A / B obtains a percentage; the A SQL template refers to a general SQL statement for calculating the A data value provided for each third-order indicator, and the B SQL template refers to a general SQL statement for calculating the B data value provided for each third-order indicator; The indicators in the indicator set meet the following rules: the weight of each first-order indicator is equal to the sum of the weights of all second-order indicators contained in the first-order indicator, and the weight of each second-order indicator is equal to the sum of the weights of all third-order indicators contained in the second-order indicator. The evaluation indicator system adopts a percentage system, and the sum of the weights of all first-order indicators is equal to 100.

3. The data asset quality assessment method according to claim 2, characterized in that: When establishing the evaluation index system in Step 4, if the A data value or the B data value cannot be directly calculated by SQL statements, the evaluation data is preprocessed, and the data involved in the calculation is merged, split, and extracted, so that the A data value or the B data value in the index can be quickly calculated using the SQL template when calculating the index; The data preprocessing can only extract evaluation data from the evaluation data source configured in step Step2 to the user data space created in step Step1, and complete the merging, splitting, and extraction processing of one or more data tables in the user data space. The evaluation user can only read data from the evaluation data source configured in step Step2, and cannot add, modify or delete data from the evaluation data source, nor can he write the data in the user data space created in Step1 back to the evaluation data source.

4. The data asset quality assessment method according to claim 3 is characterized in that: The method for configuring the evaluation rules in Step 4 includes: Configure the inspection rules for data asset quality, the attributes of the inspection rules include: rule code, rule name, evaluation index, database, data table, A inspection SQL statement, B inspection SQL statement, wherein the rule code is a generated unique string code, the rule name is a Chinese description of the inspection content of the inspection rule, the evaluation index is a third-order index selected from the evaluation index system, the database is the evaluation data source configured in step Step2 or if the evaluation data is pre-processed by step Step4, the database is the user data space created in step Step1; the data table refers to the data table involved in the A inspection SQL statement and the B inspection SQL statement; the A inspection SQL statement is the corresponding SQL statement in the A SQL template calculated according to the indicator recommended by the selected evaluation indicator, and based on the SQL statement, it is appropriately modified to a SQL statement that better suits the current inspection scenario as needed; the B inspection SQL statement is the B inspection SQL statement calculated according to the indicator recommended by the selected evaluation indicator. The corresponding SQL statement in the SQL template is modified appropriately based on the SQL statement to be a SQL statement that is more suitable for the current inspection scenario as needed; the configuration evaluation rules are configured as multiple rules, covering all evaluation data tables and all third-order indicators in the evaluation indicator system.

5. The data asset quality assessment method according to claim 1, characterized in that: The method for collecting metadata information of the evaluation data source in step 2 includes: Collect information on all tables and all field information of the evaluation data source. The table information includes: the English name of the table, the Chinese name of the table, and the number of data entries in the table. The field information includes: the English name of the field, the Chinese name of the field, the field type, length, precision, whether to allow null, and the default value. Use the collected metadata information as the evaluation object of data quality assessment.

6. The data asset quality assessment method according to claim 2, characterized in that: The method of sequentially executing each evaluation rule configured in the evaluation task in Step 5 includes: When executing each evaluation rule, connect the process to the database selected by the evaluation rule, execute the A check SQL statement and the B check SQL statement respectively, and divide the number of execution results of the A check SQL statement by the number of execution results of the B check SQL statement to obtain a percentage value with two decimal places after the decimal point, and multiply the percentage value by the weight of the evaluation indicator corresponding to the evaluation rule to obtain a score with two decimal places after the decimal point. The score is the evaluation score of the third-order indicator. After all the evaluation rules of the evaluation task are executed, the final evaluation result of the evaluation task is generated; The final evaluation result is based on the evaluation index system to generate scores for all indicators in the evaluation index system. The scores of all third-order indicators in the evaluation index system are the scores of the evaluation indicators of the evaluation rules selected in the evaluation task; the sum of all third-order indicator scores forms the score of the second-order indicator corresponding to the third-order indicator, the sum of all second-order indicator scores forms the score of the first-order indicator corresponding to the second-order indicator, and the sum of all first-order indicator scores forms the final total score of this evaluation task.

7. The data asset quality assessment method according to claim 1, characterized in that: The method for assigning database operation permissions in step 1 includes: granting the operation permissions of the newly created database only to a set evaluation user, so that only the set evaluation user has the permission to access the newly created database, and all other evaluation users cannot know and cannot access the newly created database; The evaluation function operation authority is the access and operation authority for the evaluation functions in Step 2 to Step 9. Only the evaluation users who are granted the authority can access and operate the evaluation functions in Step 2 to Step 9.

8. A data asset quality assessment system, executing the data asset quality assessment method according to any one of claims 1 to 7, characterized in that: include: Create evaluation user module: used to create evaluation users, the attributes of the evaluation users include user name, user Chinese name, user login password; Create an independent user data space for evaluation users; assign database operation permissions and evaluation function operation permissions to evaluation users; Module for configuring data source and collecting metadata: used to configure the evaluation data source for which data quality evaluation is required, the evaluation data source includes: data source IP address, port number, connection user, password information; collect metadata information of the evaluation data source; Data table management module: used to complete the basic information of the evaluation data table; set whether to perform data asset quality evaluation for the metadata information collected in Step 2; Configuration evaluation rule module: used to establish the evaluation indicator system and configure evaluation rules; Configuration evaluation task module: used to create an evaluation task for executing evaluation rules, the attributes of the evaluation task configuration include: task name, execution rules; execute each evaluation rule configured in the evaluation task in sequence to generate evaluation results; Configure the evaluation report template module: used to configure the document template in a standardized format for generating evaluation reports, configure the basic information of the template generated into the evaluation report; configure improvement suggestions for all second-order indicators in the evaluation indicator system; Generate evaluation report module: used to select the evaluation task for which the evaluation result is generated in step 5, and generate an evaluation report according to the document template and template basic information of the evaluation report configured in step 6; download the evaluation report as the final report document of this evaluation; Export evaluation log module: used to record all evaluation log information related to the access and operation of evaluation data during the data quality evaluation process, to ensure that all evaluation data operation behaviors have logs available for inspection; export all recorded evaluation log information for security auditing; Module for reclaiming evaluation user rights and destroying data space: used to reclaim evaluation user rights; reclaim and destroy the user data space created in step 1, delete the corresponding database user and database, and delete the evaluation user and all data created in step 1.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the data asset quality assessment method described in any one of claims 1 to 7 are implemented.

10. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the data asset quality assessment method as described in any one of claims 1-7 are implemented.

Citation Information

Patent Citations

  • Data thematic management method and system based on distributed architecture

    CN115292285A

  • Universal training evaluation system

    CN115994718A

  • Data asset management method and system

    CN116541372A

  • Supply chain evaluation method and device, terminal equipment and storage medium

    CN116777254A

  • Data quality comprehensive evaluation method based on dynamic weight

    CN117592822A

Cited By

  • EAM data processing method, system and device and storage medium

    CN120950497A