Data management method and device based on data consanguinity and electronic equipment

Abstract

The invention provides a data governance method and device based on data consanguinity and electronic equipment, and relates to the field of data governance. The method comprises the following steps: acquiring metadata of target data corresponding to a target governance demand; constructing a data blood relationship map of the target data according to the metadata; calculating a data quality propagation score, a data blood relationship path complexity, a data global influence score, a data leakage risk score and a data use efficiency score of the target data through the data blood relationship map; generating a data governance comprehensive score based on the data quality propagation score, the data consanguinity path complexity, the data global influence score, the data leakage risk score and the data use efficiency score; and performing data governance on the target data according to the data governance comprehensive score. By implementing the technical scheme provided by the invention, accurate treatment on the data is facilitated.

Description

Technical Field

[0001] This application relates to the technical field of data governance, and in particular, to a data governance method, apparatus, and electronic device based on data lineage. Background Art

[0002] In the process of modern data governance, the importance assessment of data is a key step in formulating governance strategies, allocating resources, and optimizing data management processes.

[0003] Currently, the importance level of data is determined by manual empirical rules or simple qualitative analysis. For example, the governance priority is directly evaluated manually based on the departmental attribution, usage frequency, or business requirements of the data. However, this qualitative method lacks a unified quantitative standard, resulting in large subjectivity and inconsistency in the evaluation results, making it difficult to meet the requirements of precise governance by simply relying on manual evaluation of the governance priority of data.

[0004] Therefore, there is an urgent need for a data governance method, apparatus, and electronic device based on data lineage. Summary of the Invention

[0005] This application provides a data governance method, apparatus, and electronic device based on data lineage, which is convenient for precise governance of data.

[0006] In the first aspect of this application, a data governance method based on data lineage is provided. The method includes: obtaining metadata of target data corresponding to a target governance requirement; constructing a data lineage graph of the target data according to the metadata; calculating a data quality propagation score, a data lineage path complexity, a data global impact score, a data leakage risk score, and a data usage efficiency score of the target data through the data lineage graph; generating a data governance comprehensive score based on the data quality propagation score, the data lineage path complexity, the data global impact score, the data leakage risk score, and the data usage efficiency score; and performing data governance on the target data according to the data governance comprehensive score.

[0007] By adopting the above technical solutions, through data quality propagation scores, path complexity, global impact scores, leakage risk scores, and usage efficiency scores, multiple key factors such as the quality, complexity, security, and efficiency of data governance are comprehensively considered, avoiding the one-sidedness of single-index evaluation. It can not only meet the quality optimization requirements but also take into account efficiency improvement and security risk control, adapting to the governance requirements of different scenarios. By quantifying each score, an objective basis for data priority evaluation is provided, avoiding the subjective bias of traditional qualitative methods. Constructing a data lineage graph provides visualization and precise analysis of data dependencies, ensuring that the data flow path and scope of influence are completely characterized. Using multiple indicators to generate a comprehensive score provides a unified governance priority ranking, helping to reasonably allocate governance resources and improve the scientific nature of decision-making. The weights can be dynamically adjusted according to changes in requirements or indicators, and the comprehensive score can be updated in real time to adapt to the governance objectives at different stages. Through the automatic construction of the data lineage graph and indicator calculation, the efficiency of data evaluation is realized, reducing manual intervention and improving the analysis speed. The priority governance driven by the comprehensive score ensures that governance resources are concentrated on the most important data, improving the overall efficiency and effectiveness of governance. The data leakage risk score helps to identify potential security hazards in advance and reduce data leakage and compliance risks. Identifying key data through the global impact score avoids business chain risks caused by incorrect operations on important data. Therefore, it is convenient to precisely govern the data.

[0008] Optionally, constructing the data lineage graph of the target data according to the metadata specifically includes: parsing the metadata to obtain data entities, where the data entities include database tables, fields, and files; extracting edges based on the data entities with dependency relationships among multiple data entities; obtaining additional data between the data entities and the edges; determining attributes according to the additional data; and constructing the data lineage graph of the target data based on the data entities, the edges, and the attributes.

[0009] By adopting the above technical solutions, data entities such as database tables, fields, and files are uniformly parsed to ensure that the construction of the lineage graph covers the entire path of data flow. By extracting the dependency relationships between entities, the upstream and downstream flow and relevance of data are revealed, providing support for comprehensively tracing the data source and impact. Using additional data to determine attributes further supplements the detailed information of the lineage graph, making the semantics of the graph clearer. By parsing the dependency relationships between entities, it is ensured that the generation of edges is based on the actual data flow logic, avoiding redundant or misleading information. The introduction of attributes enables the data lineage graph not only to display data entities and dependency relationships but also to provide context information to support more accurate analysis. Using metadata parsing and automatic extraction of dependency relationships significantly reduces manual participation and improves the construction efficiency. The constructed data lineage graph can directly serve multi-dimensional calculations such as data quality propagation and global impact analysis, significantly improving the efficiency of data governance decision-making. By combining entities, edges, and attributes, complex metadata relationships are presented in a structured graph form, facilitating understanding and operation. The constructed lineage graph can be visually displayed through a graphical tool, facilitating the data governance team to quickly locate key data and dependency paths. The additional attributes in the lineage graph support dynamic updates with business changes, ensuring its timeliness and practicality. This method supports multiple data entities (such as tables, fields, files) and adapts to the complex governance requirements of different data sources.

[0010] Optionally, through the data lineage graph, the data quality propagation score, data lineage path complexity, data global impact score, data leakage risk score, and data usage efficiency score of the target data are calculated. The data quality propagation score is specifically calculated using the following formula: ; where Q is the data quality propagation score, i is the i-th metadata, q i is the quality score of the i-th metadata, n is the total number of metadata, e is an edge, e ∈ E represents the set of edges in path e, and d(e) is the quality decay rate of path edge e.

[0011] By adopting the above technical solutions, through formula calculation, the originally fuzzy data quality propagation problem is transformed into specific numerical values, providing a basis for objective evaluation. Combining the quality scores of each metadata and the quality attenuation rate of the edges on the path, the changes in data quality during the transfer process are comprehensively considered, reflecting the actual situation. The quality attenuation rate of the path edges is introduced into the formula to accurately capture the quality loss of data during the complex transfer process, solving the problem of insufficient estimation of the impact of data transfer in traditional methods. This method can adapt to the complex data lineage graph structure with multiple paths and nodes and be flexibly applied to various complex data environments. According to the quality propagation score, the key nodes or paths with low quality or high attenuation can be quickly located, helping to prioritize the governance of problematic data. Through the quantitative analysis results, the governance resources are concentrated and allocated to the key paths and nodes, improving the overall efficiency of governance. This formula is based on dynamic data input and can be updated in real time with the changes in data quality and transfer conditions to adapt to the governance requirements at different time points. By providing a quantitative diagnosis of data quality, it assists decision-makers in judging which paths or nodes have the greatest impact on the overall quality. The data quality propagation score can be used as a reference indicator for the priority of data governance, providing a strong basis for governance planning.

[0012] Optionally, through the data lineage graph, the data quality propagation score, the complexity of the data lineage path, the global data impact score, the data leakage risk score, and the data usage efficiency score of the target data are calculated. The complexity of the data lineage path is specifically calculated using the following formula: ; where L is the complexity of the data lineage path, t is the time, e is an edge, e ∈ E represents the set of edges in path e, w(e, t) is the weight of edge e at time t, and f(v) is the complexity function related to node v.

[0013] By adopting the above technical solutions, the introduction of the weights of path edges at different times in the formula can reflect the dynamic changes of complex paths over time and capture the temporal characteristics in the actual scenario. By quantifying the complexity related to nodes through functions, it is ensured that the path complexity evaluation not only depends on the path itself but also comprehensively considers the complex attributes of nodes. The formula can handle complex data lineage networks with multiple paths and multiple nodes, and is particularly suitable for large-scale and deeply hierarchical data governance scenarios. By combining the edges and node attributes on the path, it avoids one-sided results caused by single-dimensional complexity evaluation and more comprehensively describes the complex characteristics of the data lineage graph. By calculating the complexity of the data lineage path, key high-complexity paths or nodes can be quickly located, providing a clear direction for optimizing data flow. High-complexity paths may have a negative impact on data processing performance or governance efficiency, and the calculation results of this formula can be used as a basis for resource allocation and process optimization. The formula takes into account the characteristics of the path edge weights changing over time, can analyze complexity in real time, and adapt to the dynamic changes of business requirements and data structures. The complexity evaluation can be combined with real-time monitoring to provide data support for the dynamic adjustment of governance strategies. The quantitative complexity evaluation can help the governance team optimize the data flow path, reduce redundant operations, and improve data processing efficiency. Identifying complex paths helps reduce data quality degradation or security risks caused by overly complex paths.

[0014] Optionally, through the data lineage graph, the data quality propagation score, data lineage path complexity, data global impact score, data leakage risk score, and data usage efficiency score of the target data are calculated. The data global impact score is specifically calculated using the following formula: ; where I is the data global impact score, V d is the set of directly affected nodes, α v is the weight coefficient of node v, σ(v) is the sensitivity score of node v, K is the maximum level of propagation, k is the k-th level, β k is the weight of level k, I k is the impact score of level k.

[0015] By adopting the above technical solutions, the formula accurately evaluates the influence scope and depth of data in the entire lineage graph by combining the directly affected node set and the propagation levels. The node weight coefficient and the level weight assign different weights to different nodes and levels, reflecting the priorities and importance in the actual business requirements. The node sensitivity score takes into account the importance of data nodes in the business process, ensuring that the impact assessment is more in line with the actual scenario. Through the high-weight manifestation of the sensitivity score, it helps to quickly identify and prioritize the protection of data of key nodes, reducing the risk of data leakage or misuse. The formula introduces the maximum level and the influence score of each level, which can deeply depict the global diffusion characteristics of data in a multi-level network. For complex multi-path lineage graphs, this formula can comprehensively evaluate the influences of different paths and levels without missing any potential key nodes. The weight coefficients support real-time adjustment to adapt to the dynamic changes of different business scenarios or data environments. When data or business changes, this method can quickly recalculate the global impact score to ensure the timeliness of governance strategies. The global data impact score provides a quantitative basis for data governance priorities, helping to quickly identify the key data nodes and paths that have the greatest impact on the business. Based on the global impact score, governance resources can be concentrated on high-impact nodes or paths, improving the governance efficiency and effect. By identifying high-impact paths or nodes, potential global risks caused by chain failures or misoperations can be discovered and mitigated in a timely manner. Global data impact analysis can help protect the key business data transfer links and ensure the stable operation of core businesses.

[0016] Optionally, through the data lineage graph, the data quality propagation score, the data lineage path complexity, the global data impact score, the data leakage risk score, and the data usage efficiency score of the target data are calculated. The data leakage risk score is specifically calculated using the following formula: ; where R is the data leakage risk score, v is a node, and V s is the set of nodes containing sensitive data, π(v) is the probability that node v is accessed, and r(v) is the leakage risk score of node v.

[0017] By adopting the above technical solution, the set of sensitive data nodes is concerned in the formula, clarifying that sensitive data is the core of leakage risk assessment and ensuring that resources are preferentially used to protect high-risk data. The probability of node access reflects the dynamics of leakage risk and helps to accurately predict the potential risks of frequently accessed data. The node access probability and leakage risk score can be dynamically updated according to real-time data to help cope with the changing business environment and security threats. By combining the access probability and leakage risk score, the data usage frequency and the risk characteristics of the node itself are comprehensively considered. The calculation result can quickly locate the node with the highest leakage risk, helping to formulate targeted security policies and avoid wasteful dispersion of resources. The formula provides a quantitative basis for the classification, permission control, and encryption protection of sensitive data, improving the scientific nature of security governance. By calculating the leakage risk score in the data lineage graph, a global visualization map of data leakage risk can be generated, clearly showing high-risk nodes and their influence ranges. As the node access behavior changes, the leakage risk score can be updated in real time to help the team continuously monitor and prevent potential security hazards. Based on the lineage graph, the formula extends the risk assessment of a single node to the entire data flow path, comprehensively considering the leakage risks that may be caused by chain propagation. Focusing on sensitive data and access behavior helps enterprises better meet the legal and compliance requirements of data privacy protection.

[0018] Optionally, through the data lineage graph, the data quality propagation score, data lineage path complexity, data global influence score, data leakage risk score, and data usage efficiency score of the target data are calculated. The data usage efficiency score is specifically calculated using the following formula: ; where M is the data usage efficiency score, T is the maximum time, t is the time, U(t) is the number of data accesses at time t, and C(t) is the data maintenance cost at time t.

[0019] By adopting the above technical solution, the formula combines the number of data accesses and the data maintenance cost, provides a comprehensive quantitative evaluation of data usage efficiency, and avoids the one-sidedness of only considering one aspect unilaterally. By comparing the number of accesses and the maintenance cost, while identifying efficient data usage, it helps to discover situations of resource waste or excessive consumption. The formula introduces a time variable, which can capture the dynamic changes of data usage efficiency at different time points and reflect the time series characteristics. Over time, by dynamically calculating the efficiency score, it is possible to discover and optimize data access and maintenance strategies in real time. The formula can help quickly locate inefficient data with high maintenance costs and few accesses, providing a clear optimization direction for the governance team. Based on the efficiency score, data storage, computing, and operation and maintenance resources can be reasonably allocated to improve the overall data governance efficiency. The data usage efficiency score provides a scientific basis for the data governance priority and supports the hierarchical governance of high-efficiency or low-efficiency data. By analyzing the time series changes of the efficiency score, long-term trends can be identified to formulate a more scientific resource investment and usage plan. By analyzing the data maintenance cost, costly operations can be identified and targeted optimization can be carried out to reduce the overall operating expenses. Concentrate resources on data that is frequently used and of higher value, and avoid the over-occupation of resources by inefficient data.

[0020] Optionally, generating a comprehensive data governance score based on the data quality propagation score, the data lineage path complexity, the data global impact score, the data leakage risk score, and the data usage efficiency score specifically includes: determining the ratio between the data quality propagation score and the data lineage path complexity; calculating a first product result between the ratio and a first weight; calculating a second product result between the data global impact score and a second weight; calculating a third product result between the data leakage risk score and a third weight; calculating a fourth product result between the data usage efficiency score and a fourth weight; summing the first product result, the second product result, and the fourth product result, and subtracting the third product result to obtain the comprehensive data governance score.

[0021] By adopting the above technical solution, the method comprehensively considers multiple key dimensions such as data quality propagation score, path complexity, global impact, leakage risk, and usage efficiency, and comprehensively reflects the overall situation of data governance. By combining different dimensions in the form of weights, the method achieves a balance among multiple indicators, avoiding the limitation of a single indicator deviating from the actual situation. Different weight settings can reflect business objectives, such as paying more attention to data quality or more attention to security, etc., helping to make more accurate decisions. The ratio of the data quality propagation score to the path complexity is used as an input, which reflects the propagation effect of data quality under the influence of path complexity, enhancing the logical rigor of the scoring. Through the weighted summation and difference operations of different indicators, the scoring result is more scientific and can accurately reflect the advantages and disadvantages of data governance. The comprehensive score provides an intuitive quantitative result for different data objects, which can be used for quick sorting to determine the priority of data governance. Through the comparative analysis of high-score and low-score data, it can help the governance team quickly locate the governance areas that need to be focused on. The comprehensive score can be updated in real time as the scores of each indicator change, reflecting the dynamic changes in the effectiveness of data governance and providing a basis for continuous optimization. The comprehensive score can help enterprises regularly evaluate the effectiveness of governance strategies and ensure the gradual achievement of governance goals. The comprehensive score provides a unified standard for data governance, contributing to the communication and collaboration among different departments and teams. Through the scoring result, governance resources can be preferentially allocated to high-priority data, significantly improving the governance efficiency.

[0022] In the second aspect of the present application, a data governance device based on data lineage is provided. The data governance device includes an acquisition module and a processing module. Among them, the acquisition module is used to acquire the metadata of the target data corresponding to the target governance requirement; the processing module is used to construct a data lineage graph of the target data according to the metadata; the processing module is further used to calculate the data quality propagation score, data lineage path complexity, data global impact score, data leakage risk score, and data usage efficiency score of the target data through the data lineage graph; the processing module is further used to generate a data governance comprehensive score based on the data quality propagation score, the data lineage path complexity, the data global impact score, the data leakage risk score, and the data usage efficiency score; the processing module is further used to perform data governance on the target data according to the data governance comprehensive score.

[0023] In the third aspect of the present application, an electronic device is provided. The electronic device includes a processor, a memory, a user interface, and a network interface. The memory is used to store instructions, and both the user interface and the network interface are used to communicate with other devices. The processor is used to execute the instructions stored in the memory so that the electronic device executes the method described above.

[0024] In a fourth aspect of the present application, a computer-readable storage medium is provided. The computer-readable storage medium stores instructions that, when executed, perform the method described above.

[0025] In summary, one or more technical solutions provided in the present application have at least the following technical effects or advantages: By considering data quality propagation score, path complexity, global impact score, leakage risk score, and usage efficiency score, multiple key factors such as the quality, complexity, security, and efficiency of data governance are comprehensively considered, avoiding the one-sidedness of single-index evaluation. It can not only meet the quality optimization requirements but also take into account efficiency improvement and security risk control, adapting to the governance requirements of different scenarios. By quantifying each score, an objective basis for data priority evaluation is provided, avoiding the subjective deviation of traditional qualitative methods. Constructing a data lineage graph provides visualization and precise analysis of data dependencies, ensuring that the data flow path and scope of influence are completely depicted. Using multiple indicators to generate a comprehensive score provides a unified governance priority ranking, helping to reasonably allocate governance resources and improve the scientificity of decision-making. The weights can be dynamically adjusted according to demand changes or indicators, and the comprehensive score can be updated in real time to adapt to the governance objectives at different stages. Through the automatic construction of the data lineage graph and index calculation, the efficiency of data evaluation is realized, reducing manual intervention and improving the analysis speed. The priority governance driven by the comprehensive score ensures that governance resources are concentrated on the most important data, improving the overall efficiency and effectiveness of governance. The data leakage risk score helps to identify potential security hazards in advance and reduce data leakage and compliance risks. Identifying key data through the global impact score avoids business chain risks caused by incorrect operations on important data. Therefore, it is convenient to perform precise governance on data. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a schematic flowchart of a data governance method based on data lineage provided by an embodiment of the present application; Figure 2 It is another schematic flowchart of a data governance method based on data lineage provided by an embodiment of the present application; Figure 3 It is a schematic block diagram of a data governance device based on data lineage provided by an embodiment of the present application; Figure 4 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application.

[0027] Description of the reference numerals: 31, acquisition module; 32, processing module; 41, processor; 42, communication bus; 43, user interface; 44, network interface; 45, memory. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0028] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of this specification. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all the embodiments.

[0029] In the description of the embodiments of this application, words such as "for example" or "for instance" are used to represent examples, illustrations or explanations. Any embodiment or design solution described as "for example" or "for instance" in the embodiments of this application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Exactly speaking, the use of words such as "for example" or "for instance" is intended to present relevant concepts in a specific way.

[0030] In the description of the embodiments of this application, the meaning of the term "plural" refers to two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals. In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the technical features indicated. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. The terms "include", "comprise", "have" and their variants all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0031] In modern data governance, the importance assessment of data is the core link in formulating governance strategies, optimizing data management processes, and reasonably allocating resources.

[0032] However, the current data importance assessment mostly relies on manual experience rules or simple qualitative analysis. For example, direct human judgment based on the departmental attribution, usage frequency, or business requirements of data is used to determine the governance priority. Due to the lack of a unified quantitative standard, the evaluation results of this qualitative method often have strong subjectivity and inconsistency. With the continuous increase in data scale and complexity, relying solely on manual evaluation has become difficult to meet the requirements of precise governance.

[0033] To solve the above technical problems, the embodiments of this application provide a data governance method based on data lineage, referring to Figure 1 , Figure 1 is a schematic flowchart of a data governance method based on data lineage provided by the embodiments of this application. This data governance method is applied to a server and includes steps S110 to S150. The above steps are as follows: S110. Obtain the metadata of the target data corresponding to the target governance requirement.

[0034] Specifically, a server refers to a computing device or system that hosts data governance work and is the main execution unit for data analysis and processing. Target governance requirements are the specific requirements or issues for data governance. For example, improving data quality, optimizing query efficiency, reducing security risks, etc. Target data refers to specific data sets or data resources directly related to governance requirements, such as a certain database table, a specific data file, etc. Metadata is data that describes target data and includes the following: data structure (table name, field name, field type); data source (data generation time, collection channel); data relationship (primary key, foreign key, dependency relationship); data operation log (modification time, access count, etc.). The server obtains this metadata through interfaces or tools (such as a metadata management system) to provide support for subsequent data governance.

[0035] For example, suppose an enterprise hopes to optimize the efficiency of report generation in its financial system. Then the target governance requirement is to improve the efficiency of financial report generation and reduce generation latency. The target data is the report-related data stored in the financial database, such as the "income statement", "expense statement", etc. The server extracts the metadata of these data through a metadata management system or a database management tool. The metadata includes the structure of the data table, such as the field name and field type, and also includes the data storage relationship, such as the primary-foreign key relationship between tables. In addition, it includes data volume information, such as the number of rows and field size of the data table. And it includes the data access log, such as which queries frequently use these data and the access path when generating reports. The metadata is used to analyze the reasons for slow report generation, such as whether there are missing indexes for certain fields or whether the join queries are too complex. At the same time, optimize the query path, reduce the access to useless fields, and improve the data retrieval speed.

[0036] S120. Construct a data lineage graph of the target data according to the metadata.

[0037] Specifically, the server extracts information related to data lineage from the metadata, including: entity information, such as specific data entities like data tables, fields, files, etc.; relationship information, the dependency relationships between data entities, for example, a field in a certain table is derived from another table; additional attributes, such as data generation time, operation records, sensitive information marks, etc. Among them, each data entity serves as a node. For example, the table Sales is a node, and the field Revenue is another node. If a data entity depends on another data entity, then an edge is added between the two. For example, the Revenue field is calculated from the Orders table through ETL. Based on the above nodes and edges, a graph structure is constructed and presented in a visual form. For example: Intra-table lineage: A field depends on other fields. Cross-table lineage: Tables are connected through primary-foreign key associations or ETL processes.

[0038] In a possible implementation, according to the metadata, a data lineage graph of the target data is constructed, specifically including: parsing the metadata to obtain data entities, where the data entities include database tables, fields, and files; extracting edges based on the data entities with dependency relationships among multiple data entities; obtaining additional data between the data entities and the edges; determining attributes according to the additional data; and constructing the data lineage graph of the target data based on the data entities, edges, and attributes.

[0039] Specifically, the server parses the metadata to extract core information, including data entities and their relationships. The metadata may come from a database management system (DBMS), a data warehouse tool, or the logs of an ETL process. The data entity is a specific component unit of data, including: Database table: a collection of structured data, such as the Orders table for orders; Field: a refined unit of the data table, such as OrderID, Price, etc.; File: a storage unit for unstructured or semi-structured data, such as CSV and JSON files. The edge is the dependency relationship between data entities, reflecting the flow of data from the source to the target. The edge may be defined by foreign key relationships, data calculation logics (such as ETL), application operations, etc. The additional data describes the characteristics of the data entities and edges, for example: Edge weight: indicating the importance of the dependency relationship. Timestamp: marking the update or processing time of the data. Operation type: insert, delete, modify, etc. Attributes are used to further characterize the features of nodes and edges, such as the sensitivity of nodes and the transmission efficiency of edges.

[0040] Further, first, the server parses the metadata to extract data entities, extracting entity information such as data tables, fields, and files from the metadata. The basic descriptions of the entities (table names, field names, data types, etc.) are extracted. Second, the server extracts the dependency relationships between the data entities, and determines the dependency relationships between the data entities according to the information in the metadata, such as foreign key relationships, ETL logs, or data processing scripts. Each dependency relationship is represented by an edge, and the direction from the source entity to the target entity is indicated. Then, the server obtains the additional data, collecting the additional information between the entities and the edges, for example: data transmission frequency, access rights or sensitivity levels, and the calculation logic of the dependency relationship. The server uses the additional data to determine the attributes of the nodes (data entities) and edges (dependency relationships), for example: Nodes: the update time of the table, the sensitivity level of the field. Edges: the dependency strength (for example, the weight of the dependency relationship with high-frequency calls is higher). Finally, the server uses the data entities as nodes, the dependency relationships as edges, and the attributes as the features of the graph to generate the final data lineage graph. The form of the data lineage graph is a directed graph, representing the full path of data from the source to the target.

[0041] S130. Calculate the data quality propagation score, data lineage path complexity, data global impact score, data leakage risk score, and data usage efficiency score of the target data through the data lineage graph.

[0042] Specifically, the server calculates various scores of the target data through the data lineage graph, which describes the process of multi-dimensional analysis based on the data lineage graph. The purpose of these scores is to comprehensively evaluate the performance of the target data in terms of quality, complexity, influence, risk, and usage efficiency in a quantitative manner, so as to provide a reliable basis for data governance. The data quality propagation score is used to evaluate the transmission and change of data quality in the data lineage path, reflecting the overall quality performance of the data from the source to the target.

[0043] Trace the data flow path from upstream to downstream through the lineage graph, analyze the quality of each data entity, and consider the impact of each path on the quality of the target data. Suppose SalesReport.NetProfit depends on Orders.Revenue and Products.Cost. If the data quality of Orders.Revenue is low (e.g., there are many missing values), then this defect will be transmitted along the lineage path to NetProfit, resulting in a decrease in its quality score. The data lineage path complexity is used to measure whether the lineage path structure of the target data is complex, including the number of nodes and edges, the depth of the path, and the complexity of the dependency relationship. The server analyzes the topological structure of the data lineage graph to identify the number of nodes and edges in the path, the depth of the hierarchy, and the degree of nesting of the relationships. Suppose if the generation path of SalesReport.NetProfit is relatively complex and requires passing through multiple intermediate nodes (such as Orders.Revenue and Products.Profit) and multiple edges, then the complexity score will be higher. This means that the path needs to be optimized in governance to reduce maintenance costs. The data global impact score is used to evaluate the scope and degree of the impact of changes or problems in the target data on the entire system. The server analyzes the downstream nodes and paths of the target data in the lineage graph to quantify its impact on downstream data entities and the overall system. For example, suppose Products.Cost is a key data source for SalesReport.NetProfit, and reports from multiple other departments also depend on Cost. If an error occurs in Cost, it will affect a large number of downstream reports, and its global impact score will be higher.

[0044] Furthermore, the data leakage risk score is used to evaluate the possible leakage risk of the target data in the lineage path, especially the exposure probability of sensitive data. The server identifies sensitive data nodes involved in the lineage path, analyzes the frequency of data access, the security level of the storage location, and the encryption and permission control during the transmission process. For example, if Products.Cost is a highly sensitive field and it is passed to SalesReport.NetProfit through an unencrypted path, there is a high data leakage risk. The data usage efficiency score is used to measure the relationship between the usage value and maintenance cost of the target data in the system, reflecting the resource utilization efficiency of the data. The server analyzes the utilization efficiency of the data by counting the access frequency and call volume of the target data and combining its storage and processing costs. For example, if Orders.Revenue is accessed multiple times a day for different analysis tasks but its storage and computing costs are low, the usage efficiency score is high. Conversely, if some data is computationally complex but has a low usage frequency, the efficiency score will decrease. By analyzing the lineage graph, the server can evaluate the target data from multiple dimensions such as quality, structural complexity, global impact, leakage risk, and usage efficiency. These scores are quantitative analyses based on the lineage graph, which can provide a comprehensive control over the entire life cycle of the data, help identify key points and weak links in data governance, and provide data-driven quantitative basis for data governance decisions.

[0045] In a possible implementation, through the data lineage graph, the data quality propagation score, data lineage path complexity, data global impact score, data leakage risk score, and data usage efficiency score of the target data are calculated. The data quality propagation score is specifically calculated using the following formula: ; where Q is the data quality propagation score, i is the i-th metadata, q i is the quality score of the i-th metadata, n is the total number of metadata, e is an edge, e ∈ E represents the set of edges in path e, and d(e) is the quality decay rate of path edge e.

[0046] Specifically, this passage describes a method for calculating the target data quality propagation score based on the data lineage graph. The data quality propagation score is a quantitative indicator used to evaluate the attenuation and accumulation of data quality during the process of data quality being transmitted from upstream source data to target data. The purpose is to measure the contribution of the quality of each data entity on the data lineage path to the target data quality and consider the quality attenuation of data on each path. Among them, the data quality propagation score represents the comprehensive quality of the target data. The iii-th metadata refers to each data entity or field in the data lineage path. The quality score of the i-th metadata is evaluated based on field integrity, accuracy, etc. The total number of metadata is the number of nodes on the path. The edges in the path represent the dependency relationships between data entities. The quality attenuation rate of the path edge e represents the degree of quality reduction during the transmission of data from upstream to downstream.

[0047] Among them, the server first extracts the paths related to the target data from the data lineage graph, including upstream nodes (metadata) and edges (dependency relationships). And calculate the quality score for each metadata node. The quality score can be evaluated through the following dimensions: data integrity (whether there are missing values), data accuracy (whether it conforms to business logic), and data consistency (whether it matches across systems). Next, the server analyzes the attenuation rate of each edge in the path. The attenuation rate may be related to information loss caused by data transformation (such as aggregation, filtering) and errors or biases existing in the data processing process. Finally, the server comprehensively calculates the data quality propagation score of the target data based on the quality scores of the nodes and the attenuation rates of the edges in the path.

[0048] In a possible implementation manner, through the data lineage graph, the data quality propagation score, data lineage path complexity, data global impact score, data leakage risk score, and data usage efficiency score of the target data are calculated. The data lineage path complexity is specifically calculated using the following formula: ; Among them, L is the data lineage path complexity, t is the time, e is the edge, e ∈ E represents the set of edges in the path e, w(e, t) is the weight of the edge e at time t, and f(v) is the complexity function related to the node v.

[0049] Specifically, the complexity of the data lineage path is a quantitative metric used to evaluate the complexity of the dependency paths of target data in the data lineage graph. It reflects the difficulty of data flow and dependencies by analyzing the edges, nodes, and their weights and complexity functions in the path. This complexity helps identify and optimize complex paths in data governance, improving the efficiency and accuracy of data processing. The complexity of the data lineage path measures the overall complexity of the dependency paths of target data. It is used to handle dynamic scenarios at all times, reflecting the change of complexity over time. The weight of edge e in the path at time t represents the importance of a certain dependency relationship. For example, some edges may have a higher weight due to high-frequency calls or greater importance. f(v) is the complexity function related to node v, representing the complexity of the node itself. For example, the number of fields contained in the data node, the difficulty of data operations (such as aggregation, cleaning, etc.) involved in the node, and the computing resource consumption of the node.

[0050] Among them, the server first extracts the paths on which the target data depends from the data lineage graph, including edge e and node v. The server calculates the weight w(e,t) for each edge e in the path. The weight calculation may be based on the following factors: access frequency of the edge: edges with higher access frequency have higher weights; data volume size: edges involving large data flows have higher weights. Then, the server evaluates the complexity f(v) of each node, which may refer to: the number of fields of the node: the more fields, the higher the complexity. Such as whether the node contains complex aggregation operations. Finally, the server combines the weights on the path and the node complexity and calculates the path complexity L according to the formula.

[0051] In a possible implementation manner, through the data lineage graph, the data quality propagation score, the complexity of the data lineage path, the data global impact score, the data leakage risk score, and the data usage efficiency score of the target data are calculated. The data global impact score is specifically calculated using the following formula: ; where I is the data global impact score, V d is the set of directly affected nodes, α v is the weight coefficient of node v, σ(v) is the sensitivity score of node v, K is the maximum level of propagation, k is the k-th level, β k is the weight of level k, I k is the impact score of level k.

[0052] Specifically, the data global impact score is a quantitative metric used to measure the degree of influence of target data on other data entities in the entire data lineage graph. This metric comprehensively evaluates from two perspectives: direct influence and multi-level propagation. It focuses on analyzing the set of nodes directly associated with the target data (directly affected nodes), as well as the comprehensive influence at each level when the influence propagates layer by layer along the path. The data global impact score measures the global influence of the target data in the entire lineage graph. The set of nodes directly affected by the target data contains entities directly associated with the target data. The weight coefficient of node v represents the importance of this node. For example: the weight of a core business node may be higher; the weight of a non-keyword field node may be lower. The sensitivity score of node v is used to evaluate the sensitivity of the node to changes in data quality. For example: nodes with a greater impact on business due to data errors are more sensitive. K is the maximum level of propagation, defining the depth of influence propagation, which depends on the complexity of the business scenario. The weight of level k reflects the importance of propagation at this level. For example, the weight of levels farther from the target data may be lower. The impact score of level k represents the comprehensive influence of all nodes at the k-th level.

[0053] Among them, the server first starts from the target data and finds all the nodes directly related to it. These nodes are directly affected by changes in the target data. Then, the server calculates the influence of each directly affected node, and the weight and sensitivity jointly determine the importance of the node. Secondly, the server calculates the influence scores of the influence propagating to different levels layer by layer. The influence score of each layer is the comprehensive value of the influence of all nodes in this layer. The contribution of each layer of propagation is adjusted using the layer weight. Finally, the server performs a weighted sum of the direct influence and the layer-by-layer propagation influence to obtain the global influence score I.

[0054] In a possible implementation, through the data lineage graph, the data quality propagation score, data lineage path complexity, data global impact score, data leakage risk score, and data usage efficiency score of the target data are calculated. The data leakage risk score is specifically calculated using the following formula: ; where R is the data leakage risk score, v is the node, and V s is the set of nodes containing sensitive data, π(v) is the probability that node v is accessed, and r(v) is the leakage risk score of node v.

[0055] Specifically, the data leakage risk score is an indicator used to evaluate the degree of leakage risk of target data or its related nodes in the data lineage graph. This score comprehensively considers the following factors. For example, the set of sensitive data nodes in the data lineage graph, and also the probability of each node being accessed, which reflects its exposure level, as well as the inherent leakage risk score of each node, indicating the impact of data characteristics on the likelihood of leakage. By weighted summing these factors, the comprehensive risk of sensitive data leakage in the entire data lineage graph is obtained. The data leakage risk score measures the overall leakage likelihood of the target data. The nodes in the data lineage graph represent data entities (such as tables, fields, or files). The set of nodes containing sensitive data. For example: fields involving personal privacy, such as user_id, email_address; tables involving trade secrets, such as financial_records; the probability of node v being accessed. The access probability can be estimated based on logs, permission assignments, or historical records. The leakage risk score of node v depends on the data type, storage security, and its potential impact in the leakage scenario. The essence of the formula is to sum the products of the access probabilities and leakage risks of all nodes in the set of sensitive nodes.

[0056] Among them, the server first identifies the set of sensitive data nodes, that is, extracts sensitive information from the metadata, such as marking the nodes containing fields such as identity information and financial information. The server classifies the data according to business rules or sensitivity grading. The server calculates the historical access frequency based on the access logs and considers the user permission distribution to calculate the likelihood of the node being exposed to users. Next, the server comprehensively considers factors such as the data storage method, encryption policy, and security vulnerabilities. For example, unencrypted fields have a higher leakage risk score, while encrypted fields or fields with a lower sensitivity grading have a lower risk score. Finally, the server sums the products of the access probabilities and leakage risk scores of each node in the set of sensitive nodes to obtain the overall data leakage risk score R.

[0057] In a possible implementation manner, through the data lineage graph, the data quality propagation score, data lineage path complexity, data global impact score, data leakage risk score, and data usage efficiency score of the target data are calculated. The data usage efficiency score is specifically calculated using the following formula: ; where M is the data usage efficiency score, T is the maximum time, t is the time, U(t) is the number of data accesses at time t, and C(t) is the data maintenance cost at time t.

[0058] Specifically, the data usage efficiency score is an indicator used to measure the actual utilization efficiency of target data at different time points. By calculating the ratio of the number of data accesses to the maintenance cost and combining the time dimension, the rationality of the data usage value and resource investment is evaluated. The data usage efficiency score represents the comprehensive utilization efficiency of the target data. The maximum moment refers to the last time point within the data life cycle or monitoring cycle. Moment t is used to break down the data usage situation. The number of data accesses at moment t reflects the data usage frequency. The data maintenance cost at moment t represents the resource consumption of the data at this time point, such as storage, backup, or computing resources. This formula measures whether the data is efficiently utilized during its life cycle by dividing the number of accesses by the maintenance cost at each moment and summing them up.

[0059] Among them, the server first collects data access logs, extracts access records from the database, file system, or data lake, and counts the access frequency by moment. For example, if a certain table is accessed 10 times a day, then U(t) = 10. Secondly, the server analyzes the data storage and operation resource consumption, such as hardware, network bandwidth, and computing processing. For example, if the data storage cost is 5 yuan per GB per day, then the cost can be calculated based on the storage volume as C(t) = 5 × storage volume. Then, at each time point t, the server divides the number of accesses by the maintenance cost to obtain the data usage efficiency at that moment. Finally, the server accumulates the single-moment efficiencies at all moments to obtain the total data usage efficiency score M.

[0060] S140. Generate a comprehensive data governance score based on the data quality propagation score, data lineage path complexity, data global impact score, data leakage risk score, and data usage efficiency score.

[0061] Specifically, the comprehensive data governance score is the result of a comprehensive quantitative evaluation of the target data from multiple dimensions. It comprehensively considers the performance in five dimensions: data quality, data complexity, data global impact, leakage risk, and usage efficiency. The server generates a unified indicator by calculating the weighted sum and difference of these scores as the basis for evaluation and decision-making.

[0062] In a possible implementation manner, referring to Figure 2 , Figure 2Another process schematic diagram of a data governance method provided by an embodiment of this application. Based on the data quality propagation score, the data lineage path complexity, the data global impact score, the data leakage risk score, and the data usage efficiency score, a data governance comprehensive score is generated, specifically including steps S210 to S260. The above steps are as follows: S210. Determine the ratio between the data quality propagation score and the data lineage path complexity; S220. Calculate the first product result of the ratio and the first weight; S230. Calculate the second product result of the data global impact score and the second weight; S240. Calculate the third product result of the data leakage risk score and the third weight; S250. Calculate the fourth product result of the data usage efficiency score and the fourth weight; S260. Sum the first product result, the second product result, and the fourth product result, and subtract the third product result to obtain the data governance comprehensive score.

[0063] Specifically, the server performs a ratio operation on the data quality propagation score and the data lineage path complexity to reflect the level of data quality relative to data complexity. The purpose is to identify whether the data still maintains high quality in a relatively complex lineage structure. The first weight is the weight of the ratio of data quality to complexity, indicating the importance of this part of the ratio. The second weight is the weight of the data global impact score, reflecting the degree of emphasis on the global impact of the data on other parts of the system. The third weight is the weight of the data leakage risk score, used to measure the security risk of sensitive data. The fourth weight is the weight of the data usage efficiency score, indicating the degree of emphasis on the actual usage value of the data. Among them, the above weights can be custom-set according to the target data governance requirements. The server adds the above weighted product results to summarize the contributions of each dimension to the comprehensive score. At the same time, subtract the product result of the leakage risk weight because the data leakage risk is a negative impact and should be deducted. Finally, a quantitative comprehensive score is obtained. The higher the value, the higher the priority of the data in governance.

[0064] For example, assume the comprehensive score is 39.6, which is higher than the preset scoring threshold. This indicates that this customer data table has a high priority in data governance for the following reasons: high data quality dissemination, relatively controllable path complexity, significant global impact, and strong relevance to other parts of the system. High usage efficiency, with high access frequency and controllable maintenance costs. There are certain potential risks in the leakage risk score, but overall it still belongs to high-priority data. Therefore, by comprehensively considering multi-dimensional data performance, the one-sidedness of a single indicator is avoided. Flexible weight configuration can balance the governance goals of different enterprises, such as emphasizing quality, efficiency, or security. It supports complex data environments, such as multi-table dependencies, high access frequencies, or high-risk data. Subjective interference is eliminated, making the decision-making process of data governance more scientific and efficient. Among them, high-impact data such as core business-related data is prioritized for governance. High-risk data is quickly identified, such as planning resources in advance for security enhancement. The efficiency of data governance is improved, such as directly locating high-value data through scoring to enhance the governance effect.

[0065] S150. Perform data governance on the target data according to the comprehensive data governance score.

[0066] Specifically, this passage describes the specific operations for governing the target data based on the comprehensive scoring results. The server, based on the previously calculated comprehensive data governance score, allocates governance resources, determines governance strategies, or adjusts governance priorities for the target data according to the score. The goals of data governance include: improving data quality, such as fixing errors in the data, deleting duplicate data, and completing missing data; optimizing the data structure, such as optimizing the data lineage path and reducing complexity; enhancing data security, such as strengthening the protection of sensitive data and reducing the data leakage risk; and improving usage efficiency, such as reducing unnecessary data storage or redundancy and improving access efficiency.

[0067] Among them, the server screens high-priority data by sorting the comprehensive scores of all data. The higher the score of the target data, the higher the priority for governance. More human, computing resources, and time may be allocated to high-priority data. Low-priority data may use batch automation governance tools to reduce governance costs. The server takes targeted governance measures according to the characteristics of the data and the composition of the score. Example 1: If the score of a certain data is low mainly because the quality dissemination score is low, then focus on fixing the errors in the data. Example 2: If the score is low due to a low leakage risk score, then strengthen the access control of the data. During the governance process, the score can be adjusted dynamically. If the governance measures are effective, the score will increase, and the server can update the strategy in a timely manner.

[0068] Therefore, the server comprehensively considers multiple key factors such as the quality, complexity, security, and efficiency of data governance through data quality propagation scores, path complexity, global impact scores, leakage risk scores, and usage efficiency scores, avoiding the one-sidedness of single-index evaluation. It can not only meet the quality optimization requirements but also take into account efficiency improvement and security risk control, adapting to the governance requirements of different scenarios. By quantifying each score, the server provides an objective basis for data priority evaluation, avoiding the subjective bias of traditional qualitative methods. Constructing a data lineage graph provides visualization and accurate analysis of data dependencies, ensuring that the data flow path and scope of influence are completely characterized. The server generates a comprehensive score using multiple indicators, providing a unified governance priority ranking, helping to reasonably allocate governance resources, and improving the scientific nature of decision-making. The weights can be dynamically adjusted according to demand changes or indicators, and the comprehensive score can be updated in real time to adapt to the governance goals of different stages. Through the automatic construction of the data lineage graph and indicator calculation, the server realizes the high efficiency of data evaluation, reduces manual intervention, and improves the analysis speed. The priority governance driven by the server's comprehensive score ensures that governance resources are concentrated on the most important data, improving the overall efficiency and effectiveness of governance. The data leakage risk score helps to identify potential security hazards in advance and reduce data leakage and compliance risks. The server identifies key data through the global impact score, avoiding the business chain risks brought by incorrect operations on important data. Therefore, it is convenient to precisely govern the data.

[0069] This application also provides a data governance device based on data lineage. Refer to Figure 3 , Figure 3 which is a schematic diagram of the modules of a data governance device based on data lineage provided by an embodiment of this application. The data governance device is a server, and the server includes an acquisition module 31 and a processing module 32. Among them, the acquisition module 31 acquires the metadata of the target data corresponding to the target governance requirement; the processing module 32 constructs a data lineage graph of the target data according to the metadata; the processing module 32 calculates the data quality propagation score, data lineage path complexity, data global impact score, data leakage risk score, and data usage efficiency score of the target data through the data lineage graph; the processing module 32 generates a data governance comprehensive score based on the data quality propagation score, data lineage path complexity, data global impact score, data leakage risk score, and data usage efficiency score; the processing module 32 performs data governance on the target data according to the data governance comprehensive score.

[0070] In a possible implementation, the processing module 32 constructs a data lineage graph of the target data according to the metadata, which specifically includes: the processing module 32 parses the metadata to obtain data entities, and the data entities include database tables, fields, and files; the processing module 32 extracts edges according to the data entities with dependency relationships among multiple data entities; the acquisition module 31 acquires additional data between the data entities and the edges; the processing module 32 determines attributes according to the additional data; the processing module 32 constructs a data lineage graph of the target data based on the data entities, edges, and attributes.

[0071] In a possible implementation, the processing module 32 calculates the data quality propagation score, data lineage path complexity, data global impact score, data leakage risk score, and data usage efficiency score of the target data through the data lineage graph. The data quality propagation score is specifically calculated using the following formula: ; where Q is the data quality propagation score, i is the i-th metadata, q i is the quality score of the i-th metadata, n is the total number of metadata, e is an edge, e ∈ E represents the set of edges in path e, and d(e) is the quality decay rate of path edge e.

[0072] In a possible implementation, the processing module 32 calculates the data quality propagation score, data lineage path complexity, data global impact score, data leakage risk score, and data usage efficiency score of the target data through the data lineage graph. The data lineage path complexity is specifically calculated using the following formula: ; where L is the data lineage path complexity, t is the time, e is an edge, e ∈ E represents the set of edges in path e, w(e, t) is the weight of edge e at time t, and f(v) is a complexity function related to node v.

[0073] In a possible implementation, the processing module 32 calculates the data quality propagation score, data lineage path complexity, data global impact score, data leakage risk score, and data usage efficiency score of the target data through the data lineage graph. The data global impact score is specifically calculated using the following formula: ; where I is the data global impact score, V d is the set of directly affected nodes, α v is the weight coefficient of node v, σ(v) is the sensitivity score of node v, K is the maximum level of propagation, k is the k-th level, β k is the weight of level k, I k is the impact score of level k.

[0074] In a possible implementation, the processing module 32 calculates, through the data lineage graph, the data quality propagation score, the data lineage path complexity, the data global impact score, the data leakage risk score, and the data usage efficiency score of the target data. The data leakage risk score is specifically calculated using the following formula: ; where R is the data leakage risk score, v is a node, V s is the set of nodes containing sensitive data, π(v) is the probability that node v is accessed, and r(v) is the leakage risk score of node v.

[0075] In a possible implementation, the processing module 32 calculates, through the data lineage graph, the data quality propagation score, the data lineage path complexity, the data global impact score, the data leakage risk score, and the data usage efficiency score of the target data. The data usage efficiency score is specifically calculated using the following formula: ; where M is the data usage efficiency score, T is the maximum time, t is the time, U(t) is the number of data accesses at time t, and C(t) is the data maintenance cost at time t.

[0076] In a possible implementation, the processing module 32 generates a comprehensive data governance score based on the data quality propagation score, the data lineage path complexity, the data global impact score, the data leakage risk score, and the data usage efficiency score. Specifically, the processing module 32 determines the ratio between the data quality propagation score and the data lineage path complexity; the processing module 32 calculates the first product result between the ratio and the first weight; the processing module 32 calculates the second product result between the data global impact score and the second weight; the processing module 32 calculates the third product result between the data leakage risk score and the third weight; the processing module 32 calculates the fourth product result between the data usage efficiency score and the fourth weight; the processing module 32 sums the first product result, the second product result, and the fourth product result, and subtracts the third product result to obtain the comprehensive data governance score.

[0077] It should be noted that when the device provided in the above embodiment implements its functions, only the division of the above functional modules is used for illustration. In actual applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process can be seen in the method embodiment, which will not be elaborated here.

[0078] The present application also provides an electronic device. Referring to Figure 4 , Figure 4 which is a schematic structural diagram of an electronic device provided by an embodiment of the present application. The electronic device may include: at least one processor 41, at least one network interface 44, a user interface 43, a memory 45, and at least one communication bus 42.

[0079] Among them, the communication bus 42 is used to realize the connection and communication between these components.

[0080] Among them, the user interface 43 may include a display screen (Display) and a camera (Camera). Optionally, the user interface 43 may further include a standard wired interface and a wireless interface.

[0081] Among them, the network interface 44 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface).

[0082] Among them, the processor 41 may include one or more processing cores. The processor 41 uses various interfaces and lines to connect various parts within the entire server. By running or executing instructions, programs, code sets, or instruction sets stored in the memory 45, and by calling data stored in the memory 45, it executes various functions of the server and processes data. Optionally, the processor 41 may be implemented in at least one of the following hardware forms: Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The processor 41 may integrate one or several combinations of a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), and a modem, etc. Among them, the CPU mainly processes the operating system, the user interface, and application programs, etc.; the GPU is responsible for the rendering and drawing of the content to be displayed on the display screen; the modem is used to process wireless communication. It can be understood that the above-mentioned modem may not be integrated into the processor 41 and may be implemented separately by a single chip.

[0083] Among them, the memory 45 may include a Random Access Memory (RAM), or may also include a Read-Only Memory. Optionally, the memory 45 includes a non-transitory computer-readable storage medium. The memory 45 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 45 may include a program storage area and a data storage area. Among them, the program storage area may store instructions for implementing the operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above method embodiments, etc.; the data storage area may store the data involved in the above method embodiments. Optionally, the memory 45 may also be at least one storage device located far from the aforementioned processor 41. As Figure 4 shown, in the memory 45 as a computer storage medium, there may be included an operating system, a network communication module, a user interface module, and an application program of a data governance method based on data lineage.

[0084] In Figure 4 the electronic device shown, the user interface 43 is mainly used to provide an input interface for the user to obtain the data input by the user; and the processor 41 can be used to call the application program of a data governance method stored in the memory 45, and when executed by one or more processors, cause the electronic device to execute the method as described in one or more of the above embodiments.

[0085] It should be noted that for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that this application is not limited by the described action sequence, because according to this application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0086] This application also provides a computer-readable storage medium, and the computer-readable storage medium stores instructions. When executed by one or more processors, cause the electronic device to execute the method as described in one or more of the above embodiments.

[0087] In the above embodiments, the descriptions of the various embodiments have their own emphases. For the parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0088] In several embodiments provided in the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some service interfaces. The indirect coupling or communication connection of the device or unit can be in electrical or other forms.

[0089] The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0090] In addition, in each embodiment of the present application, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0091] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in each embodiment of the present application. And the aforementioned memory includes: various media such as USB flash drives, mobile hard disks, magnetic disks, or optical discs that can store program codes.

[0092] The above are only exemplary embodiments of the present disclosure, and the scope of the present disclosure cannot be limited thereby. That is, all equivalent changes and modifications made in accordance with the teachings of the present disclosure still fall within the scope covered by the present disclosure. Those skilled in the art will easily think of other implementation schemes of the present disclosure after considering the specification and the disclosure of the practical truth. The present application aims to cover any variations, uses, or adaptive changes of the present disclosure, and these variations, uses, or adaptive changes follow the general principles of the present disclosure and include the common general knowledge or conventional technical means in the technical field not recorded in the present disclosure. The specification and the embodiments are only regarded as exemplary, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A data governance method based on data lineage, characterized in that: The method comprises: Obtain metadata of target data corresponding to target governance requirements; Constructing a data lineage graph of the target data according to the metadata; The data quality propagation score, data lineage path complexity, data global impact score, data leakage risk score and data usage efficiency score of the target data are calculated through the data lineage graph; Generate a comprehensive data governance score based on the data quality propagation score, the data lineage path complexity, the data global impact score, the data leakage risk score, and the data usage efficiency score; According to the comprehensive data governance score, data governance is performed on the target data.

2. The data management method based on data lineage according to claim 1 is characterized in that: The step of constructing a data lineage graph of the target data according to the metadata specifically includes: Parsing the metadata to obtain a data entity, wherein the data entity includes a database table, a field, and a file; Extracting edges according to data entities having dependency relationships among the plurality of data entities; Acquire additional data between the data entity and the edge; Determining attributes according to the additional data; Based on the data entities, the edges and the attributes, a data lineage graph of the target data is constructed.

3. The data management method based on data lineage according to claim 1 is characterized in that: The data quality propagation score, data lineage path complexity, data global impact score, data leakage risk score and data usage efficiency score of the target data are calculated through the data lineage graph. The data quality propagation score is specifically calculated using the following formula: ； Among them, Q is the data quality propagation score, i is the i-th metadata, q i is the quality score of the i-th metadata, n is the total number of metadata, e is the edge, e∈E represents the edge set in path e, and d(e) is the quality decay rate of path edge e.

4. The data management method based on data lineage according to claim 1 is characterized in that: The data quality propagation score, data lineage path complexity, data global impact score, data leakage risk score and data usage efficiency score of the target data are calculated through the data lineage graph. The data lineage path complexity is specifically calculated using the following formula: ； Among them, L is the data lineage path complexity, t is the time, e is the edge, e∈E represents the edge set in path e, w(e, t) is the weight of edge e at time t, and f(v) is the complexity function associated with node v.

5. The data management method based on data lineage according to claim 1 is characterized in that: The data quality propagation score, data lineage path complexity, data global impact score, data leakage risk score and data usage efficiency score of the target data are calculated through the data lineage graph. The data global impact score is specifically calculated using the following formula: ； Among them, I is the global impact score of the data, V d is the set of directly affected nodes, α v is the weight coefficient of node v, σ(v) is the sensitivity score of node v, K is the maximum level of propagation, k is the kth level, β k is the weight of level k, I k is the influence score of level k.

6. The data management method based on data lineage according to claim 1 is characterized in that: The data quality propagation score, data lineage path complexity, data global impact score, data leakage risk score and data usage efficiency score of the target data are calculated through the data lineage graph. The data leakage risk score is specifically calculated using the following formula: ； Among them, R is the data leakage risk score, v is the node, V s is the set of nodes containing sensitive data, π(v) is the probability that node v is accessed, and r(v) is the leakage risk score of node v.

7. The data management method based on data lineage according to claim 1 is characterized in that: The data quality propagation score, data lineage path complexity, data global impact score, data leakage risk score and data usage efficiency score of the target data are calculated through the data lineage graph. The data usage efficiency score is specifically calculated using the following formula: ； Among them, M is the data usage efficiency score, T is the maximum time, t is the time, U(t) is the number of data accesses at time t, and C(t) is the data maintenance cost at time t.

8. The data management method based on data lineage according to claim 1 is characterized in that: The data governance comprehensive score is generated based on the data quality propagation score, the data lineage path complexity, the data global impact score, the data leakage risk score, and the data usage efficiency score, specifically including: Determining a ratio between the data quality propagation score and the data lineage path complexity; Calculating a first product result between the ratio and a first weight; Calculating a second product result between the data global impact score and a second weight; Calculating a third product result between the data leakage risk score and a third weight; Calculating a fourth product result between the data usage efficiency score and a fourth weight; The first product result, the second product result and the fourth product result are summed, and the third product result is subtracted to obtain the comprehensive data governance score.

9. A data management device based on data lineage, characterized in that: The data management device comprises an acquisition module (31) and a processing module (32), wherein: The acquisition module (31) is used to acquire metadata of target data corresponding to target governance requirements; The processing module (32) is used to construct a data lineage graph of the target data according to the metadata; The processing module (32) is further used to calculate the data quality propagation score, data lineage path complexity, data global impact score, data leakage risk score and data usage efficiency score of the target data through the data lineage graph; The processing module (32) is further used to generate a comprehensive data governance score based on the data quality propagation score, the data lineage path complexity, the data global impact score, the data leakage risk score and the data usage efficiency score; The processing module (32) is also used to perform data governance on the target data according to the data governance comprehensive score.

10. An electronic device, characterized in that: The electronic device comprises a processor (41), a memory (45), a user interface (43) and a network interface (44), wherein the memory (45) is used to store instructions, the user interface (43) and the network interface (44) are both used to communicate with other devices, and the processor (41) is used to execute the instructions stored in the memory (45) so that the electronic device executes the method according to any one of claims 1 to 8.

Images