Data governance method and apparatus, electronic device, and storage medium

Abstract

The present disclosure relates to a data governance method and device, electronic equipment and storage medium. The method comprises: obtaining metadata and quality evaluation results of each data set in a plurality of data sets, the metadata of each data set comprising data source information; determining the to-be-governed information of each data set according to the metadata and the quality evaluation results of each data set; generating data output links of the plurality of data sets according to the data source information of the plurality of data sets, the data output links being links between data sets having data blood relationship; determining link governance information of the data output links according to the quality evaluation results and the to-be-governed information of each data set, the link governance information of the data output links comprising overall quality evaluation results of the data output links and the to-be-governed information of each data set in the data output links. The present disclosure can improve the data governance efficiency.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, and in particular to a data governance method, apparatus, electronic device, and storage medium. Background Technology

[0002] In data development and production processes, data governance is a common scenario. Data governance is a set of activities (such as planning, monitoring, and execution) that exercise power and control over data asset management. Data governance functions guide other data management functions in their execution, ultimately ensuring data availability, data quality, and data security. Based on data governance, it is possible to identify data models with quality problems, redundant data models, and low-value, high-energy-consuming data computation tasks, thereby improving data quality and reducing costs related to data development and operation.

[0003] In related technologies, data governance is performed at the table level, providing overview information, field information, data quality information, and governance optimization suggestions. However, because only information from a single table can be viewed, troubleshooting data problems is inefficient and cumbersome. Furthermore, attributing the causes of data governance problems is difficult, resulting in slow progress and low efficiency in data governance. Summary of the Invention

[0004] This disclosure provides a data governance method, apparatus, electronic device, and storage medium to at least address the problem of low data governance efficiency in related technologies. The technical solution of this disclosure is as follows:

[0005] According to a first aspect of the present disclosure, a data governance method is provided, comprising:

[0006] Obtain metadata and quality assessment results for each of multiple datasets, wherein the metadata for each dataset includes data source information;

[0007] Based on the metadata of each data set and the quality assessment results, determine the information to be governed for each data set;

[0008] Based on the data lineage information of multiple data sets, a data output link is generated for multiple data sets, wherein the data output link is a link between data sets with data dependencies.

[0009] Based on the quality assessment results and governance information corresponding to each of the multiple data sets, the link governance information of the data output link is determined. The link governance information of the data output link includes the overall quality assessment result of the data output link and the governance information of each data set in the data output link.

[0010] Optionally, the data output link is at least one, and the method further includes:

[0011] Construct a mapping relationship between the link identifier of the data output link and the link governance information;

[0012] After determining the link governance information of the data output link based on the quality assessment results and governance information corresponding to each of the multiple data sets, the method further includes:

[0013] Based on the information viewing request of the target link identifier, obtain the target link governance information corresponding to the target link identifier.

[0014] Optionally, the target link governance information includes target link overview information and target link details information;

[0015] After obtaining the target link governance information corresponding to the target link identifier, the method further includes:

[0016] Generate a first display interface corresponding to the target link overview information, wherein the target link overview information includes the overall quality assessment result of the target data output link;

[0017] A second display interface corresponding to the target link details information is generated. The target link details information includes the governance information of each data set in the target data output link and the quality assessment results of each data set.

[0018] Optionally, after obtaining the target link governance information corresponding to the target data output link, the method further includes:

[0019] After obtaining the target link details information corresponding to the target data production link, a display list corresponding to the hierarchical data dependency relationship between the data sets in the target data production link is generated. The display list includes the governance information of each data set in the target data production link.

[0020] Optionally, after generating the display list corresponding to the data dependencies of the hierarchy, the method further includes:

[0021] Based on the request to view similar sets of the target data set, obtain the similar data set corresponding to the target data set.

[0022] Optionally, after obtaining the target link governance information corresponding to the target data output link, the method further includes:

[0023] After obtaining the target link details information corresponding to the target data production link, a display graph corresponding to the hierarchical data dependency relationship between data sets in the target data production link is generated. The display graph includes the governance information of each data set in the target data production link.

[0024] Optionally, after generating the display graph corresponding to the hierarchical dependency relationship, the method further includes:

[0025] Based on the request to obtain the graphics of the similar set corresponding to the target data set, obtain the similar data set corresponding to the target data set.

[0026] Optionally, after obtaining the target link governance information corresponding to the target data output link, the method further includes:

[0027] Based on the data governance request of the target data set, data governance operations are performed on the target data set according to the information to be governed in the target data set.

[0028] Optionally, after obtaining the target link governance information corresponding to the target data output link, the method further includes:

[0029] Based on the detailed viewing request of the target data set, obtain the detailed information of the target data set and the collection information of the downstream data set.

[0030] Optionally, before obtaining the target link governance information corresponding to the target link identifier based on the information viewing request of the target link identifier, the method further includes:

[0031] Determine the platform identifier of the business platform corresponding to each data output link;

[0032] Construct the correspondence between the link identifier and the platform identifier of the data output chain to obtain the business identifier association relationship;

[0033] Based on the data viewing request of the target business platform, determine the target platform identifier of the target business platform;

[0034] Based on the business identifier association relationship, obtain the filtering link identifier that matches the target platform identifier;

[0035] Based on the filtered link identifier, the target link identifier is determined.

[0036] Optionally, determining the link governance information of the data output link based on the quality assessment results and governance information corresponding to each of the multiple data sets includes:

[0037] Based on the quality assessment results corresponding to each of the multiple data sets, the overall quality assessment result of the data output link is determined, and a link overview information including the overall quality assessment result is generated.

[0038] Based on the quality assessment results and governance information corresponding to each of the multiple data sets, link details information of the data output link is generated;

[0039] The link overview information and the link details information are determined as the link governance information of the data output link.

[0040] Optionally, determining the governance information for each data set based on its metadata and the quality assessment results includes:

[0041] For each of the data sets, the metadata and quality assessment results are inspected according to the governance project detection rules to determine the information to be governed for the data set.

[0042] According to a second aspect of the present disclosure, a data governance apparatus is provided, comprising:

[0043] The acquisition module is configured to acquire metadata and quality assessment results for each of a plurality of data sets, wherein the metadata for each data set includes data source information;

[0044] The governance information determination module is configured to determine the governance information of each data set based on the metadata of each data set and the quality assessment results;

[0045] The data link generation module is configured to generate data output links for multiple data sets based on data source information of multiple data sets, wherein the data output links are links between data sets with data dependencies.

[0046] The link governance information determination module is configured to determine the link governance information of the data output link based on the quality assessment results and governance information corresponding to each of the multiple data sets. The link governance information of the data output link includes the overall quality assessment result of the data output link and the governance information of each data set in the data output link.

[0047] Optionally, the data output link is at least one, and the device further includes:

[0048] The mapping relationship construction module is configured to execute the construction of a mapping relationship between the link identifier and the link governance information of the data output link;

[0049] The target link information acquisition module is configured to execute an information viewing request based on the target link identifier and acquire the target link governance information corresponding to the target link identifier.

[0050] Optionally, the target link governance information includes target link overview information and target link details information;

[0051] The device further includes:

[0052] The first interface generation module is configured to generate a first display interface corresponding to the target link overview information, wherein the target link overview information includes the overall quality assessment result of the target data output link;

[0053] The second interface generation module is configured to generate a second display interface corresponding to the target link details information. The target link details information includes the governance information of each data set in the target data output link and the quality assessment results of each data set.

[0054] Optionally, the device further includes:

[0055] The display list generation module is configured to, after obtaining the target link details information corresponding to the target data production link, generate a display list corresponding to the hierarchical data dependency relationship between the data sets in the target data production link. The display list includes the governance information of each data set in the target data production link.

[0056] Optionally, the device further includes:

[0057] The first similarity set acquisition module is configured to execute a similarity set viewing request based on the target data set and acquire the similar data set corresponding to the target data set.

[0058] Optionally, the device further includes:

[0059] The display graph generation module is configured to, after obtaining the target link details information corresponding to the target data production link, generate a display graph corresponding to the hierarchical data dependency relationship between the data sets in the target data production link. The display graph includes the governance information of each data set in the target data production link.

[0060] Optionally, the device further includes:

[0061] The second similarity set acquisition module is configured to execute an acquisition request for displaying a graphic based on the similarity set corresponding to the target data set, and to acquire the similar data set corresponding to the target data set.

[0062] Optionally, the device further includes:

[0063] The data governance module is configured to execute data governance requests based on a target data set, and to perform data governance operations on the target data set according to the information to be governed in the target data set.

[0064] Optionally, the device further includes:

[0065] The details display module is configured to execute a details viewing request based on a target data set, and obtain the details information of the target data set and the set information of the downstream data set.

[0066] Optionally, the device further includes:

[0067] The platform identifier determination module is configured to determine the platform identifier of the business platform corresponding to each data output link;

[0068] The association construction module is configured to perform the construction of the correspondence between the link identifier and the platform identifier of the data output link, and obtain the business identifier association relationship;

[0069] The target platform identifier determination module is configured to execute a data viewing request based on the target business platform to determine the target platform identifier of the target business platform;

[0070] The filter link identifier acquisition module is configured to acquire the filter link identifier that matches the target platform identifier based on the business identifier association relationship;

[0071] The target link identifier determination module is configured to determine the target link identifier based on the filtered link identifiers.

[0072] Optionally, the link governance information determination module includes:

[0073] The overview information generation unit is configured to perform an overall quality assessment of the data output link based on the quality assessment results corresponding to the multiple data sets, and generate an overview of the link including the overall quality assessment results.

[0074] The details information generation unit is configured to generate link details information of the data output link based on the quality assessment results and governance information corresponding to each of the multiple data sets;

[0075] The link governance information determination unit is configured to perform link governance information that determines the link overview information and the link details information as the data output link.

[0076] Optionally, the module for determining the information to be governed is configured to execute:

[0077] For each of the data sets, the metadata and quality assessment results are inspected according to the governance project detection rules to determine the information to be governed for the data set.

[0078] According to a third aspect of the present disclosure, an electronic device is provided, comprising:

[0079] processor;

[0080] Memory used to store the processor's executable instructions;

[0081] The processor is configured to execute the instructions to implement the data governance method as described in the first aspect.

[0082] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided that, when instructions in the computer storage medium are executed by a processor of an electronic device, enables the electronic device to perform the data governance method as described in the first aspect.

[0083] According to a fifth aspect of the present disclosure, a computer program product is provided, including a computer program or computer instructions, which, when executed by a processor, implement the data governance method as described in the first aspect.

[0084] The technical solutions provided by the embodiments of this disclosure bring at least the following beneficial effects:

[0085] This embodiment of the disclosure determines the governance information of each data set based on its metadata and quality assessment results. It generates data production links for multiple data sets based on their data source information. Based on the quality assessment results and governance information corresponding to each data set, it determines the link governance information for each data production link. By combining the data quality assessment results and governance information with the data production links, it provides link analysis capabilities, revealing quality details of intermediate data production stages in the form of links. This allows for rapid location, analysis, and evaluation of data value, thereby quickly identifying data sets to be governed and their problems, thus improving data governance efficiency.

[0086] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0087] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0088] Figure 1 This is a flowchart illustrating a data governance method according to an exemplary embodiment;

[0089] Figure 2 This is a flowchart illustrating a data governance method according to an exemplary embodiment;

[0090] Figure 3 This is a schematic diagram illustrating the display of target link details in the form of a list in this embodiment of the disclosure;

[0091] Figure 4 This is a schematic diagram showing the target link details in the form of a graphic in this embodiment of the disclosure;

[0092] Figure 5 This is an architecture diagram of a data governance platform illustrated according to an exemplary embodiment;

[0093] Figure 6 This is a block diagram illustrating a data governance apparatus according to an exemplary embodiment;

[0094] Figure 7 This is a block diagram illustrating an electronic device according to an exemplary embodiment. Detailed Implementation

[0095] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0096] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0097] Among related technologies, platform tools for data governance include "data maps" and "data observatories." "Data maps" provide a one-stop solution for data developers to find the data they need, offering information such as data fields, lifecycles, permissions, and business domains. This helps developers pinpoint the data's flow, output status, partitioning status, and asset level. "Data observatories" provide information on data quality health scores, data quality issue classifications, and data governance optimization suggestions at different levels, including data table owners, teams, and departments. They also display the progress made on projects requiring governance and those already governed.

[0098] However, "data maps" only describe the overview, field information, and path lineage information of data tables at the table level, failing to provide data quality health information or effectively organize and display data table dependencies. Data governance personnel can only examine a single data table when troubleshooting, leading to inefficient troubleshooting, cumbersome operations, and difficulties in model quality assessment. "Data observatories" merely list the quality indicators, problems, and results of data tables. Because they assess the structure and calculations of data tables in isolation, they struggle to meet the personalized assessment requirements of data business stakeholders, resulting in unconvincing indicators. Furthermore, since data development inherently exists as a processing chain, viewing and governing data tables in isolation makes it difficult to pinpoint the key points of problems or quickly find key nodes and methods for optimization. This makes quality indicators easily questionable, attributing data governance problems difficult, slow governance progress, and minimal governance effectiveness. Therefore, related technologies all perform data governance at the table level, leading to difficulties in attributing data governance problems, slow data governance progress, and low data governance efficiency.

[0099] In order to solve the above-mentioned technical problems, the present disclosure provides the following technical solutions.

[0100] Figure 1 This is a flowchart illustrating a data governance method according to an exemplary embodiment, such as... Figure 1 As shown, this data governance method is used in electronic devices such as computers or servers and includes the following steps.

[0101] In step S11, the metadata and quality assessment results of each of the multiple data sets are obtained, and the metadata of each data set includes data source information.

[0102] The data set is a collection of stored data, including at least one of data tables, datasets, and data dashboards. Metadata describes the data, mainly information describing data attributes, used to support functions such as indicating storage location, historical data, resource lookup, and file records. Data source information of the data set indicates the origin of the data set; data source information can also be called data lineage information. Data lineage refers to the origin and development of data, mainly including the source of the data, the processing method of the data, mapping relationships, and data exit points.

[0103] It can obtain the metadata corresponding to each data set. The metadata includes the data source information of the corresponding data set, which represents the data set or data model that the data set depends on.

[0104] In this embodiment, metadata may include multiple categories, such as personnel organization, data production, data assets, data quality, data standards, data resources, data services, and data security. Personnel organization includes business processes such as adding or changing data users, adding or changing data developers, adding or changing roles, and adding or changing organizations; data production includes business processes such as task creation, task review, task lineage, generating task instances, and scheduling; data assets include business processes such as configuring and modifying data tables, fields, products, and datasets; data quality includes business processes such as configuring data quality inspection platforms, baseline configuration, data quality inspection platform instances, and baseline instances; data standards include business processes such as defining business rules, model specifications, and security rules; data resources include business processes such as data computation and data storage; data services include business processes such as accessing data tables, datasets, and products; and data security includes business processes such as creating permissions, granting permissions, handling unauthorized access, and auditing logs. The data tables may be based on Hive, ClickHouse, or Druid.

[0105] The quality assessment results are obtained by evaluating the data quality of each dataset based on the data quality assessment system. The core of the data quality assessment system is the production of a quality health score, which is achieved as follows: Multiple quality standard rules are established based on the potential causes of quality problems at different stages of data production; then, based on the aforementioned collected metadata, each rule generates multiple positive or negative health indicators; these health indicators are normalized, aggregated, and weighted, ultimately resulting in a unified overall quality health score. The quality health score model is constructed as follows: Model dimensions may include business, project team, table owner, model table, etc.; indicators are health scores (e.g., 0 to 100 points); quality rule classifications may include unreasonable dataset (or model) design, substandard data quality inspection and output timeliness, unreasonable data calculation and storage, and incomplete metadata or unreasonable usage frequency of data services.

[0106] In step S12, the information to be governed for each data set is determined based on the metadata of each data set and the quality assessment results.

[0107] Based on pre-defined detection rules corresponding to governance projects, each dataset can be inspected according to its metadata and quality assessment results to determine the governance information for each dataset. This governance information can include the projects to be governed and the corresponding governance methods.

[0108] In an exemplary embodiment, determining the governance information for each data set based on the metadata of each data set and the quality assessment result includes: for each data set, detecting the metadata and quality assessment result according to the governance project detection rules to determine the governance information for the data table.

[0109] Among them, the detection rules for each governance project are the detection rules that are set in advance for each governance project. A governance project is a project to be governed in advance, and one or more governance projects can be set in advance.

[0110] For each dataset, the corresponding metadata and quality assessment results can be inspected according to the governance project's detection rules to determine the information to be governed for that dataset. By identifying the information to be governed for each dataset at the dataset level, a data foundation is provided for the analysis of the overall data production chain.

[0111] In step S13, based on the data source information of the multiple data sets, a data output link for the multiple data sets is generated. The data output link is a link between data sets with data dependencies.

[0112] Based on the data source information of multiple datasets, the data dependencies between them are determined. These datasets with dependencies are then linked together to generate a data output chain. Each data output chain can include multiple chain nodes, each corresponding to a dataset. The final chain node is designated as a leaf node, and its annotation information can be used to determine the chain name and other chain identifiers. The annotation information of a leaf node can include the dataset name or the attributes of the stored data.

[0113] In step S14, the link governance information of the data output link is determined based on the quality assessment results and governance information corresponding to each of the multiple data sets. The link governance information of the data output link includes the overall quality assessment results of the data output link and the governance information of each data set in the data output link.

[0114] By linking the quality assessment results and governance information of each data set within the data production chain, the chain governance information of the data production chain is obtained. This chain governance information can include the overall quality assessment result of the data production chain, the governance information for each data set, and the quality assessment results and governance information of the upstream data sets that each data set depends on. Within the chain governance information of the data production chain, for each data set, the overall governance information for that data set can be derived by combining the quality assessment results and governance information of its upstream data sets.

[0115] In an exemplary embodiment, determining the link governance information of the data output link based on the quality assessment results and governance information corresponding to each of the plurality of data sets includes: determining the overall quality assessment result of the data output link based on the quality assessment results corresponding to each of the plurality of data sets, and generating link overview information including the overall quality assessment result; generating link detail information of the data output link based on the quality assessment results and governance information corresponding to each of the plurality of data sets; and determining the link overview information and the link detail information as the link governance information of the data output link.

[0116] The overall quality assessment result of the data output link is determined by statistically analyzing the quality assessment results of each data set in the data output link. The overall quality assessment result may include the overall health score of the link, the maximum health score of the link node (i.e., the maximum health score among all data sets in the data output link), the minimum health score of the link node (i.e., the minimum health score among all data sets in the data output link), the cumulative calculation time of the link, and the link end time (the end time of data output for each data set in the data output link).

[0117] For a given data production chain, the link details information of the data production chain is generated by combining the quality assessment results and governance information of each data set in the data production chain. That is, based on the data dependencies between the data sets in the data production chain, the quality assessment results and governance information of each data set are concatenated to obtain the link details information of the data production chain.

[0118] After obtaining the overall link information and link details of the data output link, the overall link information and link details are determined as the link governance information of the data output link.

[0119] The data output chain governance information includes an overview of the chain and detailed information about the chain, making it easy for users to view the overall situation and details of the data output chain.

[0120] In this embodiment of the disclosure, the link governance information of the data output link can be determined periodically based on a timed scheduling task.

[0121] The data governance method provided in this exemplary embodiment determines the governance information of each data set based on its metadata and quality assessment results, generates data production links for multiple data sets based on their data source information, and determines the link governance information of the data production links based on their respective quality assessment results and governance information. By combining the data quality assessment results and governance information with the data production links, it can provide link analysis capabilities, revealing the quality details of intermediate data production links in the form of links. This allows for rapid location, analysis, and evaluation of data value, thereby quickly identifying the data sets to be governed and the existing problems, and improving data governance efficiency.

[0122] Figure 2 This is a flowchart illustrating a data governance method according to an exemplary embodiment, such as... Figure 2 As shown, this data governance method is used in electronic devices such as computers or servers and includes the following steps.

[0123] In step S21, the metadata and quality assessment results of each data set in the multiple data sets are obtained, and the metadata of each data set includes data source information.

[0124] In step S22, the information to be governed for each data set is determined based on the metadata of each data set and the quality assessment results.

[0125] In step S23, based on the data source information of the multiple data sets, a data output link for the multiple data sets is generated. The data output link is a link between data sets with data dependencies.

[0126] In step S24, the link governance information of the data output link is determined based on the quality assessment results and governance information corresponding to each of the multiple data sets.

[0127] In step S25, a mapping relationship is established between the link identifier of the data output link and the link governance information.

[0128] The data output link is at least one.

[0129] One or more data output chains can be generated based on multiple data sets. The data sets in each data output chain do not exist in other data output chains; that is, one data set corresponds to one data output chain, and each data output chain includes multiple data sets.

[0130] For each data output link, a link identifier is generated for that data output link, and a mapping relationship is constructed between the link identifier and the link governance information. Then, based on this mapping relationship, the link governance information corresponding to the link identifier is stored in the database.

[0131] In step S26, based on the information viewing request of the target link identifier, the target link governance information corresponding to the target link identifier is obtained.

[0132] The target link governance information may include target link overview information and target link details information; the target link overview information includes the overall quality assessment results of the target data output link; and the target link details information includes the governance information of each data set in the target data output link.

[0133] The system allows for fuzzy searches based on target link names or other identifiers entered by the user. It can also query one or more data production links based on the target data production unit (e.g., department or project team) queried by the user, and can display one or more data production links on the client side. The system determines the target link identifier based on the user's selection and confirms receipt of requests to view information related to that identifier. When displaying one or more data production links, they can be sorted based on their health scores, and then displayed in a list based on this sorting. For each data production link, the system can also display information such as the cumulative number of projects awaiting remediation, the amount of computation that can be optimized, and the amount of storage that can be optimized.

[0134] When a user requests to view information about a target link identifier, the system retrieves the target link governance information corresponding to the target link identifier from the database, generates a display interface for the target link governance information, and then displays the interface through the client.

[0135] In one exemplary embodiment, the target link governance information includes target link overview information and target link details information;

[0136] After obtaining the target link governance information corresponding to the target link identifier, the method further includes: generating a first display interface corresponding to the target link overview information, wherein the target link overview information includes the overall quality assessment result of the target data output link; generating a second display interface corresponding to the target link details information, wherein the target link details information includes the governance information of each data set in the target data output link and the quality assessment result of each data set.

[0137] The target link governance information includes target link overview information and target link detail information. The target link overview information represents the overall summary information of the target data output link, while the target link detail information represents the detailed information of the target data output link, including the governance information for each data set in the target data output link. After obtaining the target link governance information corresponding to the target link identifier, a first display interface corresponding to the target link overview information and a second display interface corresponding to the target link detail information are generated. That is, the target link overview information of the target data output link can be displayed in the first display interface on the client, and the target link detail information can be displayed in the second display interface on the client. The overall target quality assessment result may include information such as the overall link health score, the maximum health score of a link node, the minimum health score of a link node, the link depth, the number of link nodes, the link termination time, and the cumulative calculation time of the link. The link depth is the number of layers of data sets included in the target data output link, and the number of link nodes is the number of data sets included in the link. The link details include the governance information and quality assessment results for each data set in the target data output link. The governance information for each data set can include the governance information for that data set and the governance information for the upstream data sets that the data set depends on, which can help users quickly find the problem and improve data governance efficiency.

[0138] It should be noted that the first display interface and the second display interface can be two different display interfaces, or they can be two different display areas within the same display interface.

[0139] The scenarios in which this disclosure can be used include the following: when the initiator of data governance needs to view and understand the quality and health status of production data from different departments in order to plan the department's data governance strategy; when data developers need to understand the production and processing paths of existing models in the daily data model development process in order to build suitable new models or update old models; when data developers need to adjust the existing data model structure when they receive a "cost reduction and efficiency improvement" task notification from management; and when employees transfer or leave, or when new employees undergo onboarding training and work surveys, it is necessary to analyze and understand the bloodline dependencies of existing models.

[0140] The data governance method provided in this exemplary embodiment constructs a mapping relationship between the link identifier and the link governance information of the data production link after determining the link governance information of the data production link. Based on the information viewing request of the target link identifier, the target link governance information corresponding to the target link identifier is obtained. Then, the target link governance information corresponding to the target link identifier can be displayed on the client. Based on the displayed information, problems in the entire link can be quickly discovered. This provides a global link perspective for the location, analysis, attribution, and discovery of problem solutions in the data governance process. It breaks the problem of the isolation between the data quality indicator system and the data production link, prevents the fragmentation of governance actions, connects the data governance actions required in different links of data production, improves the data governance process, and can improve the efficiency of data governance.

[0141] In an exemplary embodiment, after obtaining the target link governance information corresponding to the target data production link, the method further includes: after obtaining the target link details information corresponding to the target data production link, generating a display list corresponding to the hierarchical data dependency relationship between data sets in the target data production link, the display list including the governance information of each data set in the target data production link.

[0142] The data dependencies between data sets in the target data production chain are hierarchical. For example, data set A depends on data sets B and C, data set B depends on data sets D and E, and data set C depends on data set F. Thus, data set A can be considered a first-level set, data sets B and C are second-level sets, and data sets D, E, and F are third-level sets. When displaying the chain details corresponding to the target data production chain on the client side, it is necessary to display each data set under that chain and its pending management information. This can be achieved by generating a display list corresponding to the hierarchical data dependencies between data sets in the target data production chain, and then displaying this list on the client side to present the pending management information of each data set in the target data production chain in list form. Using the above data dependency relationship as an example... Figure 3 This is a schematic diagram illustrating the display of target link details in the form of a list in this embodiment of the disclosure, such as... Figure 3 As shown, data set A depends on data sets B and C, data set B depends on data sets D and E, and data set C depends on data set F. The governance information for each data set can include the number of issues and details of governance projects. Governance project details may include information such as unreasonable lifecycle settings or excessive computation time.

[0143] After obtaining the target link details corresponding to the target data production link, a hierarchical data dependency list is generated based on the hierarchical lineage dependency relationship between data sets in the target data production link. This allows the client to display the governance information of each data set in the target data production link in a list format. It can clearly show the other data sets that the data set depends on, making it easier to quickly discover problems in the entire link and helping to improve data governance efficiency.

[0144] Based on the above technical solution, after generating the display list corresponding to the hierarchical data dependency relationship, the method further includes: displaying the similar data set corresponding to the target data set according to the similar set viewing request of the target data set.

[0145] When displaying the governance information of each data set in the target data production chain in the form of a list through the client, operation options can also be displayed for each data set. The operation options can include the option to view similar sets. When a user triggers the operation of viewing similar sets corresponding to the target data set, it is determined that a request to view similar sets of the target data set has been received. At this time, the similar data sets corresponding to the target data set can be obtained. That is, if the similar data sets are in the same target data production chain as the target data set, the similar data sets can be displayed in the form of selection. If the target data production chain does not include similar data sets, the client can jump to the display page of the similar data sets to display the governance information of the similar data sets.

[0146] By requesting similar data sets based on the target data set, the system obtains the corresponding similar data sets. These similar data sets can then be displayed on the client side, making it easier for users to compare the target data set with the similar data sets, thereby improving the efficiency of data governance.

[0147] In another exemplary embodiment, after obtaining the target link governance information corresponding to the target data production link, the method further includes: after obtaining the target link details information corresponding to the target data production link, generating a display graph corresponding to the hierarchical dependency relationship between data sets in the target data production link, wherein the display graph includes the governance information of each data set in the target data production link.

[0148] After obtaining the target link details corresponding to the target data output link, it is also possible to generate a display image corresponding to the hierarchical dependency relationship between data sets in the target data output link. This allows the client to display the governance information of each data set in the target data output link in the form of a graphical representation. Figure 4 This is a schematic diagram illustrating the target link details in graphical form in this embodiment of the disclosure. Figure 4 Taking the data dependencies of the aforementioned datasets A, B, C, D, E, and F as an example, as follows: Figure 4 As shown, the graphical display clearly shows the other data sets that each data set depends on, and displays the corresponding governance problem graph for that data set. Triggering actions on the governance problem graph reveals specific governance problem details (such as storage problems, computational problems, the existence of similar models, etc.). When displaying target link details in graphical form, a health status graph can also be displayed for each data set, showing its health level (i.e., quality assessment results). Problem category labels for the data set can also be displayed; when the cursor hovers over a label, the problem category level (e.g., level 2 or level 3) floats, and specific problem content is displayed based on triggering actions on the label (e.g., right-click triggering). Node type graphs can also be displayed for each data set, with each node type corresponding to a specific graph. Node types can include Hive, Druid, ClickHouse, Kafka, etc. A popularity graph can also be displayed for each data set, representing its popularity; the popularity graph can use color or watermarks to distinguish popularity levels.

[0149] After obtaining the link details corresponding to the target data production link, a display image corresponding to the hierarchical data dependency relationship is generated based on the hierarchical lineage dependency relationship between data sets in the target data production link. This allows the client to display the governance information of each data set in the target data production link in a graphical form, clearly showing the other data sets that the data set depends on, facilitating the rapid discovery of problems in the entire link, and helping to improve data governance efficiency.

[0150] Based on the above technical solution, after generating the display graphics corresponding to the hierarchical dependency relationship, the method further includes: obtaining the similar data set corresponding to the target data set according to the request for obtaining the display graphics of the similar set corresponding to the target data set.

[0151] When displaying the governance information of each data set in the target data production chain in graphical form through the client, a governance problem graph can also be displayed for each data set. Different governance problem graphs are displayed for different governance problems, such as computational problems, storage problems, and the existence of similar sets. For data sets with investigation reports, an investigation report graph can also be displayed, and the investigation report for that data set is displayed based on a trigger operation on the investigation report graph. When the client receives a user's request to obtain (trigger) a graph displaying similar sets corresponding to the target data set in the target data production chain, the similar data sets corresponding to the target data set are obtained. If the similar data sets are in the same target data production chain as the target data set, the client's display interface will jump from the target data set to the location of the corresponding similar data set, which can be displayed by selecting or highlighting the similar data set.

[0152] By requesting the acquisition of a graph based on the similar data set corresponding to the target data set, the similar data set corresponding to the target data set can be obtained. Then, the similar data set can be displayed through the client. That is, the client's display interface will jump to the location of the similar data set corresponding to the target data set, which makes it convenient for users to compare the target data set with the similar data set, thereby improving the efficiency of data governance.

[0153] When displaying link details on the client side, the display can switch between list and graphical formats. The link details display interface can show both list and graphical operation buttons. Based on the triggering of one of these buttons, the corresponding display format is used to show the link details. The link details display interface (including list and graphical formats) also provides a function to view downstream data sets. If the data set is a table-based calculation model, it displays the distribution of its downstream models using a tree-like model diagram, automatically categorizing and aggregating models according to their business and displaying them in order of popularity. The link details display interface (including list and graphical formats) can also provide a function to notify the node responsible person for each data set. When the user triggers the option to notify the node responsible person, a notification message is sent to the node responsible person of the target data set to notify them to repair the data set. The link details display interface (including list and graphical formats) also provides an input box for fuzzy querying of the data set in the current data output link, i.e., locating the node, or querying the input... The system displays detailed information about other data output links for the node name. In the link details display interface (including list and graphical formats), due to the large number of link nodes, it also provides filtering options to select link nodes that meet the filtering criteria. This allows users to quickly view the link nodes they are interested in, such as filtering temporary tables, intermediate layer calculation tables, Druid, data synchronization tasks, dataset nodes, filtering storage governance nodes, filtering computation governance nodes, filtering inactive nodes, filtering nodes whose output time exceeds a preset time (e.g., 2 hours), filtering data table owners, filtering specific departments, filtering baseline nodes, and so on. The link details display interface (including list and graphical formats) also provides one-click message group creation, message template notifications, one-click exploration of selected and all nodes, and one-click viewing of the distribution of nodes by department.

[0154] Based on the above technical solution, after obtaining the target link governance information corresponding to the target data output link, the method further includes: a data governance request based on the target data set, and performing data governance operations on the target data set according to the information to be governed in the target data set.

[0155] When displaying the link details of the target data output chain in a list format via the client, operation options can also be displayed for each data set. These operation options may include data governance options. Upon receiving a user's trigger operation on the data governance option corresponding to the target data set, the client sends a data governance request for the target data set to a server or other electronic device that executes the data governance method. Alternatively, when displaying the link details of the target data output chain in a graphical format via the client, the right-click menu may include data governance options. Upon detecting a right-click operation on the target data set, the right-click menu is displayed. Upon detecting a trigger operation on a data governance option within the right-click menu, the client sends a data governance request for the target data set to a server or other electronic device that executes the data governance method. Upon receiving a data governance request for the target data set, the server or other electronic device executing the data governance method can perform data governance operations on the target data set based on the information to be governed. This means performing data governance operations based on the data to be governed items in the information, using the data governance method corresponding to the data to be governed item, such as changing the lifecycle. When electronic devices such as servers that perform data governance methods receive a data governance request, they can also be redirected to a data governance page, which displays detailed information about the data set to be governed. Users can then perform corresponding data governance operations based on the detailed information about the data set.

[0156] By implementing data governance operations on the target data set based on the data governance request and the information to be governed in the target data set, the data governance operations can be performed on the target data set in the client link details information display page.

[0157] Based on the above technical solution, after obtaining the target link governance information corresponding to the target data output link, the method further includes: obtaining the detailed information of the target data set and the set information of the downstream data set based on the detailed viewing request of the target data set.

[0158] The "View Details" request is used to request access to the exploration report of the target dataset. The exploration report includes detailed information about the target dataset and the collection information of downstream datasets. The detailed information about the target dataset may include, for example, detailed information about each project to be addressed.

[0159] When displaying the link details of the target data output chain in a list format via the client, operation options can also be displayed for each data set. These operation options may include a details view option (i.e., a probe option). Upon receiving a user's trigger operation for the details view option corresponding to the target data set, a details view request for the target data set is sent to the server or other electronic device executing the data governance method. Alternatively, when displaying the link details of the target data output chain in a graphical format via the client, a details view option may be included in the right-click menu for each data set. Upon detecting a right-click operation on the target data set, the right-click menu is displayed. Upon detecting a trigger operation for the details view option within the right-click menu, a details view request for the target data set is sent to the server or other electronic device executing the data governance method. Upon receiving a details view request for the target data set, the server or other electronic device executing the data governance method can obtain a probe report of the target data set, i.e., obtain the details of the target data set and the set information of the downstream data sets, and send these information to the client for display.

[0160] By requesting detailed information about the target data set, the system obtains detailed information about the target data set and the downstream data set, and then displays this information on the client side. This allows users to view more detailed information about the target data set, helps them quickly identify problems, and improves data governance efficiency.

[0161] Based on the above technical solution, before obtaining the target link governance information corresponding to the target link identifier based on the information viewing request of the target link identifier, the method further includes: determining the platform identifier of the business platform corresponding to each data output link; constructing the correspondence between the link identifiers of the data output links and the platform identifiers to obtain the business identifier association relationship; determining the target platform identifier of the target business platform according to the data viewing request of the target business platform; obtaining the filter link identifier matching the target platform identifier according to the business identifier association relationship; and determining the target link identifier according to the filter link identifier.

[0162] One business platform corresponds to one data production unit, which can be a department or a project team, for example.

[0163] For each data output link, after the data output link is generated, the platform identifier of the business platform corresponding to the data set in the data output link is determined as the platform identifier corresponding to the data output link. The correspondence between the link identifier of the data output link and the platform identifier is constructed to obtain the business identifier association relationship.

[0164] The client's interface can provide platform identifiers for users to select. After the user selects a target platform identifier, the client sends a data viewing request for the target platform identifier to the server or other electronic devices that perform data governance methods. The server or other electronic devices that perform data governance methods obtain one or more link identifiers that match the target platform identifier based on the business identifier association relationship, obtain one or more filter link identifiers, and send one or more filter link identifiers to the client. The client can display one or more filter link identifiers, or display at least one or more filter link identifiers through a list. It can sort one or more filter link identifiers based on the health score of the data output link and display one or more filter link identifiers in the list based on the sorting. For each filter link identifier, information such as the cumulative number of projects to be governed, the amount of computation that can be optimized, and the amount of storage that can be optimized can also be displayed.

[0165] Each data output link displayed in the list by the client can be operated by the user. When a user triggers an operation on one or more filter link identifiers, the filter link identifier is identified as the target link identifier. The user's request to view the information of the target link identifier is received, and the request to view the information of the target link identifier is sent to the server or other electronic devices that perform data governance methods.

[0166] By constructing a platform identifier for each business platform corresponding to each data output link, and establishing a correspondence between the link identifier and the platform identifier, the business identifier association is obtained. Based on the data viewing request of the target business platform, the target platform identifier of the target business platform is determined. Based on the business identifier association, the filter link identifier matching the target platform identifier is obtained. Based on the filter link identifier, the target link identifier is determined. This provides a way to obtain the platform identifier based on the business platform and display the corresponding filter link identifier through the client. In this way, the link governance information of the target link identifier can be obtained and displayed, making it convenient for users to view the information of the data output link corresponding to the target platform identifier. Moreover, from the displayed one or more filter link identifiers, a request to view the information of the target link identifier can be received to further view the detailed information of the target data output link.

[0167] Figure 5 This is an architecture diagram of a data governance platform illustrated according to an exemplary embodiment, such as... Figure 5As shown, the task flow and other data are stored in the Hive data warehouse, Druid database, Clickhouse database, and Nebula database through the offline task platform. The backend project performs quality assessment based on the stored data set, obtains the quality assessment results, and generates data output links based on the metadata of the data set and the quality assessment results. It can also use the data interfaces (such as metadata interfaces, data detection open interfaces, etc.) provided by the open platform (second-party capabilities or third-party capabilities) to determine and determine the link governance information of each data output link, and store the link governance information of the data output link in the MySQL database. The specific data governance operations are implemented in cooperation with the frontend project.

[0168] This disclosure, through its embodiments, determines the governance information of the entire data output chain based on the data output chain. Compared to related technologies that only provide information on a single data set, this solves the problems of not being able to quickly view the overall quality distribution of the data output chain and assess the health status of specific output chains; it also solves the problem of not being able to quickly locate output delays caused by unreasonable upstream dependency settings; it solves the problem of not being able to quickly locate poor downstream node quality due to the propagation of quality problems in upstream key computing nodes; it solves the problem of not being able to grasp the overall data computing logic and quickly find replacement models when optimizing models; and it solves the problem of not being able to quickly identify and notify the person in charge in the data output chain for rapid model repair.

[0169] Figure 6 This is a block diagram illustrating a data governance apparatus according to an exemplary embodiment. (Refer to...) Figure 6 The device includes an acquisition module 61, a governance information determination module 62, a data link generation module 63, and a link governance information determination module 64.

[0170] The acquisition module 61 is configured to acquire metadata and quality assessment results for each of a plurality of data sets, wherein the metadata for each data set includes data source information;

[0171] The governance information determination module 62 is configured to determine the governance information for each data set based on the metadata of each data set and the quality assessment results.

[0172] The data link generation module 63 is configured to generate data output links for multiple data sets based on data source information of multiple data sets, wherein the data output links are links between data sets with data dependencies.

[0173] The link governance information determination module 64 is configured to determine the link governance information of the data output link based on the quality assessment results and governance information corresponding to each of the multiple data sets. The link governance information of the data output link includes the overall quality assessment results of the data output link and the governance information of each data set in the data output link.

[0174] Optionally, the data output link is at least one, and the device further includes:

[0175] The mapping relationship construction module is configured to execute the construction of a mapping relationship between the link identifier and the link governance information of the data output link;

[0176] The target link information acquisition module is configured to execute an information viewing request based on the target link identifier and acquire the target link governance information corresponding to the target link identifier.

[0177] Optionally, the target link governance information includes target link overview information and target link details information;

[0178] The device further includes:

[0179] The first interface generation module is configured to generate a first display interface corresponding to the target link overview information, wherein the target link overview information includes the overall quality assessment result of the target data output link;

[0180] The second interface generation module is configured to generate a second display interface corresponding to the target link details information. The target link details information includes the governance information of each data set in the target data output link and the quality assessment results of each data set.

[0181] Optionally, the device further includes:

[0182] The display list generation module is configured to, after obtaining the target link details information corresponding to the target data production link, generate a display list corresponding to the hierarchical data dependency relationship between the data sets in the target data production link. The display list includes the governance information of each data set in the target data production link.

[0183] Optionally, the device further includes:

[0184] The first similarity set acquisition module is configured to execute a similarity set viewing request based on the target data set and acquire the similar data set corresponding to the target data set.

[0185] Optionally, the device further includes:

[0186] The display graph generation module is configured to, after obtaining the target link details information corresponding to the target data production link, generate a display graph corresponding to the hierarchical data dependency relationship between the data sets in the target data production link. The display graph includes the governance information of each data set in the target data production link.

[0187] Optionally, the device further includes:

[0188] The second similarity set acquisition module is configured to execute an acquisition request for displaying a graphic based on the similarity set corresponding to the target data set, and to acquire the similar data set corresponding to the target data set.

[0189] Optionally, the device further includes:

[0190] The data governance module is configured to execute data governance requests based on a target data set, and to perform data governance operations on the target data set according to the information to be governed in the target data set.

[0191] Optionally, the device further includes:

[0192] The details display module is configured to execute a details viewing request based on a target data set, and obtain the details information of the target data set and the set information of the downstream data set.

[0193] Optionally, the device further includes:

[0194] The platform identifier determination module is configured to determine the platform identifier of the business platform corresponding to each data output link;

[0195] The association construction module is configured to perform the construction of the correspondence between the link identifier and the platform identifier of the data output link, and obtain the business identifier association relationship;

[0196] The target platform identifier determination module is configured to execute a data viewing request based on the target business platform to determine the target platform identifier of the target business platform;

[0197] The filter link identifier acquisition module is configured to acquire the filter link identifier that matches the target platform identifier based on the business identifier association relationship;

[0198] The target link identifier determination module is configured to determine the target link identifier based on the filtered link identifiers.

[0199] Optionally, the link governance information determination module includes:

[0200] The overview information generation unit is configured to perform an overall quality assessment of the data output link based on the quality assessment results corresponding to the multiple data sets, and generate an overview of the link including the overall quality assessment results.

[0201] The details information generation unit is configured to generate link details information of the data output link based on the quality assessment results and governance information corresponding to each of the multiple data sets;

[0202] The link governance information determination unit is configured to perform link governance information that determines the link overview information and the link details information as the data output link.

[0203] Optionally, the module for determining the information to be governed is configured to execute:

[0204] For each of the data sets, the metadata and quality assessment results are inspected according to the governance project detection rules to determine the information to be governed for the data set.

[0205] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0206] Figure 7 This is a block diagram illustrating an electronic device according to an exemplary embodiment. For example, electronic device 700 may be provided as a server. (Refer to...) Figure 7 The electronic device 700 includes a processing component 722, which further includes one or more processors, and memory resources represented by memory 732 for storing instructions executable by the processing component 722, such as application programs. The application programs stored in memory 732 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 722 is configured to execute instructions to perform the data governance methods described above.

[0207] Electronic device 700 may also include a power supply component 726 configured to perform power management of electronic device 700, a wired or wireless network interface 750 configured to connect electronic device 700 to a network, and an input / output (I / O) interface 758. Electronic device 700 may operate on an operating system stored in memory 732, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, or similar.

[0208] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory 732 including instructions, which can be executed by a processing component 722 of an electronic device 700 to perform the data governance method described above. Optionally, the computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, or optical data storage device, etc.

[0209] In an exemplary embodiment, a computer program product is also provided, including a computer program or computer instructions, which, when executed by a processor, implement the data governance method described above.

[0210] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0211] It should be understood that the present invention is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A data governance method, characterized in that, include: Obtain metadata and quality assessment results for each of multiple datasets, wherein the metadata for each dataset includes data source information; Based on the metadata of each data set and the quality assessment results, determine the information to be governed for each data set; The quality assessment results mentioned above are health score indicators obtained by conducting data quality assessments on each of the data sets based on a data quality assessment system. Based on the data source information of multiple data sets, multiple data output links are generated for the data sets. The data output links are links between data sets with data dependencies, and there is at least one data output link. Based on the quality assessment results and governance information corresponding to each of the multiple data sets, the link governance information of the data output link is determined. The link governance information of the data output link includes the overall quality assessment result of the data output link and the governance information of each data set in the data output link. Construct a mapping relationship between the link identifier of the data output link and the link governance information; Based on the information viewing request of the target link identifier, obtain the target link governance information corresponding to the target link identifier; The step of determining the link governance information of the data output link based on the quality assessment results and governance information corresponding to each of the multiple data sets includes: Based on the quality assessment results corresponding to each of the multiple data sets, the overall quality assessment result of the data output link is determined, and a link overview information including the overall quality assessment result is generated; wherein the overall quality assessment result includes at least one of the following: overall link health score, maximum link node health score, minimum link node health score, link depth, number of link nodes, cumulative link calculation time, and link end time. Based on the quality assessment results and governance information corresponding to each of the multiple data sets, link details information of the data output link is generated, wherein the governance information of the data set also includes the governance information of the data set and the governance information of the upstream data set on which the data set depends; The link overview information and the link details information are determined as the link governance information of the data output link.

2. The method according to claim 1, characterized in that, The target link governance information includes target link overview information and target link details information; After obtaining the target link governance information corresponding to the target link identifier, the method further includes: Generate a first display interface corresponding to the target link overview information, wherein the target link overview information includes the overall quality assessment result of the target data output link; A second display interface corresponding to the target link details information is generated. The target link details information includes the governance information of each data set in the target data output link and the quality assessment results of each data set.

3. The method according to claim 2, characterized in that, After obtaining the target link governance information corresponding to the target data output link, the method further includes: After obtaining the target link details information corresponding to the target data production link, a display list corresponding to the hierarchical data dependency relationship between the data sets in the target data production link is generated. The display list includes the governance information of each data set in the target data production link.

4. The method according to claim 3, characterized in that, After generating the display list corresponding to the hierarchical data dependencies, the method further includes: Based on the request to view similar sets of the target data set, obtain the similar data set corresponding to the target data set.

5. The method according to claim 2, characterized in that, After obtaining the target link governance information corresponding to the target data output link, the method further includes: After obtaining the target link details information corresponding to the target data production link, a display graph corresponding to the hierarchical data dependency relationship between data sets in the target data production link is generated. The display graph includes the governance information of each data set in the target data production link.

6. The method according to claim 5, characterized in that, After generating the display graph corresponding to the hierarchical dependency relationship, the method further includes: Based on the request to obtain the graphics of the similar set corresponding to the target data set, obtain the similar data set corresponding to the target data set.

7. The method according to any one of claims 2-6, characterized in that, After obtaining the target link governance information corresponding to the target data output link, the method further includes: Based on the data governance request of the target data set, data governance operations are performed on the target data set according to the information to be governed in the target data set.

8. The method according to any one of claims 2-6, characterized in that, After obtaining the target link governance information corresponding to the target data output link, the method further includes: Based on the detailed viewing request of the target data set, obtain the detailed information of the target data set and the collection information of the downstream data set.

9. The method according to any one of claims 2-6, characterized in that, Before obtaining the target link governance information corresponding to the target link identifier in the information viewing request based on the target link identifier, the method further includes: Determine the platform identifier of the business platform corresponding to each data output link; Construct the correspondence between the link identifier and the platform identifier of the data output chain to obtain the business identifier association relationship; Based on the data viewing request of the target business platform, determine the target platform identifier of the target business platform; Based on the business identifier association relationship, obtain the filtering link identifier that matches the target platform identifier; Based on the filtered link identifier, the target link identifier is determined.

10. A data governance device, characterized in that, include: The acquisition module is configured to acquire metadata and quality assessment results for each of a plurality of data sets, wherein the metadata for each data set includes data source information; The governance information determination module is configured to determine the governance information of each data set based on the metadata of each data set and the quality assessment results; The quality assessment results mentioned above are health score indicators obtained by conducting data quality assessments on each of the data sets based on a data quality assessment system. The data link generation module is configured to generate data output links for multiple data sets based on data source information of multiple data sets. The data output links are links between data sets with data dependencies, and there is at least one data output link. The link governance information determination module is configured to determine the link governance information of the data output link based on the quality assessment results and governance information corresponding to each of the multiple data sets. The link governance information of the data output link includes the overall quality assessment result of the data output link and the governance information of each data set in the data output link. The mapping relationship construction module is configured to construct a mapping relationship between the link identifier of the data output link and the link governance information. The target link information acquisition module is configured to acquire target link governance information corresponding to the target link identifier based on the information viewing request of the target link identifier. The link governance information determination module includes: The overview information generation unit is configured to perform an overall quality assessment of the data output link based on the quality assessment results corresponding to the multiple data sets, and generate link overview information including the overall quality assessment results; wherein the overall quality assessment results include at least one of the following: overall link health score, maximum link node health score, minimum link node health score, link depth, number of link nodes, cumulative link calculation time, and link termination time. The detail information generation unit is configured to generate link detail information of the data output link based on the quality assessment results and governance information corresponding to each of the multiple data sets. The governance information of the data sets also includes the governance information of the data sets and the governance information of the upstream data sets on which the data sets depend. The link governance information determination unit is configured to perform link governance information that determines the link overview information and the link details information as the data output link.

11. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the data governance method as described in any one of claims 1 to 9.

12. A computer-readable storage medium, wherein instructions in the computer-readable storage medium, when executed by a processor of an electronic device, enable the electronic device to perform the data governance method as described in any one of claims 1 to 9.

Images