A data governance method and system for digital government affairs

Abstract

The present application relates to a data governance method and system for digital government affairs, and relates to the technical field of digital government affairs. The method includes: parsing SQL statements and corresponding table structures of different databases in the digital government affairs system based on a constructed SQL parser to obtain data operation information; constructing a global data relationship view based on the data operation information obtained by parsing; deploying monitoring points on the data flow path based on the global data relationship view to track data flow changes on the data flow path; when data flow changes are detected, identifying abnormal data based on the field mapping relationship, processing logic and dependency relationship recorded in the global data relationship view; and repairing the identified abnormal data according to the repair strategy. The present application has the effect of being able to quickly identify abnormal data and process it according to the repair strategy.

Description

Technical Field

[0001] The present application relates to the technical field of digital government affairs, and in particular to a data governance method and system for digital government affairs. Background Art

[0002] Against the backdrop of the rapid development of digital government affairs, government departments generate and process massive amounts of data through information systems, covering multiple areas such as administrative approval, public services, and decision-making analysis.

[0003] Currently, digital government systems typically consist of multiple heterogeneous databases, lacking unified management and coordination for data operations across them. Traditional methods struggle to fully parse SQL statements and table structures across different databases, making it difficult to accurately obtain data operation information, making data tracing and management extremely challenging.

[0004] How to solve the above technical problems is a technical difficulty that needs to be overcome by those skilled in the art. Summary of the Invention

[0005] In order to at least partially solve the above technical problems, the present application provides a data governance method and system for digital government affairs.

[0006] On the first aspect, the data governance method for digital government affairs provided in this application adopts the following technical solution.

[0007] A data governance method for digital government affairs, characterized by comprising:

[0008] Based on the constructed SQL parser, the SQL statements and corresponding table structures of different databases in the digital government system are parsed to obtain data operation information;

[0009] A global data relationship view is constructed based on the data operation information obtained through analysis; the global data relationship view uses nodes to represent data entities and edges to represent data flows, recording the entire life cycle of digital government data from collection, storage, processing to use;

[0010] Based on the global data relationship view, monitoring points are deployed on the data flow path to track data flow changes on the data flow path; when data flow changes are detected, abnormal data is identified based on the field mapping relationship, processing logic and dependency relationship recorded in the global data relationship view; the abnormal data includes missing data, data values ​​that do not conform to the value range, and data logic relationship errors;

[0011] Repair the identified abnormal data according to the repair strategy.

[0012] By employing this technical solution, an SQL parser parses SQL statements and table structures, extracting data operation information and constructing a global data relationship view. This clearly presents relationships throughout the data lifecycle, resolving the issue of ambiguous data relationships. This solution deploys monitoring points along the data flow path to track data flow changes in real time. Combined with the information recorded in the global data relationship view, this solution can quickly identify abnormal data and address it based on remediation strategies, effectively improving the efficiency and quality of digital government data governance.

[0013] Optionally, constructing a global data relationship view based on the data operation information obtained through analysis includes:

[0014] Establish a metadata center to store metadata for all data sources, data processing processes, and data products in the digital government system; the metadata includes data structure definitions, data processing logic, and data flow relationships;

[0015] Generate a relationship graph between data entities based on the parsed data operation information; the data entities include data tables, fields, processing procedures and data products;

[0016] In the relationship graph, nodes are used to represent data entities, and directed edges are used to represent dependencies and data flows between data entities. The dependencies include table-level dependencies and field-level dependencies. When a new SQL operation is detected, only the affected relationship paths are updated by comparing the new and old data operation information.

[0017] Among them, the establishment of a metadata center includes:

[0018] For data source metadata: establish interfaces with each data source system to collect and categorize and store the data source system name, data collection frequency, and data storage location; verify the collected data source metadata, and update the metadata if changes to the data source system, collection frequency adjustment, or storage location migration are found;

[0019] For data processing process metadata: During the ETL job operation, the job execution steps, data conversion rules, and calculation logic are recorded in real time; when the ETL job process is modified, the corresponding metadata is updated synchronously;

[0020] For data product metadata: When publishing the data service interface, generating reports, and creating analysis models, record the calling parameters and return format of the data service interface, the data source and statistical dimensions of the report, and the input and output variables of the analysis model and their dependencies.

[0021] Optionally, a relationship graph between data entities is generated based on the parsed data operation information, including:

[0022] Identify table-level dependencies in SQL statements and establish relationships between tables; the table-level dependencies include table creation, deletion, query, insert, and update operations;

[0023] Parse the field-level mapping relationship in the SQL statement and determine the mapping relationship between each field in different tables; the field-level mapping relationship includes the source of the field, calculation method and conversion rules;

[0024] Extract business rules from the data processing process and sort out the relationship between business rules and tables and fields; the business rules include filter conditions, aggregate functions and join operations;

[0025] Based on table-level dependencies, field-level mappings, and business rules, data entities are converted into nodes, and dependencies between tables generated based on the associations, dependencies between fields generated based on the mappings, and dependencies between entities and processing procedures, as well as data flows, are converted into directed edges;

[0026] The logical relationship between nodes and edges is checked for correctness based on the association relationship between tables and the mapping relationship between fields. During the verification process, if a conflict is found in the association relationship between tables or the mapping relationship between fields, adjustments are made according to the preset priority rules.

[0027] Optionally, when a new SQL operation is detected, only the affected relationship paths are updated by comparing the new and old data operation information, including:

[0028] Parsing newly detected SQL operations based on the SQL parser to extract change information related to data entities; the change information includes the change type, change content, and the association relationship involved in the change operation;

[0029] Compare and analyze the extracted data entity change information with the existing global data relationship view to determine the scope of affected data entities;

[0030] Based on the affected data entity range, calculate the affected relationship path in the global data relationship view;

[0031] Update the changed parts of the global data relationship view; if there are new data entities, create corresponding nodes in the graph database, and create corresponding directed edges based on the association relationship; if the data entity is modified, update the attribute information of the corresponding node and the attributes of the related directed edges; if the data entity is deleted, delete the corresponding node in the graph database and all related directed edges.

[0032] Optionally, deploy monitoring points including:

[0033] Extract all data flow paths from the constructed global data relationship view; analyze the data entity dependencies, data flow depth and impact range on each data flow path, and identify key nodes;

[0034] Setting the initial monitoring density based on the importance level of the data flow path; the importance of the data flow path is based on business criticality, historical anomaly frequency, and data dependency complexity; different importance levels correspond to different monitoring densities;

[0035] Deploy a runtime probe module on selected key nodes; the probe is used to collect runtime data features; the runtime data features include: data structure information, data content statistics, data operation behavior, and data processing time and resource consumption;

[0036] The density of monitoring points is adjusted based on the collected runtime data characteristics; when the indicators of a certain path show at least one of a change in data structure, a significant shift in data content distribution, and a sudden change in operation behavior, it is determined that enhanced monitoring is required.

[0037] Optionally, the parsing process of the SQL parser includes:

[0038] Collect historical SQL statement samples and classify them by operation type; extract grammatical and semantic features of each type of SQL statement to generate a multi-dimensional feature vector; cluster the multi-dimensional feature vectors to form a SQL feature vector library;

[0039] Extract the table name set, field name set, and operator sequence in the sentence through lexical analysis; identify the business keywords in the sentence by combining the business term library of the metadata center; and convert the extracted table name, field name, operator, and business keywords into feature vectors;

[0040] Calculate the cosine similarity between the feature vector of the SQL to be parsed and each template in the feature template library; when the similarity is greater than the preset value, directly reuse the parsing result corresponding to the SQL feature vector library; when the similarity is less than the preset value, trigger the syntax parsing process and add the new parsing result to the SQL feature vector library for iterative update.

[0041] Optionally, the method further includes:

[0042] Construct an anomaly prediction model; the anomaly prediction model constructs a training sample set based on historical data flow change records, field mapping relationships, processing logic and dependency relationships; and uses supervised learning for training;

[0043] When the monitoring point collects runtime data features, the data is input into the anomaly prediction model; the anomaly prediction model outputs the anomaly probability distribution of each field or table; for data entities whose anomaly probability is higher than the set threshold, a repair strategy is generated and an early warning is issued; the repair strategy includes field completion rules, default value filling strategy, cross-system data backtracking path and adaptive SQL constructor; the adaptive SQL constructor generates SQL statements that comply with grammatical specifications and can correct anomalies based on the parsed field-level mapping relationship and business rules combined with the current database structure; after the repair is completed, the repair process and repair results are fed back to the metadata center.

[0044] Secondly, the data governance system for digital government affairs provided in this application adopts the following technical solution.

[0045] A data governance system for digital government affairs, characterized by including:

[0046] The first processing module is used to: parse SQL statements and corresponding table structures of different databases in the digital government system based on the constructed SQL parser to obtain data operation information;

[0047] The second processing module is used to construct a global data relationship view based on the data operation information obtained through analysis; the global data relationship view uses nodes to represent data entities and edges to represent data flows, and records the entire life cycle of digital government data from collection, storage, processing to use;

[0048] A third processing module is configured to: deploy monitoring points on the data flow path based on the global data relationship view to track data flow changes on the data flow path; when a data flow change is detected, identify abnormal data based on the field mapping relationship, processing logic, and dependency relationship recorded in the global data relationship view; the abnormal data includes missing data, data values ​​that do not conform to the value range, and data logic relationship errors;

[0049] The fourth processing module is used to repair the identified abnormal data according to the repair strategy.

[0050] In summary, the present application has at least the following technical effects: it can quickly identify abnormal data and process it according to the repair strategy. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 This is a flow chart of a data governance method for digital government affairs according to an embodiment of the present application;

[0052] Figure 2 This is a system block diagram of a data governance system for digital government affairs according to an embodiment of the present application;

[0053] In the figure, 201 is the first processing module; 202 is the second processing module; 203 is the third processing module; 204 is the fourth processing module. DETAILED DESCRIPTION

[0054] The following is combined with Figure 1-2 The present application is further described with reference to the following specific examples:

[0055] The present application discloses a data governance method for digital government affairs, including the following steps:

[0056] Step 101: Using the constructed SQL parser, parse the SQL statements and corresponding table structures of different databases in the digital government system to obtain data operation information. Specifically, the SQL parser is a tool that analyzes the syntax and semantics of SQL statements. In the digital government system, it can disassemble the SQL statements and corresponding table structures of different databases to extract data operation information such as addition, deletion, modification, and query.

[0057] Step 102: Construct a global data relationship view based on the parsed data operation information. This global data relationship view uses nodes to represent data entities and edges to represent data flows, recording the entire lifecycle of digital government data, from collection, storage, processing, to use. Specifically, the global data relationship view is a graphical model that presents the relationships throughout the entire lifecycle of digital government data. Nodes represent data entities, such as data tables, fields, processing procedures, and data products, and edges represent data flows and dependencies between data entities.

[0058] Step 103: Based on the global data relationship view, monitoring points are deployed on the data flow path to track data flow changes on the data flow path; when data flow changes are detected, abnormal data is identified based on the field mapping relationship, processing logic and dependency relationship recorded in the global data relationship view; the abnormal data includes missing data, data values ​​that do not conform to the value range, and data logical relationship errors. Specifically, the data flow path is the route for data to be transmitted and processed between various links in the digital government system, and records the entire process from data generation to final use. Monitoring points can be understood as key locations deployed on the data flow path for real-time monitoring of data flow changes.

[0059] Step 104: Repair the identified abnormal data according to the repair strategy.

[0060] Specifically, the SQL parser is used to parse the SQL statements and table structures of different databases in the digital government system to obtain data operation information; a global data relationship view is constructed based on the data operation information to sort out the data life cycle relationship; monitoring points are deployed on the data flow path to track data flow changes in real time; once changes are detected, abnormal data is identified based on the field mapping relationship, processing logic and dependency relationship in the global data relationship view; abnormal data is repaired according to the repair strategy to achieve full-process governance of digital government data.

[0061] In digital government systems, the presence of heterogeneous databases results in fragmented and difficult-to-integrate data operation information, making traditional data tracing and management difficult. This solution uses a SQL parser to parse SQL statements and table structures, obtain data operation information, and then construct a global data relationship view, clearly presenting the relationships throughout the data lifecycle and resolving the issue of unclear data relationships. This solution deploys monitoring points along the data flow path to track data flow changes in real time. Combined with the information recorded in the global data relationship view, this solution can quickly identify abnormal data and process it according to repair strategies, effectively improving the efficiency and quality of digital government data governance.

[0062] As a specific implementation of a data governance method for digital government affairs, a global data relationship view is constructed based on the data operation information obtained through analysis, including:

[0063] Establish a metadata center to store metadata for all data sources, data processing processes, and data products in the digital government system; the metadata includes data structure definitions, data processing logic, and data flow relationships;

[0064] Generate a relationship graph between data entities based on the parsed data operation information; the data entities include data tables, fields, processing procedures and data products;

[0065] In the relationship graph, nodes are used to represent data entities, and directed edges are used to represent dependencies and data flows between data entities. The dependencies include table-level dependencies and field-level dependencies. When a new SQL operation is detected, only the affected relationship paths are updated by comparing the new and old data operation information.

[0066] Among them, the establishment of a metadata center includes:

[0067] For data source metadata: establish interfaces with each data source system to collect and categorize and store the data source system name, data collection frequency, and data storage location; verify the collected data source metadata, and update the metadata if changes to the data source system, collection frequency adjustment, or storage location migration are found;

[0068] For data processing process metadata: During the ETL job operation, the job execution steps, data conversion rules, and calculation logic are recorded in real time; when the ETL job process is modified, the corresponding metadata is updated synchronously;

[0069] For data product metadata: When publishing the data service interface, generating reports, and creating analysis models, record the calling parameters and return format of the data service interface, the data source and statistical dimensions of the report, and the input and output variables of the analysis model and their dependencies.

[0070] Specifically, the metadata center is the core for centrally storing metadata for all data sources, data processing processes, and data products in the digital government system. Metadata is data that describes data, covering information such as data structure definitions (such as data table field types), data processing logic (such as data calculation rules), and data flow relationships (such as which table the data flows from). The relationship graph is a network structure that graphically displays the relationships between data entities. In the relationship graph, nodes represent data entities, and directed edges represent the dependencies between entities and the flow of data, intuitively presenting the complex associations between data. Table-level dependency refers to the interdependence between data tables. For example, the creation, deletion, query, insertion, and update operations of a table will affect another table. This impact relationship is a table-level dependency. Field-level dependency focuses on the relationship between fields in a data table, including the source, calculation method, and conversion rules of the fields. For example, the value of a field may be calculated from other fields. This is a field-level dependency. ETL job: The abbreviation of Extract, Transform, and Load. It is a series of operations in the data processing process that extracts data from the data source, transforms it, and then loads it into the target database.

[0071] In this application, by identifying table-level and field-level dependencies and incorporating them into business rules during the construction of the relationship graph, the complex network of relationships between data entities can be fully presented, avoiding data management confusion caused by unclear relationships. When faced with new SQL operations, the solution adopts a strategy of updating only the affected relationship paths. Compared with traditional full disk updates, this significantly reduces data processing and improves the efficiency of updating the data relationship view, ensuring that it can reflect data changes in a timely manner.

[0072] As a specific implementation of a data governance method for digital government affairs, a relationship map between data entities is generated based on the parsed data operation information, including:

[0073] Identify table-level dependencies in SQL statements and establish relationships between tables; the table-level dependencies include table creation, deletion, query, insert, and update operations;

[0074] Parse the field-level mapping relationship in the SQL statement and determine the mapping relationship between each field in different tables; the field-level mapping relationship includes the source of the field, calculation method and conversion rules;

[0075] Extract business rules from the data processing process and sort out the relationship between business rules and tables and fields; the business rules include filter conditions, aggregate functions and join operations;

[0076] Based on table-level dependencies, field-level mappings, and business rules, data entities are converted into nodes, and dependencies between tables generated based on the associations, dependencies between fields generated based on the mappings, and dependencies between entities and processing procedures, as well as data flows, are converted into directed edges;

[0077] The logical relationship between nodes and edges is checked for correctness based on the association relationship between tables and the mapping relationship between fields. During the verification process, if a conflict is found in the association relationship between tables or the mapping relationship between fields, adjustments are made according to the preset priority rules.

[0078] Specifically, in traditional digital government data management, the relationships between tables and fields in different databases are vague, and the connection between business rules and data entities is difficult to clarify, resulting in chaotic data flow. During data query, update, or system upgrade, data errors and business logic conflicts are very likely to occur. This solution can accurately establish the relationship between tables and define the scope of data operations between different tables by identifying table-level dependencies in SQL statements, avoiding data linkage errors caused by table operations; parsing field-level mapping relationships can clarify the source and conversion rules of each field, and solve problems such as inconsistent field data and calculation errors. Extracting business rules from the data processing process and sorting out their connections with tables and fields can effectively avoid business anomalies caused by the disconnection between business rules and data operations.

[0079] As a specific implementation of a digital government data governance method, when a new SQL operation is detected, only the affected relationship paths are updated by comparing the new and old data operation information, including:

[0080] Parsing newly detected SQL operations based on the SQL parser to extract change information related to data entities; the change information includes the change type, change content, and the association relationship involved in the change operation;

[0081] Compare and analyze the extracted data entity change information with the existing global data relationship view to determine the scope of affected data entities;

[0082] Based on the affected data entity range, calculate the affected relationship path in the global data relationship view;

[0083] Update the changed parts of the global data relationship view; if there are new data entities, create corresponding nodes in the graph database, and create corresponding directed edges based on the association relationship; if the data entity is modified, update the attribute information of the corresponding node and the attributes of the related directed edges; if the data entity is deleted, delete the corresponding node in the graph database and all related directed edges.

[0084] Specifically, in traditional digital government data management systems, whenever a new SQL operation occurs, a full scan and update of the data relational view are often performed. This strategy not only consumes significant computing resources and time, but can also easily lead to incorrect associations of unaffected data due to excessive updates, resulting in reduced accuracy and stability of the data relational view. This solution uses a SQL parser to extract change information about data entities in new SQL operations, enabling rapid localization of the actual changes, avoiding indiscriminate scanning of the entire data relational view. This change information is then compared with the existing global data relational view to determine the scope of affected data entities, further narrowing the target area for the update operation. Relational paths are calculated based on the scope of affected data entities, ensuring that only the updates are performed on the areas that truly need to be updated, significantly reducing unnecessary computation and data processing. During the update process, operations such as creating nodes, updating attributes, and deleting nodes and related edges are implemented for different types of data entity changes, such as additions, modifications, and deletions. This ensures that the data relational view promptly reflects the latest SQL operation changes while avoiding interference with unaffected data. This solution significantly improves the update efficiency of the data relational view, reduces system resource consumption, and effectively avoids data errors and inconsistencies caused by frequent updates.

[0085] As one of the implementation methods of digital government data governance, the deployment of monitoring points includes:

[0086] Extract all data flow paths from the constructed global data relationship view; analyze the data entity dependencies, data flow depth and impact range on each data flow path, and identify key nodes;

[0087] Setting the initial monitoring density based on the importance level of the data flow path; the importance of the data flow path is based on business criticality, historical anomaly frequency, and data dependency complexity; different importance levels correspond to different monitoring densities;

[0088] Deploy a runtime probe module on selected key nodes; the probe is used to collect runtime data features; the runtime data features include: data structure information, data content statistics, data operation behavior, and data processing time and resource consumption;

[0089] The density of monitoring points is adjusted based on the collected runtime data characteristics; when the indicators of a certain path show at least one of a change in data structure, a significant shift in data content distribution, and a sudden change in operation behavior, it is determined that enhanced monitoring is required.

[0090] Specifically, traditional digital government data monitoring systems, lacking a systematic monitoring strategy, often employ a uniform deployment of monitoring points or blindly cover key nodes. This leads to irrational allocation of monitoring resources: insufficient monitoring of critical data paths and excessive monitoring of non-critical paths, resulting in not only wasted resources but also difficulty in quickly locating data anomalies. This solution extracts data flow paths from a global data relationship view and analyzes each path's data entity dependencies, flow depth, and impact range to identify key nodes that truly impact data flow, thus avoiding inefficient investment of monitoring resources. Path importance is ranked based on business criticality, historical anomaly frequency, and data dependency complexity, and differentiated initial monitoring densities are set to ensure that monitoring resources prioritize high-risk, high-value data paths, achieving efficient resource allocation. Monitoring point density is dynamically adjusted based on collected features, promptly increasing monitoring efforts when abnormal signals such as data structure changes, content distribution shifts, or sudden changes in operational behavior are detected. This approach reduces resource consumption when data is stable and enables rapid response when potential risks emerge, reducing monitoring costs and enhancing the ability to warn and address data anomalies.

[0091] As one implementation of a data governance method for digital government affairs, the parsing process of the SQL parser includes:

[0092] Collect historical SQL statement samples and classify them by operation type; extract grammatical and semantic features of each type of SQL statement to generate a multi-dimensional feature vector; cluster the multi-dimensional feature vectors to form a SQL feature vector library;

[0093] Extract the table name set, field name set, and operator sequence in the sentence through lexical analysis; identify the business keywords in the sentence by combining the business term library of the metadata center; and convert the extracted table name, field name, operator, and business keywords into feature vectors;

[0094] Calculate the cosine similarity between the feature vector of the SQL to be parsed and each template in the feature template library; when the similarity is greater than the preset value, directly reuse the parsing result corresponding to the SQL feature vector library; when the similarity is less than the preset value, trigger the syntax parsing process and add the new parsing result to the SQL feature vector library for iterative update.

[0095] As one implementation of a data governance method for digital government affairs, the method further includes:

[0096] Construct an anomaly prediction model; the anomaly prediction model constructs a training sample set based on historical data flow change records, field mapping relationships, processing logic and dependency relationships; and uses supervised learning for training;

[0097] When the monitoring point collects runtime data features, the data is input into the anomaly prediction model; the anomaly prediction model outputs the anomaly probability distribution of each field or table; for data entities whose anomaly probability is higher than the set threshold, a repair strategy is generated and an early warning is issued; the repair strategy includes field completion rules, default value filling strategy, cross-system data backtracking path and adaptive SQL constructor; the adaptive SQL constructor generates SQL statements that comply with grammatical specifications and can correct anomalies based on the parsed field-level mapping relationship and business rules combined with the current database structure; after the repair is completed, the repair process and repair results are fed back to the metadata center.

[0098] This application also provides a data governance system for digital government affairs, including:

[0099] The first processing module 201 is used to parse SQL statements and corresponding table structures of different databases in the digital government system based on the constructed SQL parser to obtain data operation information;

[0100] The second processing module 202 is configured to construct a global data relationship view based on the data operation information obtained through analysis; the global data relationship view uses nodes to represent data entities and edges to represent data flows, and records the entire life cycle of digital government data from collection, storage, processing to use;

[0101] The third processing module 203 is configured to: deploy monitoring points on the data flow path based on the global data relationship view to track data flow changes on the data flow path; when a data flow change is detected, identify abnormal data based on the field mapping relationship, processing logic, and dependency relationship recorded in the global data relationship view; the abnormal data includes missing data, data values ​​that do not conform to the value range, and data logic relationship errors;

[0102] The fourth processing module 204 is configured to repair the identified abnormal data according to a repair strategy.

[0103] It should be noted that the above embodiments are only used to illustrate the present application and are not intended to limit the technical solutions described in the present application. Although this specification has described the present application in detail with reference to the above embodiments, ordinary technicians in this field should understand that technicians in the relevant technical field can still modify or replace the present application with equivalents, and all technical solutions and improvements that do not depart from the spirit and scope of the present application should be included in the scope of the claims of the present application.

Claims

1. A data governance method for digital government affairs, characterized in that: include: Based on the constructed SQL parser, the SQL statements and corresponding table structures of different databases in the digital government system are parsed to obtain data operation information; Constructing a global data relationship view based on the data operation information obtained by parsing; The global data relationship view uses nodes to represent data entities and edges to represent data flows, recording the entire life cycle of digital government data from collection, storage, processing to use; Based on the global data relationship view, monitoring points are deployed on the data flow path to track data flow changes on the data flow path; when data flow changes are detected, abnormal data is identified based on the field mapping relationship, processing logic and dependency relationship recorded in the global data relationship view; the abnormal data includes missing data, data values ​​that do not conform to the value range, and data logic relationship errors; Repair the identified abnormal data according to the repair strategy; Constructing a global data relationship view based on the data operation information obtained through analysis, including: Establish a metadata center to store metadata for all data sources, data processing processes, and data products in the digital government system; the metadata includes data structure definitions, data processing logic, and data flow relationships; Generate a relationship graph between data entities based on the parsed data operation information; the data entities include data tables, fields, processing procedures and data products; In the relationship graph, nodes are used to represent data entities, and directed edges are used to represent dependencies and data flows between data entities. The dependencies include table-level dependencies and field-level dependencies. When a new SQL operation is detected, only the affected relationship paths are updated by comparing the new and old data operation information. Among them, the establishment of a metadata center includes: For data source metadata: establish interfaces with each data source system to collect and categorize and store the data source system name, data collection frequency, and data storage location; verify the collected data source metadata, and update the metadata if changes to the data source system, collection frequency adjustment, or storage location migration are found; For data processing process metadata: During the ETL job operation, the job execution steps, data conversion rules, and calculation logic are recorded in real time; when the ETL job process is modified, the corresponding metadata is updated synchronously; For data product metadata: When publishing the data service interface, generating reports, and creating analysis models, record the calling parameters and return format of the data service interface, the data source and statistical dimensions of the report, and the input and output variables of the analysis model and their dependencies.

2. A data governance method for digital government affairs according to claim 1, characterized in that: Generate a relationship graph between data entities based on the parsed data operation information, including: Identify table-level dependencies in SQL statements and establish relationships between tables; the table-level dependencies include table creation, deletion, query, insert, and update operations; Parse the field-level mapping relationship in the SQL statement and determine the mapping relationship between each field in different tables; the field-level mapping relationship includes the source of the field, calculation method and conversion rules; Extract business rules from the data processing process and sort out the relationship between business rules and tables and fields; the business rules include filter conditions, aggregate functions and join operations; Based on table-level dependencies, field-level mappings, and business rules, data entities are converted into nodes, and dependencies between tables generated based on the associations, dependencies between fields generated based on the mappings, and dependencies between entities and processing procedures, as well as data flows, are converted into directed edges; The logical relationship between nodes and edges is checked for correctness based on the association relationship between tables and the mapping relationship between fields. During the verification process, if a conflict is found in the association relationship between tables or the mapping relationship between fields, adjustments are made according to the preset priority rules.

3. A data governance method for digital government affairs according to claim 2, characterized in that: When a new SQL operation is detected, only the affected relationship paths are updated by comparing the new and old data operation information, including: Parsing newly detected SQL operations based on the SQL parser to extract change information related to data entities; the change information includes the change type, change content, and the association relationship involved in the change operation; Compare and analyze the extracted data entity change information with the existing global data relationship view to determine the scope of affected data entities; Based on the affected data entity range, calculate the affected relationship path in the global data relationship view; Update the changed parts of the global data relationship view; if there are new data entities, create corresponding nodes in the graph database, and create corresponding directed edges based on the association relationship; if the data entity is modified, update the attribute information of the corresponding node and the attributes of the related directed edges; if the data entity is deleted, delete the corresponding node in the graph database and all related directed edges.

4. A data governance method for digital government affairs according to claim 3, characterized in that: Deployment monitoring points include: Extract all data flow paths from the constructed global data relationship view; analyze the data entity dependencies, data flow depth and impact range on each data flow path, and identify key nodes; Setting the initial monitoring density based on the importance level of the data flow path; the importance of the data flow path is based on business criticality, historical anomaly frequency, and data dependency complexity; different importance levels correspond to different monitoring densities; Deploy a runtime probe module on selected key nodes; the probe is used to collect runtime data features; the runtime data features include: data structure information, data content statistics, data operation behavior, and data processing time and resource consumption; The density of monitoring points is adjusted based on the collected runtime data characteristics; when the indicators of a certain path show at least one of a change in data structure, a significant shift in data content distribution, and a sudden change in operation behavior, it is determined that enhanced monitoring is required.

5. A data governance method for digital government affairs according to claim 4, characterized in that: The parsing process of the SQL parser includes: Collect historical SQL statement samples and classify them by operation type; extract grammatical and semantic features of each type of SQL statement to generate a multi-dimensional feature vector; cluster the multi-dimensional feature vectors to form a SQL feature vector library; Extract the table name set, field name set, and operator sequence in the sentence through lexical analysis; identify the business keywords in the sentence by combining the business term library of the metadata center; and convert the extracted table name, field name, operator, and business keywords into feature vectors; Calculate the cosine similarity between the feature vector of the SQL to be parsed and each template in the feature template library; when the similarity is greater than the preset value, directly reuse the parsing result corresponding to the SQL feature vector library; when the similarity is less than the preset value, trigger the syntax parsing process and add the new parsing result to the SQL feature vector library for iterative update.

6. A data governance method for digital government affairs according to claim 5, characterized in that: The method further comprises: Construct an anomaly prediction model; the anomaly prediction model constructs a training sample set based on historical data flow change records, field mapping relationships, processing logic and dependency relationships; and uses supervised learning for training; When the monitoring point collects runtime data features, the data is input into the anomaly prediction model; the anomaly prediction model outputs the anomaly probability distribution of each field or table; for data entities whose anomaly probability is higher than the set threshold, a repair strategy is generated and an early warning is issued; the repair strategy includes field completion rules, default value filling strategy, cross-system data backtracking path and adaptive SQL constructor; the adaptive SQL constructor generates SQL statements that comply with grammatical specifications and can correct anomalies based on the parsed field-level mapping relationship and business rules combined with the current database structure; after the repair is completed, the repair process and repair results are fed back to the metadata center.

7. A data governance system for digital government affairs, characterized by: include: The first processing module is used to: parse SQL statements and corresponding table structures of different databases in the digital government system based on the constructed SQL parser to obtain data operation information; A second processing module is configured to construct a global data relationship view based on the data operation information obtained through analysis; The global data relationship view uses nodes to represent data entities and edges to represent data flows, recording the entire life cycle of digital government data from collection, storage, processing to use; A third processing module is configured to: deploy monitoring points on the data flow path based on the global data relationship view to track data flow changes on the data flow path; when a data flow change is detected, identify abnormal data based on the field mapping relationship, processing logic, and dependency relationship recorded in the global data relationship view; the abnormal data includes missing data, data values ​​that do not conform to the value range, and data logic relationship errors; The fourth processing module is used to: repair the identified abnormal data according to the repair strategy; Constructing a global data relationship view based on the data operation information obtained through analysis, including: Establish a metadata center to store metadata for all data sources, data processing processes, and data products in the digital government system; the metadata includes data structure definitions, data processing logic, and data flow relationships; Generate a relationship graph between data entities based on the parsed data operation information; the data entities include data tables, fields, processing procedures and data products; In the relationship graph, nodes are used to represent data entities, and directed edges are used to represent dependencies and data flows between data entities. The dependencies include table-level dependencies and field-level dependencies. When a new SQL operation is detected, only the affected relationship paths are updated by comparing the new and old data operation information. Among them, the establishment of a metadata center includes: For data source metadata: establish interfaces with each data source system to collect and categorize and store the data source system name, data collection frequency, and data storage location; verify the collected data source metadata, and update the metadata if changes to the data source system, collection frequency adjustment, or storage location migration are found; For data processing process metadata: During the ETL job operation, the job execution steps, data conversion rules, and calculation logic are recorded in real time; when the ETL job process is modified, the corresponding metadata is updated synchronously; For data product metadata: When publishing the data service interface, generating reports, and creating analysis models, record the calling parameters and return format of the data service interface, the data source and statistical dimensions of the report, and the input and output variables of the analysis model and their dependencies.

Images