Enterprise-level data-oriented quality repair method, device, equipment and medium

Abstract

The application provides a quality repair method and device for enterprise-level data, equipment and medium, relates to the technical field of data processing, and comprises the following steps: a data chain topology graph is constructed, instances corresponding to a plurality of key nodes included in the data chain topology graph are implanted with quality measurement probes, and the quality measurement probes are used for collecting data quality indexes corresponding to the key nodes; if the data quality indexes meet preset alarm conditions, an alarm event corresponding to the data quality indexes is generated, and a target repair rule matched with the alarm event in a repair rule knowledge base is used to perform a repair action on the instances corresponding to the key nodes; and repair effect verification is performed based on data quality indexes collected by the quality measurement probes after the repair action, so that the preset alarm conditions and / or the repair rule knowledge base are updated, so that data quality repair can be continuously implemented. The application can realize automatic response and closed-loop repair of data quality alarms.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method, apparatus, equipment, and medium for quality restoration of enterprise-level data. Background Technology

[0002] Existing technologies primarily concern data quality management solutions within enterprise data governance. Current mainstream solutions typically employ a phased and modular approach to data quality control. The entire process, from problem discovery to resolution, is lengthy and heavily reliant on manual response and operation, forming an open-loop, reactive management chain. The core of existing solutions lies in defining data quality rules and generating quality reports, but they lack the ability for automatic problem response and closed-loop remediation. Summary of the Invention

[0003] In view of this, the purpose of the present invention is to provide a method, apparatus, device and medium for quality repair of enterprise-level data, which can realize automatic response and closed-loop repair of data quality alarms.

[0004] In a first aspect, the present invention provides a quality restoration method for enterprise-level data, applied to a quality restoration system, comprising: A data chain topology graph is constructed. Quality metric probes are embedded at the instances corresponding to multiple key nodes in the data chain topology graph. The quality metric probes are used to collect data quality indicators corresponding to the key nodes. If the data quality indicators meet the preset alarm conditions, an alarm event corresponding to the data quality indicators will be generated, and based on the target repair rules in the repair rule knowledge base that match the alarm event, repair actions will be performed on the instances corresponding to the key nodes. The data quality indicators collected by the quality metric probe after the repair action are used to verify the repair effect and update the preset alarm conditions and / or repair rule knowledge base to continue to achieve data quality repair.

[0005] In one implementation, the quality repair system includes a quality control center, which deploys an indicator aggregation rule base and an indicator alarm rule base. If a data quality indicator meets preset alarm conditions, an alarm event corresponding to the data quality indicator is generated, including: Data quality metrics are encapsulated as event messages, which are then asynchronously sent to the quality control center using a message middleware. The following operations are performed on the event message through the Quality Control Center: Extract data quality metrics corresponding to key nodes from event messages; Determine the indicator aggregation rules that match the key nodes from the indicator aggregation rule base, and generate the indicator aggregation results corresponding to the data quality indicators according to the indicator aggregation rules; The system determines the indicator alarm rules that match the key nodes from the indicator alarm rule library. When the data quality indicator is determined to meet the preset alarm conditions based on the indicator alarm rules, the system generates the alarm event corresponding to the data quality indicator.

[0006] In one implementation, the quality repair system further includes a visualization dashboard, and the method further includes: Obtain the data chain topology graph; Retrieve alarm events corresponding to at least one target node in the data chain topology graph to determine the health status of the target node; Based on the visual effects of health status matching, the data chain topology graph is rendered to obtain a data chain status graph, which is then displayed through a visualization dashboard. The data chain status graph is used to display the health status of the target nodes in the data chain topology graph with different visual effects.

[0007] In one implementation, the method further includes: Through a visual dashboard, in response to the selection operation of any target node in the data chain situational graph, the historical trend graph of the target node corresponding to the selection operation and / or a list of alarm events are displayed. Among them, the historical trend chart of indicators is generated based on the aggregated results of indicators of the target node at multiple historical moments, and the alarm event list is generated based on the alarm events of the target node at multiple historical moments.

[0008] In one implementation, the quality repair system further includes a repair engine, which has a repair rule knowledge base deployed within it. Based on the target repair rule in the repair rule knowledge base that matches the alarm event, repair actions are performed on the instances corresponding to the critical nodes, including: The alarm event is matched with the candidate repair rules stored in the repair rule knowledge base. The candidate repair rule with a matching degree greater than a preset matching degree threshold is taken as the target repair rule for the alarm event. The candidate repair rule and the target repair rule both include the rule triggering conditions, the repair execution script, and the security context required for the repair execution script. Initialize the repair container; Load the repair execution script into the repair container, configure the repair execution script according to the security context, and perform repair actions on the instances corresponding to the critical nodes based on the configured repair execution script.

[0009] In one implementation, the method further includes: If the matching degree between an alarm event and any alternative repair rule stored in the repair rule knowledge base is less than a preset matching degree threshold, the upstream node set and downstream node set of the key node are determined based on the data chain topology graph. Root cause analysis is performed based on the data quality indicators corresponding to key nodes, upstream node sets, and downstream node sets to obtain root cause nodes. Based on the alarm events, key nodes and their corresponding data quality indicators, root cause nodes and their corresponding data quality indicators, a diagnostic report is generated and sent to the designated associated terminal.

[0010] In one implementation, the repair effect is verified based on data quality indicators collected by a quality metric probe after the repair action, including: Acquire new data quality indicators collected by the quality measurement probe within a preset observation time window, where the start time of the observation time window is the completion time of the repair action; According to the indicator aggregation rules, generate new indicator aggregation results corresponding to the new data quality indicators; Based on the indicator alarm rules, the new data quality indicators are judged: if the new data quality indicators meet the preset alarm conditions, the repair is judged to be successful; if the new data quality indicators do not meet the preset alarm conditions, the repair is judged to be unsuccessful.

[0011] Secondly, the present invention also provides a quality repair device for enterprise-level data, applied to a quality repair system, comprising: The graph construction module is used to build a data chain topology graph. The data chain topology graph contains multiple key nodes, and quality metric probes are embedded at the corresponding instances. The quality metric probes are used to collect data quality indicators corresponding to the key nodes. The repair module is used to generate an alarm event corresponding to the data quality indicator if the data quality indicator meets the preset alarm conditions, and to perform repair actions on the instance corresponding to the key node based on the target repair rule that matches the alarm event in the repair rule knowledge base. The effect verification module is used to verify the repair effect based on the data quality indicators collected by the quality metric probe after the repair action. It is also used to update the preset alarm conditions and / or repair rule knowledge base to continue to achieve data quality repair.

[0012] Thirdly, the present invention also provides an electronic device including a processor and a memory, the memory storing computer-executable instructions executable by the processor, the processor executing the computer-executable instructions to implement any of the methods provided in the first aspect.

[0013] Fourthly, the present invention also provides a computer-readable storage medium storing computer-executable instructions, which, when invoked and executed by a processor, cause the processor to implement any of the methods provided in the first aspect.

[0014] This invention provides a method, apparatus, device, and medium for quality repair of enterprise-level data. First, a data chain topology diagram is constructed. Quality measurement probes are embedded at the instances corresponding to multiple key nodes in the data chain topology diagram. These probes collect data quality indicators corresponding to the key nodes. If the data quality indicators meet preset alarm conditions, an alarm event corresponding to the data quality indicator is generated. Based on the target repair rule matching the alarm event in the repair rule knowledge base, a repair action is performed on the instance corresponding to the key node. The repair effect is verified based on the data quality indicators collected by the quality measurement probes after the repair action, which is used to update the preset alarm conditions and / or the repair rule knowledge base to continue data quality repair. This method, by constructing a data chain topology diagram and embedding quality measurement probes at the runtime instances of key nodes within it, achieves real-time, low-overhead collection of data quality indicators. Combined with streaming alarm judgment, instance-level automated repair execution, and immediate effect verification of the same probe after repair, it ensures that the system can complete the entire process from anomaly identification to result confirmation without manual intervention, thus achieving automatic response and closed-loop repair of data quality alarms.

[0015] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention are realized and obtained through the structures particularly pointed out in the description and the drawings.

[0016] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0017] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating a method for quality repair of enterprise-level data provided in an embodiment of the present invention; Figure 2 A flowchart illustrating another quality repair method for enterprise-level data provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of a quality repair device for enterprise-level data provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] Currently, with the deepening of enterprise digital transformation, the scale of data assets is growing rapidly, data sources are becoming increasingly diversified, and the importance of data-driven decision-making has significantly increased. However, low-quality data can easily lead to distorted analysis results, decision-making biases, decreased operational efficiency, and increased compliance risks. In industries such as finance, manufacturing, and retail, real-time or near-real-time business scenarios (such as real-time risk control and dynamic pricing) place extremely high demands on the timeliness and accuracy of data, while traditional data quality management models that rely on daily batch processing and manual intervention are no longer sufficient to meet actual needs.

[0021] The existing technology has the following problems: First, the response timeliness is insufficient. Data quality checks generally adopt periodic batch processing methods such as T+1. From the occurrence of the problem, rule scanning, report generation, manual review, root cause location to remediation execution, the entire process can take several hours or even days, which cannot support the requirement of minute-level response.

[0022] Second, the repair efficiency is low and the labor cost is high. The handling of quality issues relies heavily on manual intervention, requiring data engineers, analysts, or business personnel to invest a lot of time in root cause analysis, writing repair scripts, and executing verification. When faced with occasional or periodic issues in massive amounts of data, this model is inefficient and continuously consumes highly skilled human resources.

[0023] Third, there is a lack of proactive early warning and adaptive optimization capabilities. Existing solutions only trigger detection and alarms after a problem occurs, failing to provide proactive risk warnings based on historical trends or learn from past governance practices to optimize the data flow itself. For example, they cannot automatically identify a declining trend in the connection success rate of a data access endpoint and issue an early warning, nor can they adjust the cleaning rule parameters according to error patterns to prevent similar problems from recurring.

[0024] Fourth, there is a lack of a closed-loop quality governance system. Throughout the entire lifecycle of data, from its generation and flow to consumption, quality control actions are scattered across different stages, failing to form a complete autonomous closed-loop mechanism encompassing measurement, analysis, decision-making, execution, and verification. This makes it difficult to quantify and assess the effectiveness of governance and hinders the implementation of continuous quality improvement.

[0025] Based on this, the present invention provides a method, apparatus, device, and medium for quality repair of enterprise-level data, aiming to solve the core problems of data quality management in existing enterprise data governance, such as delayed response, high dependence on manual intervention, and lack of a complete autonomous closed loop. Specific objectives include: Achieve near real-time or real-time detection and alerting of data quality issues, significantly reducing the response time window; automatically trigger predefined repair actions for rule-based data quality issues, reducing or even eliminating manual intervention and improving repair efficiency and execution consistency; conduct risk prediction based on historical quality data trends to achieve early warning of potential problems, driving quality governance from passive response to proactive defense; construct an autonomous closed loop covering the entire process of measurement, analysis, decision-making, execution, and verification, enabling data quality management to have automatic operation, continuous iteration, and self-optimization capabilities, thereby reducing governance and operation costs and improving the reliability and business value of data assets.

[0026] To facilitate understanding of this embodiment, a quality repair method for enterprise-level data disclosed in this embodiment of the invention will first be described in detail. This method is applied to a quality repair system. See [link to relevant documentation]. Figure 1 The diagram shows a process flow chart for a quality repair method for enterprise-level data. The method mainly includes the following steps S102 to S106: Step S102: Construct the data chain topology graph.

[0027] The data chain topology graph (also known as the data lineage graph) consists of nodes and directed edges. Nodes represent data assets or data processing tasks, and directed edges represent unidirectional data flow relationships. Nodes are divided into data resource nodes and data processing task nodes. Data resource nodes include, but are not limited to, relational database tables, NoSQL collections, message queue topics, file paths in object storage, and external API access endpoints. Data processing task nodes include, but are not limited to, Flink streaming jobs, Spark batch processing tasks, and transformation components in ETL tools.

[0028] The data chain topology diagram contains multiple key nodes, each with a quality metric probe embedded in its corresponding instance. The quality metric probe is a lightweight monitoring component deployed in the container where the instance resides in Sidecar mode, or embedded as an SDK in the data processing task code. It is used to collect data quality indicators corresponding to key nodes. For example, for data resource nodes, the collected data quality indicators include, but are not limited to, message inflow rate and message size. For data processing task nodes, the collected data quality indicators include, but are not limited to, the null value rate of key fields.

[0029] In one implementation, the data chain topology graph can be manually constructed by data governance engineers or automatically generated by the system through parsing SQL scripts, task logs, metadata management systems, etc.; the quality metric probe can be manually configured and deployed by engineers at the runtime instance corresponding to the key node, or automatically deployed by the system according to the preset strategy.

[0030] Step S104: If the data quality indicators meet the preset alarm conditions, an alarm event corresponding to the data quality indicators is generated, and based on the target repair rules in the repair rule knowledge base that match the alarm event, repair actions are performed on the instances corresponding to the key nodes.

[0031] Pre-configured alarm conditions refer to the quality anomaly judgment logic pre-configured in the alarm module. These conditions can take the form of static thresholds, dynamic thresholds, or compound logical expressions, and are used for rule-based comparison of real-time collected data quality indicators. Alarm events are structured data objects containing a unique event ID, trigger timestamp, associated key node ID, corresponding data object identifier (e.g., table name, field name, task ID), original indicator value, pre-configured alarm condition content, and the matched alarm level (P1–P3). The repair rule knowledge base is a set of structured rules stored in a relational database or configuration center. Each rule contains three elements: rule triggering conditions, repair execution script, and the security context required for the repair execution script. The target repair rule is the single rule in the repair rule knowledge base that matches the triggering conditions of the current alarm event and has the highest confidence level, selected by the rule matching engine.

[0032] In one implementation, when the data quality indicators meet the preset alarm conditions: a structured alarm event is generated and persisted to the event library; a target repair rule matching the alarm event is retrieved from the repair rule knowledge base; the repair action bound to the rule is invoked in the secure isolation environment, applied to the key node instance corresponding to the alarm, and the execution result (success / failure) is associated with the log summary and recorded to the alarm event.

[0033] Step S106: Verify the repair effect based on the data quality indicators collected by the quality measurement probe after the repair action. This is used to update the preset alarm conditions and / or repair rule knowledge base to continue to achieve data quality repair.

[0034] Among them, the verification of repair effect refers to the technical process of re-collecting the data quality indicators of the same key node instance through the quality metric probe after the repair action is completed, and comparing them with the preset alarm conditions to determine whether the anomaly has been eliminated; the output result is three types of deterministic status: "resolved", "unresolved" or "partially resolved".

[0035] In one implementation, after the repair action is completed, a preset observation time window is waited for (e.g., 60 seconds for SQL repair and 300 seconds for task rerun). The original quality metric probe actively collects the latest data quality indicators of the critical node instance. The newly collected data quality indicators are input into preset alarm conditions for judgment: if they are not met, the verification result is "resolved"; if they are still met, the result is "unresolved". The verification result and snapshots of the indicators before and after are associated and written into the processing record of the original alarm event to form a closed-loop evidence. The optimization analysis service extracts complete closed-loop cases from the past N days from the case library for mining and analysis, which are used to update the preset alarm conditions and / or repair rule knowledge base to continue to achieve data quality repair.

[0036] The quality repair method for enterprise-level data provided in this invention constructs a data link topology diagram and embeds quality measurement probes at the runtime instances of key nodes within it, enabling real-time, low-overhead collection of data quality indicators. Combined with streaming alarm judgment, instance-level automated repair execution, and immediate effect verification of the same probe after repair, the system ensures that the entire process from anomaly identification to result confirmation can be completed without manual intervention, thus achieving automatic response and closed-loop repair of data quality alarms.

[0037] To facilitate understanding, this invention provides a specific implementation of a quality repair method for enterprise-level data. Compared with existing technologies, the core difference lies in connecting the various key nodes in the data lifecycle into an observable and interventionist intelligent closed loop, and introducing automatic repair through logic and rule engines and a feedback-based learning optimization mechanism. See [link to relevant documentation]. Figure 2 The diagram shows another method for quality restoration of enterprise-level data, which mainly includes the following parts: Step 1: Data chain topology graph construction and quality metric probe implantation.

[0038] This step forms the basis for end-to-end quality observability.

[0039] In one implementation, the system provides a data chain modeling tool that supports graphical drag-and-drop operations, allowing data governance engineers to use enterprise data assets (including database tables, message queue topics, API interfaces, and file paths) and data processing tasks (including Flink streaming jobs, Spark batch tasks, stored procedures, and transformation components in ETL tools) as nodes, and connect them according to the actual data flow to generate a visual data chain topology diagram.

[0040] In another implementation, the system also supports automatically discovering the lineage relationships between some nodes by parsing SQL scripts, job logs, or connecting to a metadata management system. The specific implementation process is as follows: (1) When the system receives an SQL script uploaded by a user, it calls the SQL parser to identify keywords such as `FROM`, `JOIN`, and `INSERTINTO`, extracts the source table name and target table name, generates directed edges between corresponding database table nodes, and injects them into the data chain topology graph; (2) When the system connects to the log stream of a task scheduling platform (such as Apache Airflow), it parses the task dependency relationships (such as `TaskA>>TaskB`) and input / output datasets recorded in the logs, and establishes connections between data processing task nodes and data resource nodes according to the actual flow direction; (3) When the system connects to the enterprise metadata management system, it synchronizes the data asset lineage relationships it has maintained, and maps the source node, target node, and association type (such as "processes" and "reads") to nodes and directed edges in the data chain topology graph. The lineage relationships generated in the above process are all verified and merged into the current data chain topology, and marked as "automatically discovered" for governance engineers to review and confirm or manually correct.

[0041] Once the modeling is complete, engineers can declaratively configure quality metric probes on any node. The probes are lightweight monitoring components that support two deployment modes: injection into the container where the data task resides as a Sidecar, or embedding them as an SDK into the task code. Their data collection methods include listening to log files, calling monitoring interfaces exposed by the task, executing verification SQL, and parsing the task's standard output.

[0042] For different node types, probes can be configured with differentiated metrics: (1) On the Kafka data asset node, collect the message inflow rate "records_per_second" and the average message size "avg_message_size"; (2) In the task node that writes to the Hive table in Spark, execute a predefined SQL after each successful submission, such as SELECT COUNT(*) FROM target_table WHERE dt='${biz_date}', to get the number of records written, and synchronously check the null value rate of key fields; (3) At the data access node, collect the fluctuation rate of the number of records and the field fill rate; (4) After the data cleaning rules are executed, collect the number of rule triggers and the number of invalid values ​​replaced; (5) Before the data service is released, collect the response time of the interface and the consistency of the data return.

[0043] Step 2, multi-dimensional quality measurement and real-time situational awareness, the specific process is as follows: Step 2.1: Encapsulate the data quality metrics into event messages, and use message middleware to asynchronously send the event messages to the quality control center.

[0044] All raw metrics collected by the probes are encapsulated into event messages in a unified format (including timestamp, chain node ID, data object ID, metric name, and metric value), and asynchronously pushed to the quality control center via a high-throughput message middleware (such as Apache Pulsar or Kafka). Probe deployment has minimal impact on the performance of the original data task, with actual resource utilization below 1%, and is compatible with both batch and stream processing tasks, achieving non-intrusive or low-intrusive monitoring.

[0045] Step 2.2, the Quality Control Center performs the following operations on the event message: Steps S2.21 to S2.23. The quality remediation system includes a quality control center, which is the core processing unit of the system. It has a built-in streaming engine (such as Apache Flink or Apache Storm) for continuously subscribing to and processing metric event messages from the message middleware in real time. Internally, the engine maintains the metadata mapping relationship between data chain nodes, data objects, and metrics. The quality control center also deploys a metric aggregation rule base and a metric alerting rule base.

[0046] Step 2.21: Parse the data quality indicators corresponding to the key nodes from the event messages.

[0047] The streaming engine (Flink) consumes event message streams from the Pulsar topic, using a predefined JSON deserializer to extract timestamps, key node identifiers, metric names, and metric values. For each message, the engine looks up the key node identifier to determine its node type (data resource node / data processing task node) and its upstream and downstream relationships in the data chain topology, thus completing the metadata context binding.

[0048] Step 2.22: Determine the indicator aggregation rules that match the key nodes from the indicator aggregation rule library, so as to generate the indicator aggregation results corresponding to the data quality indicators according to the indicator aggregation rules.

[0049] The engine queries the indicator aggregation rule library based on the key node identifier and indicator name to determine the matching indicator aggregation rule.

[0050] For example: For simple raw data quality indicators (such as "total number of records" and "number of incoming messages"), the system directly forwards the indicator values ​​along with timestamps and key node identifiers and writes them into the time series database for subsequent queries. For data quality metrics that require multi-source correlation calculations (such as "field fill rate"), the system, based on preset aggregation rules, correlates the metric values ​​of multiple original metrics within the same key node and processing batch, and performs deterministic calculations: Field fill rate = (Total number of records) / (Total number of records) (Number of blank records) ÷ Total number of records × 100%; For data quality metrics that require time-based statistics (such as "average message inflow rate over the past hour"), the system calls the built-in window function of the streaming engine to aggregate the continuously arriving metric values ​​according to the specified type (scrolling window or sliding window) and time granularity (such as 10 seconds, 1 minute, 5 minutes), and outputs statistical results with time context.

[0051] The calculated aggregated results of the indicators are written in parallel to two types of storage systems: (1) written to a time series database in real time to support millisecond-level indicator queries, dynamic threshold calculations and streaming alarm judgments; (2) synchronously persisted to a relational database or data lake for long-term storage, multi-dimensional correlation analysis, quality trend modeling and governance report generation.

[0052] Step 2.23: Determine the indicator alarm rules that match the key nodes from the indicator alarm rule library. When the data quality indicator is determined to meet the preset alarm conditions based on the indicator alarm rules, generate the alarm event corresponding to the data quality indicator.

[0053] The alarm module continuously pulls aggregated data streams of metrics from the time-series database via streaming subscription. The data stream contains metric records arranged in chronological order, with each record carrying a key node identifier, data object identifier, metric name, metric value, and timestamp.

[0054] The alarm module performs real-time matching of data streams based on the alarm rules configured by the user on the governance platform. The alarm rules support multiple conditional logic combinations, including time window constraints (such as "continuous over-limit within the past 10 minutes"), cross-node status association (such as "the current node's indicators are abnormal and its direct upstream task execution status is successful"), threshold comparison, and alarm level setting. The matching process is completed by the rule engine based on the CEP (Complex Event Processing) model.

[0055] When a match is successful, the system generates a structured alarm event object, whose fields include: unique event identifier (UUID), trigger time (millisecond-level timestamp), unique alarm rule identifier, critical node path (directed path tracing up from the current abnormal node to the root node), data object identifier, current metric value, alarm threshold, and alarm level (P1–P3).

[0056] Furthermore, the aggregated metrics results are written to a time-series database, alarm events are persistently stored in the time-series database, and simultaneously distributed to multiple notification channels, including: a visual dashboard (which uses color to identify abnormal nodes and drills down to display details), enterprise instant messaging tools, SMS gateways, and email services.

[0057] The visual dashboard requires the following steps: (a) Obtain the data chain topology graph.

[0058] (ii) Retrieve alarm events corresponding to at least one target node in the data chain topology graph to determine the health status of the target node.

[0059] The specific process is as follows: (1) Query all alarm events generated by the target node within the specified time range from the event database; (2) Perform the following statistics and judgments on the queried alarm events: count the total number of alarms; classify and count the number of alarms of each level (P1, P2, P3) according to the level; extract the trigger time of the most recent alarm; count the number of alarms that are still in the "not closed" state; (3) make a comprehensive judgment on the statistical results obtained in step (2) according to the preset alarm-driven health status mapping rules, and generate a health status level. The mapping rules are deterministic logic. For example, if there is any P1 level not closed alarm, the health status is "abnormal"; if there is no P1 alarm but the number of P2 not closed is ≥3, the health status is "warning"; if all alarms have been closed and there are no new alarms in the past 24 hours, the health status is "normal"; (4) encapsulate the health status level, key judgment basis and not closed alarm summary into a structured health status object and return it to the caller.

[0060] (iii) Based on the visual effects of health status matching, the data chain topology map is rendered to obtain the data chain status map, which is then displayed through a visualization dashboard. The data chain status map is used to display the health status of the target node in the data chain topology map with different visual effects.

[0061] In one implementation, (1) the topology graph structure data of the data chain to be rendered is obtained, including all nodes and their directed connections; (2) for each target node in the topology graph, its current health level is queried; (3) according to the health status level, a corresponding visual style is assigned to each target node: the “normal” node icon is filled with green and displays the health score; the “warning” node icon is filled with yellow and overlaid with a slow pulse effect; the “abnormal” node icon is filled with red and overlaid with a high-frequency flashing effect and a warning badge; (4) according to the connection relationship between nodes, the state transmission mark is applied to the directed edge: when the upstream node is “abnormal” and the downstream node is “normal”, a red dashed arrow is overlaid on the edge to indicate the potential data pollution path; (5) the topology graph data with completed style mapping is serialized according to the standard format (JSONSchema) supported by the front-end visualization framework and pushed to the visualization dashboard through RESTAPI; (6) the visualization dashboard calls the graphics rendering engine to parse the data and draws a data chain status graph with interactive capabilities in real time, supporting node click drill-down, path highlighting and timeline backtracking operations.

[0062] (iv) In response to the selection operation of any target node in the data link situation diagram, display the historical trend chart of the indicator and / or the list of alarm events of the target node corresponding to the selection operation. The historical trend chart of the indicator is generated based on the aggregated results of the indicator of the target node at multiple historical times, and the list of alarm events is generated based on the alarm events of the target node at multiple historical times.

[0063] In one specific implementation: (1) In response to the user's click or hover operation on any target node in the data chain status map in the visualization dashboard, the front end initiates a health details query request to the back end service, and the request carries the node identifier of the target node and optional time range parameters; (2) The backend service executes the following two sub-processes in parallel: The sub-process for generating historical trend charts of indicators is as follows: The system retrieves the aggregated indicator results of the target node within a specified time range from the time-series database at a fixed time granularity (e.g., every minute, every 5 minutes); the retrieved data is sorted by timestamp and packaged into a time-series dataset. Alarm event list generation sub-process: Query all alarm events generated by the target node within the same time range from the event database, and sort them in reverse order of trigger time; (3) The backend returns the two types of structured data to the frontend respectively; (4) The front end calls the chart rendering component (such as ECharts or AntDesignCharts) to draw the time series dataset as a trend line chart with multiple indicators superimposed, and supports legend switching, time axis zooming and indicator drill-down; at the same time, the alarm event list is displayed in table form, and supports filtering by level, status filtering and viewing event details pop-up window; (5) Users can perform linked operations through the same interface: when a certain abnormal peak point is clicked in the trend chart, the alarm event in the corresponding time window will be automatically highlighted; when an event is clicked in the alarm list, the trend chart will be automatically located and zoomed to the trigger time.

[0064] Step 3: Intelligent diagnosis and automated repair triggering. The quality repair system also includes a repair engine, which deploys a repair rule knowledge base. Essentially, it is a scalable structured rule database, and each rule consists of the following three parts: Triggering condition: A logical expression used to match alarm events, which is constructed based on the attribute fields of the alarm event, including but not limited to alarm event identifier, key node identifier, node type, metric name, alarm level and business domain label; Repair Actions: Define the specific automated operations to be performed after matching the trigger conditions, and encapsulate them in the form of scripts or workflow templates; support multiple action types, including: (1) SQL execution class: execute predefined SQL statements; (2) Task rerun class: call the task scheduling system API to trigger the re-execution of the specified task instance; (3) Message notification class: call the external system API to send alarm collaboration requests to the specified responsible person; Execution parameters and security context: Define the runtime parameters (such as database connection strings, API access tokens, business date variables) and isolated execution environment (such as limiting CPU / memory resources and Docker containers that only grant write permissions to the target table) necessary for the execution of the repair action to ensure the security and controllability of the action execution.

[0065] Based on this, the specific process of automated repair is as follows: Step 3.1: Match the alarm event with the alternative repair rules stored in the repair rule knowledge base, and execute steps 3.2 or 3.3 to S3.5 according to the matching degree.

[0066] In one implementation, the process of the rule matching engine calculating the matching degree includes the following two stages: Phase 1: Syntax Matching and Filtering The engine parses the trigger condition expression of each rule and extracts the structured constraints, including: node type field values ​​(such as "data cleaning", "data service"); label name field values ​​(such as "field null value rate", "processing delay"); and business domain label field values ​​(such as "payment", "user profile"). Perform exact matching or wildcard matching on the corresponding fields in the alarm event (such as node type == data cleaning, or indicator name LIKE '%null rate%') to filter out all candidate rules that meet the basic constraints.

[0067] Phase Two: Semantic Similarity Ranking For the candidate rule set, the engine calculates a weighted Jaccard similarity based on the following three dimensions of the alarm event: (1) Node type matching degree (weight 0.4): 1.0 if they are completely equal, otherwise 0; (2) Normalized score of indicator name edit distance (weight 0.4): string similarity is calculated using Levenshtein distance and normalized to the [0,1] interval; (3) Business domain label overlap degree (weight 0.2): 1.0 if the labels are completely the same, otherwise 0. The weighted sum of the three items is taken as the final matching degree score, and the rule with the highest score and not lower than the threshold (0.75) is selected as the target repair rule.

[0068] Step 3.2: If there are alternative repair rules with a matching degree greater than the preset matching degree threshold, then use them as the target repair rule corresponding to the alarm event; initialize the repair container; load the repair execution script in the repair container, configure the repair execution script according to the security context, and perform repair actions on the instances corresponding to the critical nodes based on the configured repair execution script.

[0069] After an alarm event is successfully matched with a target repair rule, the system generates a structured repair work order, initializes its status to "pending execution", and establishes a bidirectional association with the alarm event and the matched target repair rule. The repair engine periodically polls the work order pool and pulls repair work orders with a status of "pending execution". The repair engine runs in a secure sandbox environment. Based on the security context specified in the work order, it initializes a resource-constrained temporary execution container (e.g., CPU quota ≤ 500m, memory limit ≤ 512MB, network policy prohibits external connections), loads the repair action script and preset parameters into the container, and performs the repair operation. The entire execution process is fully recorded in the audit log, including: container startup time, script execution commands, standard output (stdout), standard error (stderr), exit code, execution time, and peak resource usage.

[0070] Step 3.3: If the matching degree between the alarm event and any alternative repair rule stored in the repair rule knowledge base is less than the preset matching degree threshold, determine the upstream node set and downstream node set of the key node based on the data chain topology graph.

[0071] For alarm events that do not match the target repair rule, the system initiates an auxiliary diagnostic process. The auxiliary diagnostic process is based on the data chain topology graph, locates the key nodes associated with the alarm event, and queries all its direct upstream and downstream nodes in the topology. Furthermore, it pulls the aggregated index results of the above nodes within a preset time window (e.g., ±5 minutes) before and after the alarm event trigger time from the time series database as the data basis for auxiliary diagnosis.

[0072] Step 3.4: Perform root cause analysis based on the data quality indicators corresponding to the key nodes, upstream node set, and downstream node set to obtain the root cause nodes.

[0073] In one implementation: (1) Filter out nodes that have experienced anomalies (i.e., meet the preset alarm conditions) and have not recovered before the alarm trigger time; (2) Among the nodes that meet the aforementioned conditions, select the node that experienced the anomaly earliest in time as the candidate root cause node; (3) If multiple nodes are abnormal at the same earliest time point, further compare the business semantic consistency between their abnormal indicators and the current alarm indicators: (3.1) If the abnormal indicator of an upstream node is the field X null value rate, and the current alarm indicator is also the field X null value rate, then determine that the upstream node is the root cause node; (3.2) If the abnormal indicator of an upstream node is the total number of records fluctuation rate, and the current alarm indicator is the processing delay, then exclude it as the root cause node; If a unique node is found through the above determination, it is identified as the root cause node; if no unique node is found, all nodes that satisfy (1) and (2) are combined into a root cause candidate set.

[0074] Step 3.5: Based on the alarm event, key nodes and their corresponding data quality indicators, root cause nodes and their corresponding data quality indicators, generate a diagnostic report and send the diagnostic report to the designated associated terminal. In practical applications, extract the alarm event, the aggregated results of its associated key node indicators, and the aggregated results of the root cause node indicators determined by root cause analysis; encapsulate the above information into a diagnostic report according to a preset structure, including an alarm summary, root cause location, indicator comparison, and handling suggestions; if the handling suggestions in the diagnostic report are not automatically executed, or the alarm level is P1 and the root cause analysis does not yield a unique conclusion, then the alarm event, the diagnostic report, and the rule recommendations based on matching similar historical cases are all upgraded and assigned to a preset manual processing team or responsible person.

[0075] Step 4: Verify the repair effect and optimize the closed-loop feedback.

[0076] Acquire new data quality indicators collected by the quality metric probe within a preset observation time window, with the start time of the observation time window being the completion time of the repair action; generate new indicator aggregation results corresponding to the new data quality indicators according to the indicator aggregation rules; and determine the new data quality indicators based on the indicator alarm rules: if the new data quality indicators meet the preset alarm conditions, the repair is determined to be successful; if the new data quality indicators do not meet the preset alarm conditions, the repair is determined to be unsuccessful.

[0077] In one implementation, after the repair action is completed, the automatic execution engine reports an "execution completed" status to the system. In response to this status, the effect verification module is triggered and started. The effect verification module performs the verification operation with a delay based on a preset observation time window: for task rerun-type repairs, the observation time window is 5 minutes; for data update-type repairs, the observation time window is 1 minute. After the observation time window ends, the effect verification module calls the quality metric probe (or its homologous probe) associated with the original alarm to re-collect the aggregated metric results of the key node; the newly collected metric values ​​are then compared with the preset alarm bar. The threshold values ​​in the sample are compared to generate verification results: if the indicator value meets the preset alarm conditions, the verification result is "successful"; if the indicator value improves compared to before the repair but still does not meet the preset alarm conditions, the verification result is "partially successful"; if the indicator value does not improve or worsens, the verification result is "failed". The verification results, the snapshot of the indicator before the repair, and the snapshot of the indicator after the repair are written into the corresponding repair work order and the original alarm event record to complete a single governance loop. Regardless of the verification result, the complete closed-loop case is persisted to the case library in a structured format as the data basis for subsequent optimization analysis.

[0078] Furthermore, the case library serves as the data foundation for system feedback optimization. Each record is a structured closed-loop case, containing the original alarm event, the matching target repair rule, the repair work order, the execution process log, the effect verification results, and snapshots of indicators before and after the repair. The system is configured with an offline optimization analysis service that extracts historical closed-loop cases from the case library at preset intervals (e.g., daily) for statistical analysis and pattern mining. The optimization analysis service performs the following analysis tasks: (1) Perform cluster analysis on the multidimensional attributes of alarm events (including key node identifier, indicator name, alarm level, trigger time, and business domain label) to identify high-frequency co-occurring alarm clusters and generate suggestions for common problem patterns to be addressed; (2) Based on the time sequence relationship of alarm events, the association rule mining algorithm is used to discover the time sequence dependency relationship across indicators and nodes. For example, "when indicator A is abnormal at node X, the probability that indicator B will be abnormal at node Y after 10 minutes is ≥80%", and the association warning rule suggestions are generated accordingly. (3) Perform distribution analysis on the historical trigger records of static threshold alarms. If the number of times a certain indicator fluctuates within the threshold ±5% range within the preset period exceeds 40% of the total number of threshold triggers, it is recommended to update its threshold to the 95th percentile value of the historical data of the indicator in the past 30 days. (4) Statistically analyze the success rate and average repair time of each repair rule in historical closed-loop cases, sort them in descending order of success rate, and mark the rules with a success rate of less than 50% and no improvement for 3 consecutive times as inefficient rules; All analysis results are output in the form of structured reports and pushed to the data governance platform dashboard. After review and confirmation by governance personnel, they are used to update preset alarm conditions, expand the knowledge base of repair rules, or adjust the health calculation model.

[0079] In summary, the quality restoration method for enterprise-level data provided in this embodiment of the invention has at least the following characteristics: First, it enables real-time and proactive data quality management: by deploying lightweight quality metric probes at key nodes in the data chain and having the streaming engine consume the indicator event streams reported by the probes in real time, the detection latency of quality problems is effectively shortened; the system generates early warnings based on indicator time series trend analysis, so that abnormal interventions can occur before business impacts occur, reducing the risk of data service interruption.

[0080] Second, improve problem repair efficiency and reduce manual operation and maintenance costs: By working together with the repair rule knowledge base and the automatic execution engine, the repair actions are automatically matched and executed for alarm events that meet the trigger conditions, covering ruleable scenarios such as field null value completion, task failure rerun, and data consistency calibration; reduce manual diagnosis and operation steps, so that data governance personnel can focus on unstructured and highly complex data quality issues.

[0081] Third, construct an autonomous closed loop for data quality governance: the system forms a six-stage closed loop of measurement, analysis, decision-making, execution, verification, and optimization. Among them: the quality measurement probe and streaming engine complete the measurement and real-time analysis; the rule matching engine and automatic execution engine complete the decision-making and execution; the repair effect verification module completes the post-execution verification; and the case library and optimization analysis service, based on historical closed-loop cases, periodically provide feedback and optimization on preset alarm conditions, repair rule knowledge base, and health calculation model.

[0082] Fourth, enhance the observability of data governance and data-driven decision-making capabilities: The quality control center presents a unified data chain topology diagram, the health status of each key node, historical trends of indicators, and a list of alarm events through a visual dashboard, supporting drill-down analysis and cross-node linkage positioning; based on the closed-loop governance data accumulated in the case library, it generates quality trend reports and rule effectiveness analysis, providing quantitative basis for setting data governance priorities, allocating resources, and optimizing threshold strategies.

[0083] Based on the foregoing embodiments, this invention provides a quality repair device for enterprise-level data. This device is applied to a quality repair system. See [link to relevant documentation]. Figure 3 The schematic diagram shown illustrates the structure of a quality repair device for enterprise-level data, comprising: Graph construction module 302 is used to construct a data chain topology graph. The data chain topology graph contains multiple key nodes whose corresponding instances are embedded with quality metric probes. The quality metric probes are used to collect data quality indicators corresponding to the key nodes. The repair module 304 is used to generate an alarm event corresponding to the data quality indicator if the data quality indicator meets the preset alarm conditions, and to perform repair actions on the instance corresponding to the key node based on the target repair rule in the repair rule knowledge base that matches the alarm event. The effect verification module 306 is used to verify the repair effect based on the data quality indicators collected by the quality measurement probe after the repair action, and to update the preset alarm conditions and / or repair rule knowledge base to continue to achieve data quality repair.

[0084] The quality repair device for enterprise-level data provided in this embodiment of the invention constructs a data link topology diagram and embeds quality measurement probes at the runtime instances of key nodes within it, thereby achieving real-time, low-overhead collection of data quality indicators. Combined with streaming alarm judgment, instance-level automated repair execution, and immediate effect verification of the same probe after repair, the system ensures that it can complete the entire process from anomaly identification to result confirmation without manual intervention, thus realizing automatic response and closed-loop repair of data quality alarms.

[0085] The device provided in this embodiment of the invention has the same implementation principle and technical effect as the aforementioned method embodiment. For the sake of brevity, any parts not mentioned in the device embodiment can be referred to the corresponding content in the aforementioned method embodiment.

[0086] This invention provides an electronic device, specifically, the electronic device includes a processor and a memory; the memory stores a computer program, which, when run by the processor, executes the method described in any of the above embodiments.

[0087] Figure 4 The present invention provides a schematic diagram of the structure of an electronic device 100, which includes a processor 40, a memory 41, a bus 42 and a communication interface 43. The processor 40, the communication interface 43 and the memory 41 are connected through the bus 42. The processor 40 is used to execute executable modules, such as computer programs, stored in the memory 41.

[0088] The memory 41 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface 43 (which can be wired or wireless), such as the Internet, wide area network, local area network, metropolitan area network, etc.

[0089] Bus 42 can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 4 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0090] The memory 41 is used to store programs. After receiving an execution instruction, the processor 40 executes the program. The method executed by the device for defining the flow process disclosed in any of the foregoing embodiments of the present invention can be applied to the processor 40 or implemented by the processor 40.

[0091] Processor 40 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of processor 40 or by instructions in software form. Processor 40 can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this invention can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 41. The processor 40 reads the information in memory 41 and, in conjunction with its hardware, completes the steps of the above method.

[0092] The computer program product of the readable storage medium provided in the embodiments of the present invention includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods described in the foregoing method embodiments. For specific implementation, please refer to the foregoing method embodiments, which will not be repeated here.

[0093] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0094] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for quality restoration of enterprise-level data, characterized in that, Applied to quality repair systems, including: A data chain topology graph is constructed, in which quality metric probes are embedded at instances corresponding to multiple key nodes. These quality metric probes are used to collect data quality indicators corresponding to the key nodes. If the data quality indicator meets the preset alarm conditions, an alarm event corresponding to the data quality indicator is generated, and a repair action is performed on the instance corresponding to the key node based on the target repair rule in the repair rule knowledge base that matches the alarm event. The repair effect is verified based on the data quality indicators collected by the quality metric probe after the repair action, and the preset alarm conditions and / or the repair rule knowledge base are updated to continue to achieve data quality repair.

2. The method for quality restoration of enterprise-level data according to claim 1, characterized in that, The quality repair system includes a quality control center, which deploys an indicator aggregation rule base and an indicator alarm rule base. If the data quality indicator meets preset alarm conditions, an alarm event corresponding to the data quality indicator is generated, including: The data quality indicators are encapsulated into event messages, which are then asynchronously sent to the quality control center using a message middleware. The quality control center performs the following operations on the event message: Parse the data quality indicators corresponding to the key nodes from the event messages; The indicator aggregation rules that match the key node are determined from the indicator aggregation rule base, and the indicator aggregation results corresponding to the data quality indicators are generated according to the indicator aggregation rules. The system determines the indicator alarm rule that matches the key node from the indicator alarm rule base, and generates an alarm event corresponding to the data quality indicator when the data quality indicator meets the preset alarm conditions based on the indicator alarm rule.

3. The quality restoration method for enterprise-level data according to claim 2, characterized in that, The quality repair system also includes a visualization dashboard, and the method further includes: Obtain the data chain topology graph; The alarm events corresponding to at least one target node in the data chain topology graph are retrieved to determine the health status of the target node. Based on the visual effects matched by the health status, the data chain topology graph is rendered to obtain a data chain status graph, which is then displayed through the visualization dashboard. The data chain status graph is used to display the health status corresponding to the target node in the data chain topology graph with different visual effects.

4. The quality restoration method for enterprise-level data according to claim 3, characterized in that, The method further includes: The visualization dashboard, in response to the selection operation of any of the target nodes in the data chain situation diagram, displays the historical trend chart of the indicators and / or the list of alarm events of the target node corresponding to the selection operation; The historical trend chart of the indicators is generated based on the aggregated results of the indicators of the target node at multiple historical times, and the list of alarm events is generated based on the alarm events of the target node at multiple historical times.

5. The method for quality restoration of enterprise-level data according to claim 1, characterized in that, The quality repair system also includes a repair engine, which has a repair rule knowledge base deployed within it. Based on the target repair rule in the repair rule knowledge base that matches the alarm event, repair actions are performed on the instance corresponding to the key node, including: The alarm event is matched with the candidate repair rules stored in the repair rule knowledge base, and the candidate repair rule with a matching degree greater than a preset matching degree threshold is taken as the target repair rule corresponding to the alarm event; wherein, the candidate repair rule and the target repair rule both include rule triggering conditions, repair execution script and security context required by the repair execution script; Initialize the repair container; The repair execution script is loaded into the repair container, configured according to the security context, and repair actions are performed on the instance corresponding to the critical node based on the configured repair execution script.

6. The method for quality restoration of enterprise-level data according to claim 5, characterized in that, The method further includes: If the matching degree between the alarm event and any of the alternative repair rules stored in the repair rule knowledge base is less than the preset matching degree threshold, the upstream node set and downstream node set of the key node are determined based on the data chain topology graph. Root cause analysis is performed based on the data quality indicators corresponding to the key nodes, the upstream node set, and the downstream node set to obtain the root cause nodes; Based on the alarm event, the key node and its corresponding data quality index, the root cause node and its corresponding data quality index, a diagnostic report is generated and sent to the designated associated terminal.

7. The method for quality restoration of enterprise-level data according to claim 2, characterized in that, The repair effect is verified based on the data quality indicators collected by the quality metric probe after the repair action, including: The new data quality index collected by the quality measurement probe within a preset observation time window is obtained, wherein the start time of the observation time window is the completion time of the repair action. According to the aforementioned index aggregation rules, generate new index aggregation results corresponding to the new data quality index; Based on the aforementioned indicator alarm rules, the new data quality indicator is judged: if the new data quality indicator meets the preset alarm conditions, the repair is determined to be successful; if the new data quality indicator does not meet the preset alarm conditions, the repair is determined to be unsuccessful.

8. A quality repair device for enterprise-level data, characterized in that, Applied to quality repair systems, including: The graph construction module is used to construct a data chain topology graph. The data chain topology graph contains multiple key nodes, and the corresponding instances are embedded with quality metric probes. The quality metric probes are used to collect data quality indicators corresponding to the key nodes. The repair module is used to generate an alarm event corresponding to the data quality indicator if the data quality indicator meets the preset alarm conditions, and to perform repair actions on the instance corresponding to the key node based on the target repair rule in the repair rule knowledge base that matches the alarm event. The effect verification module is used to verify the repair effect based on the data quality indicators collected by the quality metric probe after the repair action, and to update the preset alarm conditions and / or the repair rule knowledge base to continue to achieve data quality repair.

9. An electronic device, characterized in that, The method includes a processor and a memory, the memory storing computer-executable instructions executable by the processor, the processor executing the computer-executable instructions to implement the method of any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions that, when invoked and executed by a processor, cause the processor to perform the method according to any one of claims 1 to 7.

Images