Data quality rule automatic generation and self-adaption method, system, equipment and medium

Abstract

The invention relates to the technical field of data processing and artificial intelligence application, and discloses a data quality rule automatic generation and self-adaption method, system and device and a medium. Performing business term inference and semantic modeling on the structured metadata to construct business concepts and relationships; analyzing the processing logic of the structured metadata based on the business concept and the relationship to identify a data consanguinity map; building a power grid domain knowledge graph based on a business concept, a semantic model and a consanguinity graph mining hidden relationship; reasoning based on the power grid domain knowledge graph and historical data to generate data quality rules and business constraints; monitoring the execution effect of the data quality rule, and performing adaptive optimization through quantitative evaluation; and executing the self-adaptively optimized rule, marking abnormal data and feeding back an execution result. According to the method, the automation and intelligence level of quality treatment can be improved, and continuous evolution and vitality of a data quality system are ensured.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing and artificial intelligence application, and in particular to a data quality rule automatic generation and self-adaption method, system, device and medium. BACKGROUND

[0002] In today's digital transformation wave, data has become the core strategic asset of enterprises. Data governance, especially data quality management, is the key link to ensure data reliability, security and value realization. Currently, the mainstream technology and method in the field of data quality management mainly relies on rule engine-based verification, automatic data cleaning tools and manually driven governance processes. Rule engine scans and monitors data through pre-defined data quality rules, and automatic tools are responsible for executing pre-set cleaning logic, while manual intervention and judgment are still indispensable in complex business scenarios.

[0003] However, with the explosive growth of data size, the increasing diversification of data sources and the increasing demand for real-time data by businesses, the existing technical system exposes a series of inherent limitations. First, it highly depends on manual experience for rule definition, maintenance and update, resulting in low efficiency, high cost and difficulty in coping with dynamic changes in data environment. Second, traditional rule engines are usually static and rigid, and once the rules are deployed, they cannot be automatically adjusted with the evolution of business logic or data model, lacking self-adaptability and high maintenance cost. Third, existing tools lack understanding of deep business semantics and contextual relationships of data, making it difficult to intelligently identify quality problems hidden in complex associations and unable to automatically generate and optimize rules. In addition, various data governance function components are often independent and scattered, lacking a unified intelligent collaborative management mechanism, making it difficult to form a governance force and easy to form data silos. Finally and most importantly, the existing technology lacks a continuous, closed-loop feedback optimization mechanism, which cannot learn from the actual execution effect of the rules and drive the self-evolution and optimization of the rules, making it difficult to continuously improve the data quality governance capability. The defects of the existing technology seriously restrict the full release of the value of enterprise data assets, and therefore there is an urgent need for a new, highly intelligent and self-adaptive solution. SUMMARY

[0004] In view of the above existing problems, the present application is proposed.

[0005] Therefore, the present application provides a data quality rule automatic generation and self-adaption method, system, device and medium, which solves the technical problems of rule generation relying on manual, rule static rigidity, lack of semantic understanding and closed-loop self-adaption in existing data quality management.

[0006] To solve the above technical problems, the present application provides the following technical solutions:

[0007] In a first aspect, the present application provides a data quality rule automatic generation and self-adaption method, comprising:

[0008] Acquiring structured metadata by actively detecting and analyzing heterogeneous data sources through a metadata acquisition agent;

[0009] Conducting business term inference and semantic modeling on the structured metadata, and constructing business concepts and relationships;

[0010] Analyzing the processing logic of the structured metadata based on the business concepts and relationships to identify a data bloodline graph;

[0011] Mining hidden relationships based on business concepts, semantic models and bloodline graphs, constructing a power grid domain knowledge graph, and generating data quality rules and business constraints based on the power grid domain knowledge graph and historical data;

[0012] Monitoring the execution effect of the data quality rules, and performing self-adaptive optimization through quantitative evaluation, executing the self-adaptive optimized rules, marking abnormal data and feeding back the execution results.

[0013] As a preferred scheme of the data quality rule automatic generation and self-adaption method, the acquisition of structured metadata comprises:

[0014] Actively scanning industrial control networks and listening to service registration centers or API gateways through a power grid data source discovery and monitoring module to identify newly added or changed power grid data sources;

[0015] According to the identified power grid data source type, a corresponding power grid special protocol adapter or general IT connector is called to establish a secure connection;

[0016] After the secure connection is successful, device data dictionaries, database system tables or API interface specifications conforming to industrial standards are parsed for the data format and business meaning of the power grid, and device properties, measurement indicators, business field meanings and their data types, precision and unit information are extracted;

[0017] The extracted metadata is output as a structured metadata dictionary and is preliminarily annotated with power grid domain semantics.

[0018] As a preferred scheme of the data quality rule automatic generation and self-adaption method, the business term inference and semantic modeling of the structured metadata comprises:

[0019] Analyzing the naming conventions, coding patterns, data types and value ranges in the structured metadata to identify the preliminary business categories of data items;

[0020] Clustering algorithms are used to summarize data items with similar patterns to infer the core business concepts and attributes of the power grid; primary and foreign key relationships, reference relationships and numerical dependencies are identified from metadata to construct the structured associations between power grid business concepts;

[0021] The inferred business concepts, attributes, and relationships are constructed into a structured business glossary and updated to the unified knowledge management center.

[0022] As a preferred embodiment of the automatic generation and adaptive method for data quality rules described in this invention, the identification data lineage map includes:

[0023] Parse the configuration information of the data integration tool, database operation logs, or file directory metadata to identify the data flow paths between systems;

[0024] Analyze ETL mapping rules, database view definitions, or calculation logic to track changes in the structure, value, and calculation method of data items during the transfer process;

[0025] The acquired data flow paths and evolution information are associated with power grid business activities to construct a structured data lineage map. The data lineage map is then managed in versions, conflict detection is performed, and incremental updates are carried out. The data lineage map is stored in a knowledge management center.

[0026] As a preferred embodiment of the automatic generation and adaptive data quality rule method described in this invention, the construction of the power grid domain knowledge graph includes:

[0027] Match metadata items with business concept attributes based on similarity in name, data type, value range, and structured location, and establish a mapping relationship;

[0028] By associating data lineage paths with business process operations, business relationships between entities can be inferred.

[0029] Hidden data relationships can be discovered through association rule mining, graph embedding, or inference rules.

[0030] Metadata, business concepts, lineage relationships, and mined data relationships are integrated into a unified knowledge graph for the power grid domain, and stored and managed using a graph database.

[0031] As a preferred embodiment of the automatic generation and adaptive method for data quality rules described in this invention, the generation of data quality rules and business constraints includes:

[0032] Analyze the numerical distribution, update frequency, uniqueness, and completeness characteristics of historical data to infer the reasonable value range, update cycle, and non-empty constraints of data items;

[0033] Infer attribute value constraints and business logic consistency rules in combination with the relationship between the business concept attribute and the power grid domain knowledge graph; check source and target data consistency and conversion logic according to data bloodline information, and generate quality constraint rules in the data flow process;

[0034] Fill the analysis results into a predefined rule template to generate a structured and executable data quality rule list, and assign priority and confidence.

[0035] As a preferred scheme of the data quality rule automatic generation and self-adaptive method, the monitoring of the execution effect of the data quality rule and the self-adaptive optimization through quantitative evaluation include:

[0036] Receive detailed result data generated by rule execution, including rule ID, verification state, violation value and timestamp, perform time window aggregation on the result of rule execution, and calculate the success rate, failure rate, false positive rate and false negative rate of each rule;

[0037] Based on the results of the time window aggregation and the real labels provided by humans or systems, the accuracy, recall rate and coverage rate of the rules are quantitatively evaluated;

[0038] According to the results of the quantitative evaluation, the rules are optimized by using parameter adjustment, priority resetting or rule start-stop strategy, and the optimization suggestions are fed back to the rule generation module.

[0039] In the second aspect, the application provides a data quality rule automatic generation and self-adaptive system, which includes:

[0040] A data acquisition module is configured to actively detect and parse heterogeneous data sources by a metadata acquisition agent to obtain structured metadata;

[0041] A semantic modeling module is configured to infer business terms and perform semantic modeling on the structured metadata to construct business concepts and relationships;

[0042] A bloodline graph identification module is configured to analyze the processing logic of the structured metadata based on the business concepts and relationships to identify a data bloodline graph;

[0043] A rule reasoning module is configured to mine hidden relationships based on business concepts, semantic models and bloodline graphs, construct a power grid domain knowledge graph, infer based on the power grid domain knowledge graph and historical data, and generate data quality rules and business constraints;

[0044] An optimization evaluation module is configured to monitor the execution effect of the data quality rules and perform self-adaptive optimization through quantitative evaluation, execute the rules after the self-adaptive optimization, mark abnormal data and feed back the execution results.

[0045] In a third aspect, the present application provides an electronic device comprising a memory and a processor; the memory is configured to store computer executable instructions, and the processor is configured to execute the computer executable instructions to implement the steps of the data quality rule automatic generation and self-adaptation method.

[0046] In a fourth aspect, the present application provides a computer readable storage medium storing computer executable instructions, and the computer executable instructions are executed by a processor to implement the steps of the data quality rule automatic generation and self-adaptation method.

[0047] Compared with the prior art, the present application has the beneficial effects that: the present application provides a data quality rule automatic generation and self-adaptation method, system, device and medium, and by constructing an autonomous system deeply integrated with an intelligent agent cooperation and management control platform, a fundamental change from manual definition to intelligent creation of data quality rules is realized. The present application can deeply understand data semantics and business association based on a knowledge graph, automatically infer to generate accurate quality rules, and continuously perceive rule execution effects and data environment changes by means of a closed-loop feedback mechanism, so as to realize self-adaptation optimization and dynamic adjustment of the rules, significantly reduce the dependence of data governance on professional talents, improve the automation and intelligent level of quality governance, and ensure the continuous evolution and vitality of the data quality system, thereby providing more reliable and real-time high-quality data support for business decision-making. BRIEF DESCRIPTION OF DRAWINGS

[0048] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0049] Figure 1 The overall flow logic diagram of the data quality rule automatic generation and self-adaptation method provided by an embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0050] In order to make the above-mentioned purposes, features and advantages of the present application more apparent and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the drawings of the specification. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should be within the scope of protection of the present application.

[0051] Embodiment 1, refer to Figure 1 For an embodiment of the present application, a data quality rule automatic generation and self-adaptation method is provided, as shown inFigure 1 The specific steps include the following steps:

[0052] S1: Acquire structured metadata by actively detecting and analyzing heterogeneous data sources through the agent;

[0053] S2: Perform business term inference and semantic modeling on the structured metadata, and construct business concepts and relationships;

[0054] S3: Analyze the processing logic of the structured metadata based on the business concepts and relationships to identify the data bloodline graph;

[0055] S4: Mine hidden relationships based on the business concepts, semantic models and bloodline graph, and construct a power grid domain knowledge graph;

[0056] S5: Perform reasoning based on the power grid domain knowledge graph and historical data to generate data quality rules and business constraints;

[0057] S6: Monitor the execution effect of the data quality rules, and perform adaptive optimization through quantitative evaluation;

[0058] S7: Execute the rules after adaptive optimization, mark abnormal data and feed back the execution results.

[0059] It should be noted that, in order to solve the technical problems of existing data quality management, such as dependence on manual rule generation, static and rigid rules, lack of semantic understanding and closed-loop adaptive optimization, the steps S1-S7 above realize a fundamental change from manual definition to intelligent creation of data quality rules by constructing an autonomous system deeply integrated with an intelligent agent collaborative and management control platform. The present application can deeply understand data semantics and business associations based on a knowledge graph, automatically generate accurate quality rules, and continuously perceive rule execution effects and data environment changes through a closed-loop feedback mechanism, realize adaptive optimization and dynamic adjustment of rules, significantly reduce the dependence of data governance on professional personnel, improve the automation and intelligence level of quality governance, and ensure the continuous evolution and vitality of the data quality system, thereby providing more reliable and real-time high-quality data support for business decision-making.

[0060] In the embodiments of the present application, the above step S1 includes the following sub-steps A1-A4:

[0061] In A1: Through the power grid data source discovery and monitoring module, actively scan the industrial control network and listen to the service registration center or API gateway to identify newly added or changed power grid data sources;

[0062] Specifically, the power grid data source discovery and monitoring module is responsible for actively detecting newly added / changed OT / IT data sources within the power grid (such as newly added substation DCS systems, upgraded SCADA historical databases, newly deployed PMUs or smart meter data concentrators, and newly online business systems), and continuously monitoring the health status and structural changes of existing data sources.

[0063] Specifically, intelligent port scanning is performed on the industrial control network within the power grid (such as IEC61850, ModbusTCP, and DNP3overIP) to discover new RTUs, IEDs, PMUs, DCSs, and other control devices or their data interfaces.

[0064] Specifically, the data asset management platform or CMDB connected to the power grid is synchronized with newly commissioned substation, line, and smart terminal device information, and based on device type, possible data sources are predicted (such as a newly commissioned smart substation usually accompanied by new SCADA telemetry data interfaces).

[0065] Specifically, enterprise-level service buses or API gateways are monitored to sense newly online business system interfaces (such as new marketing system APIs and dispatching management system interfaces). For HDFS data lake storage, new data directories or data lake partitions (such as newly added smart meter raw data directories and line loss data partitions) are sensed through HadoopNameNode event listening. Time series anomaly detection algorithms (such as ARIMA and Prophet) are used to monitor data source online / offline logs and schema change frequency, intelligently identify abnormal patterns (such as a sudden large number of online smart terminals or abnormal fluctuations in SCADA telemetry database connection numbers), and warn of possible unauthorized data source online or accidental offline, thereby triggering deeper detection and impact assessment.

[0066] In A2: According to the identified power grid data source type, the corresponding power grid-specific protocol adapter or general IT connector is called to establish a secure connection.

[0067] Specifically, the heterogeneous connection and protocol adaptation module is used to intelligently select and manage corresponding connectors and credentials based on the discovered power grid data source type, to establish secure and efficient connections with power grid-specific industrial control systems, IT systems, databases, and real-time data buses.

[0068] Specifically, the built-in and supported power grid industry mainstream industrial protocol client, wherein the IEC61850 MMS / GOOSE client is used for connecting intelligent electronic devices (IEDs) conforming to the IEC61850 standard; the Modbus TCP / RTU client is used for connecting traditional measurement and control equipment; the DNP3 over IP client is used for connecting distribution network automation equipment; the OPCUA client is used for uniformly interfacing various industrial equipment; and the general IT connector supports JDBC (relational databases such as Oracle, MySQL), HTTP / gRPC (API interface), Kafka / RabbitMQ (real-time message bus), etc.

[0069] Specifically, the PKI system integrated in the power grid enterprise is used for digital certificate management, or is connected with the IAM system, to securely manage and rotate various data source access credentials, and to ensure the secure connection of the control network.

[0070] In A3: After the secure connection is successful, the device data dictionary, database system table or API interface specification conforming to the industrial standard are parsed according to the data format and business meaning of the power grid, and the device properties, measurement indicators, business field meanings and data types, precision and unit information are extracted;

[0071] Specifically, for the industrial control system connected to the substation and power plant (such as devices conforming to the IEC61850 standard), the intelligent agent can directly parse its internal model or data dictionary (such as SCL file), extract the definitions of various logical nodes, data objects and data properties, including device name, measurement data name (such as voltage, current, frequency, power), state quantity name (such as switch position, protection action state), calculation quantity name (such as line loss, load rate), and their engineering unit, data type, precision, sampling frequency, etc.

[0072] Specifically, from the historical data repository, the storage format, timestamp field, and their identification and meaning in the system of various device operating parameters and measurement data are identified and extracted. The meter number, power consumption data (such as total power, time-of-use power), voltage, current, power factor, event code (such as power failure, alarm), etc. information in the massive smart meter data are automatically identified and parsed.

[0073] Specifically, the table name, column name, data type, constraint, index, etc. metadata are obtained by querying the system table (such as INFORMATION_SCHEMA) of the database. The interface name, request / response parameter, data model definition, etc. are extracted by parsing the API specification document such as OpenAPI / Swagger and WSDL. For undocumented APIs, dynamic API calling and response data analysis can be performed to infer their structure. For structured files (such as Parquet, Avro) in the data lake, the Schema information in the file header is read. For semi-structured (such as JSON, XML) or unstructured files (such as logs), sampling data and machine learning algorithms (such as clustering, pattern matching) are used to automatically infer their internal structure and field types.

[0074] In A4: The extracted metadata is output as a structured metadata dictionary and preliminary power grid field semantic annotation is performed.

[0075] In the embodiments of the present application, the above step S2 includes the following sub-steps B1-B4:

[0076] In B1: The naming convention, coding pattern, data type and value range in the structured metadata are analyzed to identify the preliminary business category of the data item;

[0077] Specifically, the naming convention and coding pattern identification includes deploying string pattern recognition algorithms and dictionary matching techniques in the agent. It analyzes the structured data item name, table name, identifier, etc. provided by S1, for example: it identifies that BSS in T_BSS_MEAS_VOLT represents a substation, MEAS represents measurement, and VOLT represents voltage. The suffixes _CUR, _FREQ, _TEMP usually represent current, frequency, and temperature. 10110 in 10110_S_01 is likely to be a device type code, and S_01 is a state code. Even without explicit text description, through the preset naming specification of the power grid industry and the automatic learning of the coding mapping rules, the preliminary business category of the data item (for example, it is a device measurement value or a business indicator) can be intelligently inferred.

[0078] Specifically, the data type and value range analysis includes: the agent performs statistical analysis on the data type (such as integer, floating point, boolean, timestamp) and value range (for example, the value of a certain field always fluctuates between 0 and 100) of each structured data item provided by S1. The potential business meaning of the data item can be intelligently inferred. For example, a field with a boolean type and a value of 0 / 1 may be inferred to be a switch state in combination with its name; a field with an integer type and a value in the range of 0-24 may represent an hour period. For measurement data of the power grid, it can infer its unit and precision.

[0079] Specifically, the structured association and hierarchy analysis includes: analyzing the relationship between tables, the inclusion relationship between fields, and the hierarchical structure of data organization (such as folder path, database Schema) in the metadata provided by S1. The belonging relationship of data items can be intelligently identified. For example, which device table does a certain measurement data item belong to, and which device table belongs to which substation system. This provides important clues for constructing the hierarchical structure of concepts.

[0080] In B2: using clustering algorithm to generalize data items with similar patterns, and inferring power grid core business concepts and their attributes;

[0081] Specifically, using graph clustering algorithm or clustering algorithm based on attribute similarity, the data items with similar naming patterns, data types and structured associations recognized by the structured metadata pattern analysis module are analyzed. Automatically discover those data item sets that show similar characteristics in structured metadata, which may represent the same business concept. For example, TRF_VOLT_A, MAIN_TRF_VOLT_B and AUX_TRF_VOLT_C, although the names are different, but all contain VOLT and TRF patterns, and the data type is floating point number, the agent can cluster them into transformer voltage measurement value, a core business concept.

[0082] Specifically, from the metadata provided by S1, the key attributes of the concept are inferred. For example, the attributes of the device concept can be obtained from the structured fields such as its belonging device classification code, manufacturer code, rated parameter, etc. The agent can automatically identify and standardize these attributes. For example, it can infer common attributes such as rated voltage and rated current from different device parameter codes, and map different system unit codes (such as KV_UNIT, THOUSAND_VOLTS) to standardized units KV. It can also infer the definition of the concept, for example, by analyzing the structured data related to the meter (such as METER_ID, CONSUMPTION_VALUE), it can infer that the meter is a device with a unique identifier and can record power consumption.

[0083] In B3: identifying primary foreign key relationships, reference relationships and numerical dependencies from metadata to construct structured associations between power grid business concepts;

[0084] Specifically, the primary-foreign key and reference relationship inference includes: the agent deeply analyzes the structured database Schema (including table, column, index, constraint information) provided by S1, and automatically identifies the explicitly defined primary-foreign key relationship. At the same time, even if there is no explicit primary-foreign key definition, the agent can intelligently infer the potential reference relationship through high similarity of column names, same data types, and high overlap of data value distribution (for example, the DEVICE_ID field in one table and the MAIN_ID field in another table are highly correlated in data values, and both have uniqueness, so there may be a reference relationship). It can identify the structured association between concepts such as devices and measurement data, and regions and devices, for example, the DEVICE_ID field in a certain measurement data table references the ID field in the device master data table, thereby establishing the relationship that the measurement data belongs to the device.

[0085] Specifically, the numerical dependency and behavior association inference includes: the agent analyzes the structured data itself. For example, through regression analysis, correlation analysis, or time series synchronization analysis, it discovers statistical dependencies or behavior patterns between related data values of different concepts. It automatically discovers the implicit association relationship between data items or concepts. For example, the agent may find that when a certain device shows EVENT_CODE_301 (indicating overload) in the structured log, the associated temperature measurement data field value will always rise, thereby inferring that there is an influence or causal relationship between the overload event and the device temperature, even if this relationship is not defined in any explicit structure.

[0086] Specifically, the spatial topology and hierarchical relationship construction includes: from the structured geographic information data (such as GIS coordinates, line connection point codes) and device hierarchical codes provided by S1, the spatial topology relationship and business hierarchical relationship of power grid devices and regions are automatically constructed. It can intelligently identify the connection between substations and lines, the inclusion between lines and power grid regions, and other relationships, and construct a clear power grid topology knowledge structure.

[0087] In B4: the inferred business concepts, attributes, and relationships are constructed into a structured business vocabulary table, and are updated to the unified knowledge management center;

[0088] Specifically, the inferred and defined power grid business concepts, their attributes, and standardized descriptions are constructed into a purely structured business vocabulary table. This vocabulary table is a dynamically evolving knowledge base that can automatically identify and label newly discovered business concepts and add them to the vocabulary table. It can automatically maintain the hierarchical relationship between concepts, for example, a transformer is a kind of power device, thereby forming a structured classification system.

[0089] Specifically, the feedback-driven self-optimization includes that the agent receives the correction feedback from the user (for example, the user corrects the attributes or relationships of a certain concept in a structured manner) and the rule execution effect feedback of S5 / S6 / S7 (for example, the quality rules generated based on a certain business concept always misreport). Through the reinforcement learning or supervised learning algorithm, the pattern recognition model, the clustering algorithm and the relationship inference algorithm in the agent are continuously optimized, so that the agent is more accurate in future knowledge construction. For example, when it is found that the attribute inference of a certain concept is often corrected by the user, the agent can learn better attribute inference rules.

[0090] Specifically, the few-shot learning and transfer learning include that when encountering a completely new structured pattern or encoding in the power grid field that has not been identified, the agent can request an expert to provide a small amount of structured examples (such as the encoding mapping of a few key data items). Using the few-shot learning technology, the agent can quickly learn new concept inference or relationship extraction patterns from these few structured samples, and apply them to similar new scenarios, greatly accelerating the acquisition of new business knowledge.

[0091] Specifically, the active learning and confidence evaluation include that the agent calculates a confidence score for each concept or relationship when inferring and defining knowledge. When the confidence is low (for example, there is insufficient structured evidence or ambiguity), the agent can actively request structured verification or supplementary information from the user. This active learning mechanism enables the agent to more effectively utilize expert knowledge while reducing the propagation of incorrect knowledge.

[0092] In the embodiment of the present application, the above step S3 includes the following sub-steps C1-C3:

[0093] In C1, the configuration information of the data integration tool, the database operation log or the file directory metadata are parsed, and the inter-system data flow path is identified;

[0094] Specifically, the agent receives the structured metadata about the system interface collected in S1 (for example, the configuration information of the data integration platform, the job definition of the ETL tool, the replication configuration of the database, and the interface call record of the application system). These records clearly show which system sends which data of which structure to which system. These configuration information can be intelligently parsed to construct a pipeline diagram of data flow. For example, it is identified that the real-time monitoring system writes the device running measurement data into a specific table structure of the historical database through configuration.

[0095] Specifically, the agent analyzes the structured operation logs (such as transaction logs, audit logs) of the database system itself. These logs record in detail which user / application, at what time, performed which structured operations (insert, update, delete) on which fields of which table. Through pattern recognition of the logs, the agent intelligently infers the internal path of data flowing from one table to another. For example, if it is found that a certain application always reads data from a raw measurement data table and writes to an aggregated calculation result table, it can intelligently identify this as a data conversion path even without explicit configuration.

[0096] Specifically, the agent analyzes the structured file directory metadata provided by S1, identifying whether different systems access the same structured data files (such as CSV, Parquet files). By matching structured identifiers (such as device IDs, timestamps) in the file content, it infers data sharing and flow. It intelligently identifies that data flows between systems through shared files or shared storage even without direct interfaces, and can associate the structured sources of these shared data.

[0097] In C2: Analyze ETL mapping rules, database view definitions, or calculation logic to track changes in structure, values, and calculation methods of data items during flow;

[0098] Specifically, structured mapping relationship analysis includes: the agent analyzes the structured mapping rules defined in data integration tools (for example, field mapping configurations in ETL tools, view definitions in data warehouses). These rules explicitly show how source data items are converted into target data items (such as renaming, type conversion, merging and splitting). It can automatically parse and record these explicit structured evolution processes, for example, identifying that raw voltage values (floating point numbers) are converted to standardized voltage values (fixed point numbers).

[0099] Specifically, calculation logic inference and formula inversion includes: the agent analyzes the calculation process stored in the database (such as the structured definition of stored procedures, the SQL statement of database views), or obtains the structured calculation rule definition from specific systems (such as power grid dispatching calculation platform). It intelligently infers that the target data item is derived from which source data item through which structured calculation logic (such as addition, subtraction, multiplication, division, aggregation, conditional judgment). For example, it identifies that line loss rate is calculated from the power supply and power consumption through the structured formula ((power supply-power consumption) / power supply) 100%. For complex structured calculations, it can try to reverse the input data items.

[0100] Specifically, the data quality rule impact analysis includes: the agent receives the output of S5 (quality rule intelligence generation) and S6 (quality assessment and reporting). When some data is modified or discarded because it does not meet the structured quality rules generated by S5, S3 can receive this structured processing information. The impact of data quality processing on data evolution can be intelligently identified, so as to mark the data path that is cleaned, corrected, discarded, and perfect the blood relationship information.

[0101] In C3: associate the obtained data flow path and evolution information with the power grid business activities, construct a structured data blood relationship map, manage the version of the data blood relationship map, detect conflicts and update incrementally, and store in the knowledge management center.

[0102] Specifically, the mapping based on S2 business knowledge includes: the agent uses the structured power grid business knowledge graph constructed by S2 (for example, it is known that the voltage measurement value is related to the device state monitoring concept). By structuring the matching of data items and business concepts defined by S2. It can be intelligently inferred which power grid business functions a specific data item mainly serves. For example, all device operation parameter related data blood relationships are associated with device health management business activities.

[0103] Specifically, the system function and data interaction mode analysis includes: the agent analyzes the system function list provided by S1 and the system inter-data interaction records related to the function. The structured relationship between the life cycle of a specific data item and the core business processes of the power grid (such as dispatching instruction execution, power energy metering, device fault repair) is intelligently identified. For example, if it is found that the blood relationship chain of the power failure instruction data is finally connected to the line switch action data, it can be inferred that these data are closely related to the power failure and power restoration business scenario.

[0104] Specifically, the timing and event association includes: the agent analyzes the structured data stream with time stamp, and combines the structured information of power grid event records (such as alarm events, operation logs). It intelligently identifies which data flow and evolution within a specific time window are structurally synchronized with a specific power grid event or business activity. For example, it is found that the device protection action data is generated immediately after the device fault alarm event occurs, and a business association between them is established.

[0105] Specifically, data items, systems, business activities, transformation logic, etc. are taken as nodes, and their flows, evolutions, correlations, etc. are taken as edges to construct a purely structured graph database model. Each update of the data lineage graph is managed by version, and an incremental update mechanism is supported, only recording and transmitting the changed part. The real-time and historical traceability of the lineage graph are ensured. When the structured metadata or business knowledge reported by S1 / S2 changes, the agent can automatically adjust and update the affected part of the lineage graph. The agent deploys a conflict detection algorithm based on graph theory. When a cycle, a broken link, or inconsistent structured transformation logic appears in the lineage chain, it can automatically identify and mark these problems. For structured conflicts that can be automatically repaired (such as redundant paths), the agent can intelligently optimize and correct them; for those that cannot be automatically repaired, it reports to the MCP for manual intervention.

[0106] Specifically, the agent receives feedback from users for correction (for example, users correct a certain lineage path or transformation logic in a structured manner), and feedback from S5 / S6 / S7 on the effect of rule execution (for example, the quality rules generated based on a certain lineage path always miss). Through reinforcement learning or supervised learning algorithms, the agent continuously optimizes its internal path inference model, transformation logic inversion algorithm, and business correlation algorithm, making it more accurate in future lineage construction. For example, when it is found that a certain data transformation inference is often corrected by users, the agent can learn better transformation identification rules. The agent generalizes the structured lineage patterns that have been successfully constructed. For example, it identifies that all data flows from real-time systems to historical databases follow a certain specific structured transformation pattern. When encountering new, unknown structured data flows, the agent can intelligently apply and extend the similar patterns it has learned to quickly construct their lineages and reduce repeated analysis.

[0107] Specifically, active learning and confidence assessment include: the agent calculates a confidence score for each inferred path or transformation logic when constructing the lineage. When the confidence is low (for example, there is insufficient structured evidence or ambiguity), the agent can actively request structured verification or supplementary information from the user. This active learning mechanism enables the agent to more effectively utilize expert knowledge while reducing the propagation of incorrect lineage information.

[0108] In the embodiment of the present application, the above step S4 includes the following sub-steps D1-D4:

[0109] In D1: similarity matching of the name, data type, value range, and structured location of the metadata item and the business concept attribute is performed, and a mapping relationship is established;

[0110] Specifically, name similarity comparison includes: the agent checks the spelling of field names in S1 and concept names in S2, and how similar the letter combinations are. For example, the path distance or similarity score of Meter_Reading and electric meter reading is calculated. The higher the score, the more likely they are to refer to the same thing. Even if the naming is not exactly the same, S4 can infer that they may be the same object, such as TRF_VOLT and transformer voltage, and establish a structured correspondence between them.

[0111] Specifically, data type and value range comparison includes: the agent checks whether the data type (such as number, text, date) and its possible values (such as only 'on', 'off') of a field in S1 are consistent with the type and value range of the business concept attribute defined in S2. If a field in S1 is character type, length 10, and value is only 'RUN', 'STOP', 'FAULT', and S2 defines the device state as enumeration type, and the value is 'running','shutdown', 'fault', S4 can intelligently discover the high consistency between them and establish the association.

[0112] Specifically, structured position comparison includes: if S1 provides the hierarchy of data storage (such as transformer table under substation), the agent matches it with the business relationship defined in S2 (such as substation contains transformer). The voltage field in the transformer sub-table under the substation table is linked to the structured business path in S2: substation-contains-transformer-has-attribute-voltage.

[0113] In D2: associate the blood path with the business process operation to infer the business relationship between entities;

[0114] Specifically, the agent analyzes that data A recorded in S3 is converted into data B through a certain conversion, and then sees if there is any business operation description in S2 that conforms to this conversion. If S3 shows that the original electric energy data is converted into standard electric energy data through a structured conversion process called normalize_meter_data(), and S2 has the business operation of electric energy data standardization, S4 can intelligently infer that the conversion normalize_meter_data() is part of the electric energy data standardization business operation. If S3 shows that the generation of real-time current data is always accompanied by changes in dispatching instruction data, and S2 defines that there may be an influence relationship between real-time current and dispatching system. It is found that when the line trip event record appears, it always leads to the creation of dispatching work order record, and it is inferred that the structured business relationship of line trip event leading to the generation of dispatching work order.

[0115] In D3: discover hidden data relationships through association rule mining, graph embedding or inference rules.

[0116] Specifically, the agent analyzes a large amount of historical structured data to see which structured data items or data values always appear together. An association rule mining algorithm, Apriori, is used. For example, it discovers that whenever the device type is transformer, its cooling method is almost always oil-immersed self-cooling. S4 can intelligently infer that there is a typical configuration relationship between transformer and oil-immersed self-cooling, even if S2 has not explicitly defined it before. This can help us discover many implicit business configuration specifications.

[0117] Specifically, the application of the association rule mining algorithm Apriori includes:

[0118] Suppose we have $D$ structured transaction records (or observation records), each of which consists of multiple structured items. A structured item (Item) in this scenario refers to a structured attribute-value pair, such as (device type: transformer), or a specific structured state code, such as (switch status: on). A structured item set (Itemset) is a set consisting of one or more structured items, such as {(device type: transformer), (rated voltage: 110KV)};

[0119] The formula of the support algorithm is:

[0120]

[0121] where $Support(A) = \frac{\text{number of records containing structured item set A}}{\text{total number of records D}}$, A is any structured item set (for example, {(device type: transformer)} or {(device type: transformer), (rated voltage: 110KV)});

[0122] S4 will set a minimum support threshold (Min_Support). Only structured item sets with a support higher than this threshold will be considered frequent and worthy of further analysis. This can filter out patterns that occur by chance. The formula of the confidence algorithm is:

[0123]

[0124] S4 will set a minimum confidence threshold. Only relationships with a confidence higher than this threshold will be considered strongly related, and S4 will recommend them. The formula of the lift algorithm is:

[0125]

[0126] S4 will focus on the result set where the lift value is significantly greater than 1, indicating that there is a non-chance, real structured association relationship between A and B.

[0127] Specifically, the agent converts existing knowledge graph information (entities and relationships) into digital codes that computers can understand. Then, through these digital codes, it intelligently infers possible but undiscovered structured relationships. For example, if it knows that a substation contains a transformer and the transformer is connected to a line, it can infer that the substation is also indirectly connected to the line. This method allows S4 to intelligently infer missing structured connections in the knowledge graph and discover deeper, non-intuitive relationships between power grid entities.

[0128] Specifically, the agent learns some general inference rules from known structured facts. For example, if it always sees that device A belongs to region B and region B is located in city C, it can learn a rule: if a device belongs to a region and the region is located in a city, then the device is also located in the city. Once such a rule is learned, S4 can use it to automatically infer new, undefined structured relationships. For example, when there is information that a new transformer belongs to the Guangxi Power Grid region and the Guangxi Power Grid region is located in Nanning City, S4 can automatically infer that the transformer is located in Wuhan City, a new piece of knowledge.

[0129] In D4: integrate metadata, business concepts, blood relationship and mined data relationships into a unified power grid domain knowledge graph, and use a graph database for storage and management;

[0130] Specifically, entity and relationship creation includes: according to the definition of S1 and S2, specific power grid devices (such as substation No. 1 and transformer A) are created as nodes (entities) in the graph. Then, all structured relationships (such as include, measure, and cause) discovered by modules 1, 2, and 3 are created as edges (relationships) in the graph, connecting different nodes. Example: create node substation No. 1 and node transformer A, then add relationship substation No. 1 - contains -> transformer A.

[0131] Specifically, graph storage and management includes: using a specialized graph database (such as Neo4j, JanusGraph, ArangoDB) to store and manage the constructed knowledge graph, to support efficient graph querying and traversal. This knowledge graph is the core knowledge layer of MCP, logically part of MCP, and is uniformly managed and scheduled by MCP. MCP provides a unified entry point, allowing other agents or applications to access and utilize this knowledge graph through MCP. The structured storage of the knowledge graph and its unified management by MCP enable complex knowledge in the power grid domain to be understood and utilized by machines in a unified and efficient manner, providing a strong knowledge foundation for subsequent S5 and other agents.

[0132] Specifically, incremental update includes: when S1, S2, S3 or S8 (change-aware agent) has new structured information or changes, S4 does not re-construct all, but only updates the part of knowledge affected, which is more efficient.

[0133] Specifically, if S4 infers a new relationship, but this new relationship contradicts the known knowledge or existing relationship defined in S2, S4 can intelligently discover this structured conflict, intelligently solve the conflict according to preset rules (for example, the business rules defined by human beings have the highest priority, or the relationship inferred with high confidence has priority), or mark the conflict for manual intervention. The agent also regularly checks whether there is redundant, redundant or logically inconsistent structured information in the graph. Intelligent identification and removal of these redundancies or inconsistencies ensures the simplicity and accuracy of the graph.

[0134] In the embodiment of the application, the step S5 described above comprises the following sub-steps E1-E3:

[0135] In E1: analyze the numerical distribution, update frequency, uniqueness and integrity characteristics of historical data, and infer the reasonable value range, update period and non-empty constraint of the data item;

[0136] Specifically, numerical range and distribution pattern recognition includes: calculating the historical data values of all numerical structured data items to obtain their minimum value, maximum value, average value, median, standard deviation and other numerical portraits. Calculate the 1% and 99% percentile to find the normal range of data. For a large amount of data, try to judge whether the data conforms to some common statistical law (such as normal distribution). Once the average value ($mu$) and standard deviation ($sigma$) of a data item are calculated based on historical data, it can recommend rules according to the three-sigma (3$sigma$) principle. This principle means that in a normal distribution, most (about 99.7%) of the data values will fall within 3 times the standard deviation of the average value. S5 intelligently infers the reasonable numerical range of the data item (for example, the voltage value of a substation usually fluctuates within 110KV ± 5%), and recommends the rule that the value must be between [X, Y]. For non-numeric, discrete data (such as switch state coding), S5 will intelligently identify all valid coding values that have appeared in historical data, and recommend the rule that the value must be in the enumerated list [A, B, C].

[0137] Specifically, the frequency and update period pattern recognition includes: for the structured data items with timestamp identified in S1, S5 analyzes the time interval between adjacent data records. It uses methods like Fourier transform (DFT) to detect whether there is a regular repeating pattern in the data update. S5 performs DFT on a historical power grid data (such as the voltage value sequence in the last 24 hours), it can hear which pitch (frequency) is contained in this piece of data, and how big the volume of each pitch is. S5 finds out the fewest pitches with the largest volume, which correspond to the most primary periodic pattern in the data. In this way, S5 intelligently infers the expected update frequency or refresh period of the data (for example, a certain sensor data is updated every 5 seconds, and a certain report data is updated every day), and recommends the rule that the data item should be updated at least every Z seconds / minutes / hours:

[0138]

[0139] It can be simplified as:

[0140] S5 performs DFT on a historical power grid data (such as the voltage value sequence in the last 24 hours), it can hear which pitch (frequency) is contained in this piece of data, and how big the volume of each pitch is. S5 finds out the fewest pitches with the largest volume, which correspond to the most primary periodic pattern in the data. In this way, S5 intelligently infers the expected update frequency or refresh period of the data (for example, a certain sensor data is updated every 5 seconds, and a certain report data is updated every day), and recommends the rule that the data item should be updated at least every Z seconds / minutes / hours.

[0141] Specifically, the uniqueness and integrity pattern recognition includes: calculating which fields in the historical structured data are repeated as unique identifiers (such as device code, ticket number), and the proportion of null or missing values in all fields. S5 intelligently discovers which fields are almost always unique or almost always non-empty in historical data, and recommends rules that the field value must be unique and the field value must not be null.

[0142] In E2: combine business concept attributes with relationships in the power grid domain knowledge graph to infer attribute value constraint and business logic consistency rules; according to data bloodline information, check source and target data consistency and conversion logic, and generate quality constraint rules in the data flow process;

[0143] Specifically, the concept attribute constraint inference includes mapping the business concept attributes defined in S2 (e.g. the attributes of the concept of device status are running, shutdown, fault) to the numerical encoding of the actual data items identified in S1. It also analyzes the linkage between business concepts and attributes defined in S2 (these relationships can have been reinforced or discovered through the complex relationship identification of S4). S5 can intelligently recommend rules such as if the device type is encoded as TRF_TYPE_CODE, then its COOLING_METHOD_CODE must be in the list ['oil self cooling', 'air cooling']. It can translate the business knowledge of S2 and the complex relationships of S4 into concrete structured value constraints.

[0144] Specifically, the business relationship consistency inference includes using association rule mining algorithms (such as the Apriori algorithm). The power of this algorithm is that it can discover from a large amount of structured historical data, the non-explicit but high frequency co-occurrence and strong association business logic relationships between different structured data items. It only deals with pure structured data encoding and attribute combinations. Support represents the general level of the occurrence of a certain data combination in all records. Confidence represents the reliability of the occurrence of another data state when a certain data state occurs. Lift represents the possibility of the simultaneous occurrence of two data states, whether it is much higher than the possibility of their individual occurrence (whether it is a coincidence). S5 can intelligently infer business relationship consistency rules based on these indicators. For example, according to the knowledge of S2 and the graph of S4, a substation contains a transformer, S5 can further analyze the data and recommend rules such as all devices marked as EQUIPMENT_TYPE_CODE = TRANSFORMER must have a LOCATION_CODE that matches a certain registered SUBSTATION_CODE. Or, through historical data analysis, it is found that when the BREAKER_STATUS_CODE is OPEN, the FLOW_CURRENT_VALUE must be close to 0 such business logic rules.

[0145] Specifically, the source-target data consistency check includes: along the structured data lineage path identified by S3, comparing the structured features (such as data type, length, value range) and value consistency of the source data item and the target data item. If S3 identifies an explicit structured conversion logic (such as OUTPUT = INPUT1.5), S5 will verify whether the historical data conforms to the structured conversion logic by reverse derivation or forward simulation. It can intelligently identify and recommend that if the real-time voltage is calibrated and converted into historical voltage, the value of the historical voltage should be approximately equal to the real-time voltage multiplied by the calibration coefficient. It can intelligently discover whether the data has undergone unexpected, undeclared structural or numerical changes during the flow process.

[0146] Specifically, the data integrity and timing integrity check includes: analyzing the timing data flow involved in S3 lineage, checking whether the data points are continuous, whether there are structured missing points or out of order. At the same time, it will detect whether there is redundant structured data duplication or duplicate records in the data flow process. It can intelligently discover the possible packet loss or duplication problems of the data during transmission or storage, and recommend the rule that the data transmission from system A to system B must be complete, and there should be no record missing and no duplicate records in the target system transmitted by the source system.

[0147] In E3: fill the analysis results into the pre-defined rule templates, generate a list of structured and executable data quality rules, and assign priority and confidence.

[0148] Specifically, a variety of structured quality rule templates are internally preset (for example, {data item}: value is between [minimum value, maximum value], {data item}: value belongs to {enumeration list}, {data item} is not empty). According to the inference results, S5 will automatically select the appropriate template and fill in the specific structured parameters (such as specific numerical range, enumeration value list, associated data item name, formula parameter, etc.), forming a complete and executable structured rule. For each generated rule, according to its inferred confidence (reliability), the amount of historical data covered, and the associated business importance (from S2 / S4), a structured priority is assigned. When multiple rules may conflict with each other, these conflicts can be automatically identified and marked. For conflicting rules, the best solution can be intelligently recommended (for example, selecting the rule with the highest confidence, or prompting for manual confirmation), ensuring the effectiveness of the rule set.

[0149] Specifically, the generated structured rule list and its recommended reasons (from the inference process) are sent to MCP. S5 not only recommends rules, but also provides structured, machine-readable recommended reasons (e.g., {rule content: 'voltage value > 0', recommended source: 'historical data analysis', historical range: '[100, 120]', confidence: 'high'}), helping users understand the origin of the rules and improving the adoption rate of the rules.

[0150] Specifically, structured feedback from users on the acceptance, modification, or rejection of rules is received. When a user modifies or rejects a rule, the difference between the modified and unmodified rules and the rejection reason (provided in a structured coding form) are analyzed. Through reinforcement learning or supervised learning algorithms, the internal pattern recognition model and rule inference logic are continuously optimized. For example, if a user frequently modifies the recommended value range of a certain data item, the parameters of the future value range inference algorithm for that data item will be intelligently adjusted.

[0151] Specifically, S5 receives structured rule execution results (e.g., pass rate, failure rate, false positive rate of a certain rule) from S6 (rule self-adaptive optimization and evaluation). If the false positive rate of a rule is too high (meaning that the rule recommended by S5 may be too strict or inaccurate), S5 will automatically adjust the strategy for generating rules of this type to make it recommend more reasonable rules in the future. Conversely, if the pass rate is high, the generation confidence of this type of rule is enhanced. As new equipment is put into operation in the power grid, the system is upgraded, or the data pattern changes, S1, S2, S3, and S4 provide new structured metadata, business knowledge, and blood relationship information. S5 can learn based on this incremental information. S5 can generalize the successfully identified quality patterns (such as the voltage data characteristics of a certain type of sensor) to new and similar structured data items, avoiding starting from scratch, thereby improving rule generation efficiency and coverage.

[0152] Specifically, the standardized rule representation is to map all generated quality rules to a unified structured rule model within the enterprise (e.g., a unified Schema based on JSON or XML, containing only data item identification, verification type, parameters, etc.), ensuring the readability and executability of the rules. API / message queue transmission is to push recommended rule data to MCP in real time or near real time through RESTful API or message queues (such as Kafka). Full update and incremental update are supported. Rule query service is not only to push rules to MCP, but also to provide an independent structured query interface for S6, S7, or other external systems to query recommended quality rules.

[0153] In the embodiment of the present application, the above step S6 includes the following sub-steps F1-F3:

[0154] In F1: receive the detailed result data generated by rule execution, including rule ID, verification state, violation value and timestamp, time window aggregation of the result of rule execution, statistics of success rate, failure rate, false positive rate and false negative rate of each rule;

[0155] Specifically, the structured rule list generated or updated by S5 is obtained through the API interface or message queue subscription mechanism (e.g., Kafka topic) provided by MCP. When a new rule is generated or an existing rule is modified, S6 can be synchronized in real time. According to the structured identifiers such as data item ID and business concept ID involved in the rule definition, the relevant structured metadata (S1 output) and business knowledge (S2, S4 output) are queried and obtained through MCP, which are used for subsequent rule evaluation and optimization. S6 does not need to manually configure or search rules and background information, and can automatically maintain real-time synchronization of its internal rule library and knowledge context, laying a foundation for subsequent intelligent evaluation.

[0156] Specifically, the standardized structured messages sent by S7 are received through the message queue (e.g., Kafka). Each message contains rule ID, ID of the data item being verified, verification result (pass / fail), data value when failing, time stamp, etc. Schema verification is performed on the received structured messages to ensure the correctness of the data format. The execution results of different rule types are standardized into a unified structured format to facilitate subsequent processing. Structured aggregation is performed on all execution results of the same rule or the same type of rule within a certain time period (e.g., the past hour, day). For example, the total number of executions, the number of failures, the failure rate, etc. of a certain rule in the past 24 hours are counted. S6 can automatically process massive rule execution results and convert them from raw execution logs into structured statistical data for quantitative analysis, providing a clear report card for subsequent evaluation.

[0157] In F2: based on the results of time window aggregation and the real labels provided by humans or systems, the accuracy, recall rate and coverage rate of the rules are quantitatively evaluated;

[0158] Specifically, the key evaluation indicators include: false positive rate = number of false positives / (number of false positives + number of true passes). False positives are data that are marked as abnormal by the rule but are actually normal. High false positive rate will cause alert fatigue. False negative rate / recall rate = number of false negatives / (number of false negatives + number of true abnormalities). False negatives are data that are marked as normal by the rule but are actually abnormal. High false negative rate means that many problems are ignored. Accuracy = (number of true passes + number of true abnormalities) / total number of detections. It represents the proportion of correct judgments of the rule. Coverage rate = number of data items detected by the rule / total number of data items expected to be detected by S5. It represents the range covered by the rule.

[0159] Specifically, in the automated scenario, obtaining the number of true exceptions and the number of true passes requires real labels, including: source 1: artificially annotated structured feedback: the key is to obtain structured confirmation information of a user on a specific rule failure event (for example, the user confirms that a certain voltage overrun alarm is indeed a fault, or a certain alarm is a false alarm). S6 will collect these structured artificial labels to calculate false positives and false negatives. Source 2: feedback from associated higher-level business systems: for example, a rule detects an abnormal line current, and if the higher-level dispatch system or fault diagnosis system automatically confirms that the line has indeed failed, S6 can use it as a structured real label to verify the accuracy of the rule. Source 3: structured association of abnormal events: if rule A marks a voltage too high, and at a similar time rule B marks a transformer temperature anomaly, and S4 knowledge graph shows that voltage and temperature are highly correlated under certain conditions, then it can be intelligently inferred that both events are likely to be real.

[0160] Specifically, by comparing the evaluation indicators of the rules in different time periods, a time series analysis method (for example, moving average, exponential smoothing) is used to find the structured change trend of the performance of the rules (for example, the false positive rate continues to rise). The good and bad of each rule can be quantified automatically, and an objective, data-driven rule quality evaluation report is provided through structured labels and statistical analysis, rather than simply passing or failing count. This provides a clear direction for subsequent intelligent optimization.

[0161] In F3: according to the results of the quantitative evaluation, the rules are optimized by adopting parameter adjustment, priority resetting or rule start-stop strategy, and the optimization suggestions are fed back to the rule generation module.

[0162] Specifically, the threshold-based rule parameter adjustment includes: for parameterized rules (for example, [min, max] range check rules), if the false positive rate or the false negative rate exceeds the preset structured threshold, the parameters of the rules are intelligently adjusted. Gradient descent or Bayesian optimization algorithms are used. For example, if a voltage upper limit rule leads to high false positives, S6 can iteratively increase the upper limit value by a small amount, then wait for S7 to perform feedback, and then evaluate. This process is structured and quantifiable. It can independently fine-tune rule parameters, such as adjusting the voltage upper limit 115KV to 116KV to reduce false positives, thereby reducing the need for human intervention.

[0163] Specifically, the rule priority adjustment includes: intelligently adjusting the priority of the rules according to their importance (from S2, S4 business knowledge) and their evaluation performance. For example, a key rule with a high false negative rate should have its priority increased to attract more attention, so that the most important and effective rules can receive priority attention and resources.

[0164] Specifically, the rule enable / disable suggestion includes: for long-term poor performance but low business importance rules, S6 can intelligently suggest to temporarily disable them. For new rules, S6 can suggest running in observation mode first. Maintain the simplicity and efficiency of the rule system, avoid useless rules occupying resources and producing noise.

[0165] Specifically, the complex rule restructuring suggestion includes: for some complex rules that continue to perform poorly, S6 will combine the structured knowledge of S1-S4 to intelligently suggest splitting, merging or redefining them. For example, if a comprehensive rule always has false positives, S6 may suggest splitting it into several more detailed and specific sub-rules. Fundamentally solve the structural defects of the rules, and improve the accuracy and robustness of the rules.

[0166] Further, conflict detection includes: logical analysis of the rule set. For example, if rule A requires voltage > 110KV and rule B requires voltage < 100KV, it constitutes a structured conflict. For more complex association rules, the knowledge graph constructed by S4 can be used for graph traversal and path conflict detection. Automatically identify inconsistencies in rule logic, avoid conflicting results when rules are executed.

[0167] Specifically, redundancy detection includes: comparing the structured definitions and coverage of different rules. For example, if rule C requires voltage > 0 and rule D requires voltage > 10, then rule C is redundant to some extent. Rule inclusion relationship can be determined using rule coverage analysis or set theory methods. Simplify the rule set, remove redundant rules or rules that are included by better rules, and improve rule management efficiency.

[0168] Specifically, rule coverage analysis includes: abstracting business scenarios into a multi-dimensional space: scenario space S = voltage level x device type x operating state x weather condition; each rule can cover a subspace; rule R1 coverage domain Cov(R1) = {voltage >= 220kV}∩{device = transformer}∩{overload state} rule R2 coverage domain: Cov(R2) = {voltage >= 500kV}∩{device = line}∩{short circuit state}. Theoretical coverage: Coverage = |∪Cov(R i )| / |S|; actual coverage is verified by historical fault data to determine the rule trigger rate.

[0169] Specifically, based on set theory, the rule inclusion relationship is determined: identify the redundancy and conflict between rules (e.g. rule A completely covers the conditions of rule B); subset detection: R2 contains For detected conflicts and redundancies, S6 will intelligently recommend solutions (e.g., disable one, merge, or suggest manual arbitration) according to the priority of rules, business importance (from S2 / S4), and historical performance. Maintaining a healthy and efficient rule system reduces unnecessary rule execution and management burden.

[0170] In the embodiments of the present application, the above step S7 comprises:

[0171] Specifically, by subscribing to the structured message queue topics (e.g., Kafka topics) provided by MCP, S6 or S5 publishes real-time rule update messages. These messages contain the complete structured definition of the rule (rule ID, type, parameters, priority, etc.). After receiving the update, S7 can load the new rule or modified rule into the rule execution engine in its memory without downtime or restart. This is usually achieved by maintaining a versioned rule cache, ensuring that the ongoing verification tasks are not affected when loading a new version. Before loading, S7 performs syntax and semantic verification on the structured rule definition to ensure its correctness and logical consistency. For compilable rules, S7 compiles them into efficient internal execution code or data structures. S7 can automatically and continuously adapt to changes in rules without manual intervention to deploy new rules, ensuring the flexibility and real-time performance of the rule system.

[0172] Specifically, for real-time data (such as sensor data, SCADA system telemetry values), subscribe to related data topics through message queue clients (e.g., Kafka consumer). For historical data or periodic data reporting, pull through API interfaces (e.g., RESTful API), database connectors (JDBC / ODBC), or file listening (monitoring structured log files or CSV). Perform Schema validation on the accessed raw structured data to ensure it conforms to the expected structured format. If necessary, perform data type conversion, unit unification, etc. to convert it into a unified fact object model inside the rule engine. According to the configuration, temporarily cache the incoming data and perform structured deduplication to avoid repeated verification and resource waste. It can automatically support multiple power grid data sources and formats, clean and unify them into a structured representation that is easy for the rule engine to process, ensuring smooth and accurate rule execution.

[0173] Specifically, the core of the rule engine is how to efficiently match the input fact data with the rule conditions in memory. For simple rules of discrete or enumerated data items, they can be organized into structured decision trees or decision tables to quickly locate matching rules through path traversal or table lookup. Details are shown in Table 1.

[0174] Table 1: Decision table.

[0175]

[0176] In particular, for more complex rules with multi-conditional and priority logic, a production rule system pattern can be adopted. Rules are usually expressed in the form of IF [structured condition] THEN [structured action]. For example, IF (device type = 'transformer' AND temperature > threshold) THEN (trigger alarm). The Rete algorithm is an efficient pattern matching algorithm that breaks down and shares all the conditions of rules by constructing an internal, optimized data structure (think of a network with filter nodes). When new fact data comes in, it propagates along this network, only evaluating the changed or relevant conditions, rather than traversing all rules every time, thus greatly improving matching efficiency. S7 is able to intelligently match and trigger corresponding rules in a massive data stream with extremely high efficiency, achieving near real-time anomaly detection. When rules are loaded, the Rete algorithm parses the IF conditions of all rules and breaks them down into independent predicates. These predicates are organized into a node network: a single condition corresponding to a single fact. For example, the condition device type = 'transformer' in all rules will be compiled into a shared Alpha node. When a new structured data comes in, it will first pass through these Alpha nodes for preliminary filtering. The JOIN or combination condition of multiple facts corresponding to the results of multiple Alpha nodes. For example, a rule may need to match both device type = 'transformer' and temperature > threshold. Beta nodes are responsible for connecting facts (or partial matching results) that meet different conditions to form more complex matching patterns. The Rete engine maintains a structured working memory to store the current fact data being processed and all Alpha and Beta nodes that have matched partial results (called Tokens). When new structured data (facts) enter or existing facts are modified, the Rete algorithm only evaluates the network paths associated with these changes, rather than re-evaluating all rules, greatly improving matching efficiency.

[0177] Specifically, once the rule condition is matched, S7 will execute the structured actions defined in the rule. These actions are usually to generate a verification result, update a state, or trigger further internal processing. The rule engine will decide which rule to execute first or how to handle conflicts when multiple rules are matched simultaneously, according to the structured rule priority defined in S5 / S6. When the Rete network identifies rules that satisfy all conditions, these rules are called activated or triggerable. The inference engine needs to decide the order in which to execute these rules, especially when multiple rules are activated at the same time. All activated rules and their corresponding fact combinations are placed in a structured agenda. The engine intelligently selects the next rule to execute according to the structured rule priority defined in S5 / S6, the recency of rule usage (LIFO or Recency strategy of Rete algorithm), or the specificity of the rule (more specific rules are prioritized over more general rules), etc. S7 mainly adopts a forward-chained reasoning mode, i.e., starting from the fact data, deducing all possible results and actions. It realizes data-driven automated reasoning and decision-making, ensuring that rules can be executed in a logically consistent and business priority optimal manner in complex rule interaction scenarios. The rule engine can maintain a structured working memory or context to store the data it is currently processing and related information, allowing complex rules to use the results of previous rules for further judgment. It realizes data-driven automated decision-making and problem identification, and converts complex business logic and quality standards into machine-executable code.

[0178] Specifically, the rule execution results are packaged into a unified structured data format (e.g., JSON or Protobuf message). Each result record contains: rule ID, unique identifier of the data item being verified, verification status (PASS / FAIL), violation value, violation description (structured coding), timestamp, rule version, etc. For all (or only failed) verification results, S7 will store them in the MCP's data warehouse or dedicated result library. This is usually a high-performance time-series database or NoSQL database to support fast writing and querying of massive data. When storing the results, S7 will associate relevant structured metadata (such as data source, data table) to facilitate subsequent tracing and analysis. It realizes the automated, standardized recording of verification results, providing an accurate and traceable structured data foundation for rule evaluation in S6.

[0179] Specifically, the generated structured check result records (especially failure records) are sent to S6 in real time through a dedicated message queue. S6 will use these data to perform rule performance evaluation. For failures of critical business rules or high-priority data quality problems, S7 will send structured alarm information to relevant operation and maintenance personnel or business experts according to the configuration through the notification service provided by MCP (for example, sending structured emails, short messages, or calling APIs to trigger external alarm platforms). Check failure events can be published as structured events to an event bus for subscription and response by other intelligent agents or business systems (for example, S8 change perception intelligent agent, fault diagnosis system). S7 not only executes rules, but also realizes automated, structured feedback and alarm mechanisms after problem identification, ensuring that data quality problems can be discovered, evaluated and handled in a timely manner, thereby forming a closed-loop quality management process.

[0180] Specifically, real-time collection of its own operation indicators is performed, including: rule execution throughput, rule response time, memory usage, CPU utilization, rule loading time, error rate, and other structured data. Structured logs during rule execution are recorded in detail, including rule triggering, data processing flow, error information, etc. These logs will be stored in a centralized log management system. According to pre-set structured performance thresholds, the collected indicators are monitored in real time. Once an abnormal performance indicator is found (for example, excessively long response time), S7 will automatically trigger an internal alarm and record structured diagnostic information to assist in troubleshooting. S7 has self-perception and self-monitoring capabilities, ensuring that it can operate continuously, stably and efficiently as a data quality guardian for the power grid.

[0181] In the embodiments of the present application, step S8 metadata and data model change intelligent perception and monitoring is also included, specifically including the following steps:

[0182] Specifically, the hash value (i.e., the unique digital fingerprint of the data) of the data is calculated to quickly determine whether a file or data structure has been changed. If the hash values are different, it will further use a tree structure difference comparison algorithm to accurately locate which field has been added, deleted or modified. A hash function is a function that maps data of arbitrary size to a fixed-length output. It has one-way, collision resistance, and avalanche effect;

[0183] Specifically, taking the SHA-256 algorithm as an example, its core principle is a series of complex bit operations. The specific formula is very complex, and an abstract mathematical expression is used here to represent it:

[0184] $H = \text{SHA-256}(M)$

[0185] $M$ is the input data (for example, the Schema definition of a table in the power grid SCADA system).

[0186] $H$ is the generated fixed-length 256-bit hash value.

[0187] In the implementation of S8, the stored hash value at the first scan is denoted as $H\_{\text{old}}$, and the calculated hash value at the subsequent scan is denoted as $H\_{\text{new}}$. The screening logic of S8 is: if $H\_{\text{old}}=H\_{\text{new}}$, there is no change. If $H\_{\text{old}}\neH\_{\text{new}}$, a change occurs and the next step of accurate comparison needs to be performed.

[0188] Specifically, the core idea of MyersDiff algorithm is to find the minimum number of editing operations (addition, deletion, modification) required to convert one sequence (or tree) into another sequence (or tree). We can abstract two power grid database schemas (SchemaA and SchemaB) into two node sequences:

[0189] SchemaA: $A=(a\_1,a\_2,...,a\_m)$

[0190] SchemaB: $B=(b\_1,b\_2,...,b\_n)$

[0191] The algorithm constructs a grid (called edit graph) to find a shortest path from the top-left corner $(0,0)$ to the bottom-right corner $(m,n)$. Each step of movement represents an editing operation: moving right represents deleting an element in A. Moving down represents adding an element in B. Diagonal movement represents that two elements are the same (if $a\_i=b\_j$), or an element in B is modified to match an element in A (if $a\_i\neb\_j$).

[0192] Define $D(i,j)$ as the minimum number of operations (edit distance) required to convert sequence $A\[1..i]\$ to sequence $B\[1..j]\$. Its recursive relationship is:

[0193] $D(i,j)=\min(D(i-1,j)+1,\quadD(i,j-1)+1,\quadD(i-1,j-1)+\delta(a\_i,b\_j))$

[0194] Where: $\delta(a\_i,b\_j)=0$ if $a\_i=b\_j$ (elements are the same). $\delta(a\_i,b\_j)=1$ if $a\_i ≠ b\_j$ (elements are different and need to be modified).

[0195] By solving this formula using dynamic programming, S8 can efficiently find all the differences between two schemas and pinpoint exactly which field has been modified or added.

[0196] Specifically, upon discovering a change, S8 does not act immediately but intelligently assesses its potential impact. S8 relies on knowledge graph reasoning capabilities, derived from the knowledge graph built by S4. S8 takes the identified change (e.g., the deletion of a field) as input and traverses and queries the knowledge graph. Through predefined impact rules, S8 can intelligently infer which downstream objects will be affected by the change. For example, if the knowledge graph shows that field A is referenced by multiple lineage chains and quality rules, S8 can immediately assess that deleting field A will disrupt these lineage relationships and cause related quality rules to become invalid. The assessment results include not only the scope of the impact but also its severity, helping the system prioritize more important changes. The knowledge graph can be represented using triples \\(entity, relation, entity) or (entity, attribute, value). In the power grid example, these triples can be represented as:

[0197] (Voltage field, reference, line loss calculation task)

[0198] (Voltage field, has, type)

[0199] (Type, value, int)

[0200] S8's knowledge graph reasoning is essentially graph traversal and pattern matching. When a change occurs, S8 initiates a query, which can be formalized as a SPARQL query for pattern matching on the knowledge graph.

[0201] For example, when the type of a voltage field changes from int to float, S8 will execute a query similar to the following:

[0202] SELECT? impacted_entity

[0203] WHERE{

[0204] ? impacted_entity<reference>Voltage field.}

[0205] This query will return all entities that reference the voltage field (e.g., line loss calculation tasks). S8 then further queries these entities to assess the impact of the change.

[0206] In particular, in addition to handling expected changes, S8 can also intelligently identify unexpected, abnormal change patterns as an alert system. S8 employs a time series anomaly detection model to monitor change behavior. It can leverage models such as Prophet to learn normal change patterns by analyzing historical change logs (e.g., the number of times the Schema is updated each day, the number of fields involved in each update). Once it finds that the current change pattern deviates significantly from the historical normal pattern (e.g., a core data source suddenly undergoes large-scale Schema changes outside of working hours), S8 flags it as abnormal and issues a warning as a potential risk signal. The core model of Prophet is an additive decomposition model with the formula:

[0207] \$y(t) = g(t) + s(t) + h(t) + \epsilon\_t\$

[0208] \$y(t)\$ is the predicted value at time \$t\$ (e.g., the number of changes to the power grid database Schema).

[0209] \$g(t)\$ is the trend term, which captures non-periodic trends (e.g., as the data platform scales up, the number of changes generally shows an upward trend).

[0210] \$s(t)\$ is the seasonal term, which captures seasonal or periodic changes (e.g., regular changes every week, month, or year).

[0211] \$h(t)\$ is the holiday term, which captures the impact of special events (e.g., version release dates).

[0212] \$\epsilon\_t\$ is the error term, representing random fluctuations that the model cannot explain.

[0213] In the anomaly detection application of S8, the Prophet model learns and predicts the normal number of changes based on historical change logs, and generates a confidence interval.

[0214] \$y\_{\text{forecast}}(t) \in [y\_{\text{lower}}, y\_{\text{upper}}]\$

[0215] \$y\_{\text{forecast}}(t)\$ is the predicted number of changes.

[0216] $\[y\_{\text{lower}},y\_{\text{upper}}]\$ is the confidence interval of the prediction.

[0217] When the actual number of changes \$y\_{\text{actual}}(t)\$ falls outside this interval, i.e.:

[0218] \$y\_{\text{actual}}(t)>y\_{\text{upper}}\quad\text{or}\quady\_{\text{actual}}(t)<y\_{\text{lower}}\$

[0219] S8 will mark this change as abnormal and trigger an early warning immediately.

[0220] Specifically, after completing the evaluation, the core function of S8 is to convert the change information into operational instructions to drive the entire system to respond adaptively. S8 sends the change event notification to the management control platform (MCP) in a structured data format. The notification content includes the change type, impact range, severity assessment, and other key information. After receiving the notification, MCP will distribute instructions to the corresponding agents according to the pre-set strategy:

[0221] Update the knowledge graph, for example, mark the deleted field as invalid, or add information about the new field.

[0222] Re-evaluate or adjust related quality rules. For example, if a field that a rule depends on is deleted, S6 will suggest disabling the rule or waiting for S5 to generate new, applicable rules.

[0223] From the above scheme, it can be seen that the present application realizes a fundamental change from manual definition to intelligent creation of data quality rules by constructing an autonomous system that deeply integrates intelligent agents with a management control platform. The present application can deeply understand data semantics and business associations based on a knowledge graph, automatically infer and generate accurate quality rules, and continuously perceive rule execution effectiveness and data environment changes through a closed-loop feedback mechanism, achieving adaptive optimization and dynamic adjustment of rules. This significantly reduces the dependence on professional personnel for data governance, improves the automation and intelligence level of quality governance, ensures the continuous evolution and vitality of the data quality system, and thus provides more reliable and real-time high-quality data support for business decision-making.

[0224] Embodiment 2 provides a data quality rule automatic generation and adaptive system, comprising:

[0225] A data acquisition module for actively detecting and parsing heterogeneous data sources through a metadata acquisition agent to obtain structured metadata;

[0226] a semantic modeling module configured to perform business term inference and semantic modeling on the structured metadata, and to construct business concepts and relationships;

[0227] a bloodline graph recognition module configured to analyze processing logic of the structured metadata based on the business concepts and relationships, and to recognize a data bloodline graph;

[0228] a rule reasoning module configured to mine hidden relationships based on the business concepts, the semantic model and the bloodline graph, to construct a power grid domain knowledge graph, and to perform reasoning based on the power grid domain knowledge graph and historical data to generate data quality rules and business constraints;

[0229] an optimization evaluation module configured to monitor execution effects of the data quality rules, to perform self-adaptive optimization through quantitative evaluation, to execute the rules after self-adaptive optimization, to mark abnormal data and to feed back execution results.

[0230] It should be noted that the technical scheme of the data quality rule automatic generation and self-adaptive system and the technical scheme of the data quality rule automatic generation and self-adaptive method described above belong to the same concept. The technical scheme of the data quality rule automatic generation and self-adaptive system in the present embodiment is not described in detail, and the details can be referred to the description of the technical scheme of the data quality rule automatic generation and self-adaptive method.

[0231] The present embodiment also provides an electronic device including a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. The processor of the electronic device is configured to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The communication interface of the electronic device is configured to communicate with external terminals in wired or wireless manner. The wireless manner can be achieved through WIFI, operator network, NFC (near field communication) or other technologies. The computer program is executed by the processor to implement a data quality rule automatic generation and self-adaptive method. The display screen of the electronic device can be a liquid crystal display screen or an electronic ink display screen. The input device of the electronic device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the electronic device, or an external keyboard, touchpad or mouse, etc.

[0232] The present embodiment also provides a computer readable storage medium having a computer program stored thereon. The program is executed by the processor to implement the method proposed in the above embodiment.

[0233] The storage medium proposed in the embodiment belongs to the same inventive concept as the method proposed in the above embodiment, and the technical details not described in the embodiment can be seen from the above embodiment, and the embodiment has the same beneficial effects as the above embodiment.

[0234] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not limiting. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced equivalently without departing from the spirit and scope of the technical solutions of the present application, and they should be covered in the scope of the claims of the present application.

Claims

1. A method for automatic generation and adaptation of data quality rules, characterized in that, The method comprises the following steps: acquiring structured metadata by actively detecting and analyzing heterogeneous data sources through a metadata collection agent; inference of business terms and semantic modeling of the structured metadata to construct business concepts and relationships; resolving the processing logic of the structured metadata based on the business concepts and relationships to identify a data bloodline graph; mining hidden relationships based on business concepts, semantic models, and bloodline graphs to construct a power grid domain knowledge graph, reasoning based on the power grid domain knowledge graph and historical data to generate data quality rules and business constraints; monitoring the execution effect of the data quality rules and performing self-adaptive optimization through quantitative evaluation, executing the self-adaptive optimized rules, marking abnormal data, and feeding back the execution results.

2. The method for automatic generation and adaptive generation of data quality rules as described in claim 1, characterized in that, The acquisition of structured metadata comprises: actively scanning industrial control networks and listening to service registration centers or API gateways through a power grid data source discovery and monitoring module to identify newly added or changed power grid data sources; according to the identified power grid data source types, calling corresponding power grid special protocol adapters or general IT connectors to establish a secure connection; after successful secure connection, parsing device data dictionaries, database system tables, or API interface specifications that conform to industrial standards for the data format and business meaning of the power grid, and extracting device properties, measurement indicators, business field meanings, and data types, precision, and unit information; outputting the extracted metadata as a structured metadata dictionary and performing preliminary power grid domain semantic annotation.

3. The method for automatic generation and adaptive generation of data quality rules as described in claim 2, characterized in that, The inference of business terms and semantic modeling of the structured metadata comprises: analyzing the naming conventions, coding patterns, data types, and value ranges in the structured metadata to identify the preliminary business categories of data items; using clustering algorithms to induce power grid core business concepts and their attributes by inducting data items with similar patterns; identifying primary and foreign key relationships, reference relationships, and numerical dependencies from metadata to construct structured associations between power grid business concepts; constructing the inferred business concepts, attributes, and relationships into a structured business glossary and updating it to a unified knowledge management center.

4. The method for automatic generation and adaptive generation of data quality rules as described in claim 3, characterized in that, The identification of the data bloodline graph comprises: analyzing the configuration information, database operation logs, or file directory metadata of data integration tools to identify data flow paths between systems; analyzing ETL mapping rules, database view definitions, or computing logic to track changes in data item structure, value, and computing method during the flow process; associating the obtained data flow paths and evolution information with power grid business activities to construct a structured data bloodline graph, performing version management, conflict detection, and incremental updates on the data bloodline graph, and storing it in the knowledge management center.

5. The method for automatic generation and adaptive generation of data quality rules as described in claim 4, characterized in that, The construction of the power grid domain knowledge graph comprises: matching metadata items with business concept attributes in terms of name, data type, value range, and structured location, and establishing a mapping relationship; associating data bloodline paths with business process operations to infer business relationships between entities; discovering hidden data relationships through association rule mining, graph embedding, or reasoning rules. Integrate the metadata, business concepts, blood relationship and mined data relationship into a unified power grid domain knowledge graph, and store and manage it using a graph database.

6. The method of claim 5, wherein the data quality rule is automatically generated and adapted based on the data quality rule library and the data quality rule library is automatically updated based on the data quality rule library update module. 5 The generating data quality rules and business constraints comprises: Analyze the numerical distribution, update frequency, uniqueness and integrity characteristics of historical data to infer the reasonable value range, update period and non-empty constraints of data items; Infer attribute value constraints and business logic consistency rules by combining business concept attributes with relationships in the power grid domain knowledge graph; according to data blood relationship information, check source and target data consistency and conversion logic to generate quality constraint rules in the data flow process; Fill the analysis results into predefined rule templates to generate a structured and executable data quality rule list, and assign priority and confidence.

7. The method for automatic generation and adaptive generation of data quality rules as described in claim 6, characterized in that, The monitoring the execution effect of the data quality rules and the adaptive optimization through quantitative evaluation comprises: Receive detailed result data generated by rule execution, including rule ID, verification state, violation value and timestamp, and perform time window aggregation on the results of rule execution to calculate the success rate, failure rate, false positive rate and false negative rate of each rule; Based on the results of the time window aggregation and the real labels provided by humans or systems, quantitatively evaluate the accuracy, recall rate and coverage rate of the rules; According to the results of the quantitative evaluation, use parameter adjustment, priority reset or rule start-stop strategies to optimize the rules, and feed back the optimization suggestions to the rule generation module.

8. A system for automatic generation and adaptation of data quality rules, applying a method for automatic generation and adaptation of data quality rules according to any one of claims 1 to 7, characterized in that, Comprise: A data acquisition module for actively detecting and analyzing heterogeneous data sources through a metadata acquisition agent to obtain structured metadata; A semantic modeling module for inferring business terms and modeling semantics based on the structured metadata to construct business concepts and relationships; A blood graph identification module for analyzing the processing logic of the structured metadata based on the business concepts and relationships to identify the data blood graph; A rule reasoning module for mining hidden relationships based on business concepts, semantic models and blood graphs, constructing a power grid domain knowledge graph, and generating data quality rules and business constraints based on the power grid domain knowledge graph and historical data; An optimization evaluation module for monitoring the execution effect of the data quality rules and adaptively optimizing through quantitative evaluation, executing the rules after adaptive optimization, marking abnormal data and feeding back the execution results. 9.An electronic device comprising a memory and a processor, the electronic device characterized by: The memory is configured to store computer executable instructions, and the processor executes the computer executable instructions to implement the steps of the data quality rule automatic generation and adaptive method in any one of claims 1-7.

10. A computer-readable storage medium having computer-executable instructions stored thereon, characterized in that: The computer executable instructions are executed by the processor to implement the steps of the data quality rule automatic generation and adaptive method in any one of claims 1-7.

Images