Data management method and system based on power data intermediate station service

Abstract

The invention discloses a data management method and system based on electric power data intermediate station services, and relates to the technical field of electric power data intermediate stations. The method comprises the following steps: constructing a dynamic semantic modeling mechanism, and forming an evolvable semantic ontology library based on a domain knowledge system; constructing a data behavior graph evolution mechanism, and carrying out semantic monitoring on the whole data governance process; constructing a business semantic consistency verification mechanism, and establishing a multi-layer verification system covering grammar compliance, logic rationality and business integrating degree; a credible data collaboration mechanism is constructed, credible identification is generated for high-quality data, and hierarchical data service is provided. The method solves the problems that in the prior art, a semantic model is static, data behavior tracing is difficult, the quality evaluation dimension is single and credible collaboration is insufficient, and whole-process management of electric power data from semantic unification, behavior tracing, quality guarantee to service collaboration is achieved. And the data service capability and the business support level of the power data medium station are obviously improved.

Description

Technical Field

[0001] This invention relates to the field of power data middleware technology, and specifically to a data governance method and system based on power data middleware business. Background Technology

[0002] With the continuous improvement of the digitalization and intelligence level of the power system, the power data platform, as the core data infrastructure supporting the operation of the power grid, undertakes the tasks of accessing, integrating, and providing services for massive amounts of heterogeneous data from multiple sources. Power data exhibits significant characteristics such as diverse sources, complex formats, inconsistent semantics, and long lifecycles, which places extremely high demands on data governance.

[0003] However, traditional data governance methods often focus on syntactic standardization and consistency, such as ensuring the "formal correctness" of data through data cleaning and format standardization, but lack dynamic modeling and consistency assurance mechanisms for business semantics. In the power business scenario, the specific business meaning of data (such as the calculation method of "line loss" and the identification criteria of "interruptible load") evolves dynamically with adjustments to national policies, industry rules, and power grid operation modes. Existing static semantic models cannot adapt to this evolution, leading to a disconnect between data meaning and actual business operations, forming "data silos" and a "semantic gap."

[0004] Specifically, existing technologies have significant shortcomings in the following aspects:

[0005] Semantic modeling is static: Once an existing semantic model or knowledge graph is built, its concepts, relationships, and rules are relatively fixed and cannot be adaptively updated according to the evolution of the power business knowledge system. When new business terms appear or the meanings of existing terms change, manual intervention is required to reconstruct the model. This process is cumbersome, slow, and error-prone, making it difficult to adapt to the needs of the rapid development of the power business.

[0006] The lack of end-to-end semantic monitoring of data behavior: Existing methods struggle to trace the changing trajectory of data's business meaning throughout its entire lifecycle, from access and integration to processing and use. For example, after multiple processing stages, the business context behind a piece of "power generation" data (such as which power plant it came from and what analysis it was used for) may be lost or confused, leading to misunderstandings when using the data and affecting the accuracy of decision-making.

[0007] Data quality assessment is often limited to a single dimension: existing data quality assessments are mostly confined to the levels of syntax (such as format and type) and logic (such as value range and correlation), failing to deeply align with business semantics. This makes it impossible to assess the applicability and value of data in specific business scenarios, leading to the use of syntactically correct but business-useless data, while truly high-value data that meets business needs may be incorrectly judged as low-quality due to minor formatting issues.

[0008] Lack of a reliable data collaboration mechanism based on semantics and quality: Due to the lack of a unified semantic benchmark and authoritative quality certification, business systems struggle to quickly and accurately identify and acquire trustworthy data. This results in even high-quality data being difficult to apply accurately, securely, and efficiently in cross-departmental and cross-system collaborations, preventing the full realization of the value of data elements.

[0009] Therefore, there is an urgent need in this field for a power data governance method that can achieve dynamic semantic evolution, panoramic monitoring of behavior, multi-dimensional quality verification, and reliable service collaboration, so as to fundamentally improve the data service capabilities and business support level of the power data platform. Summary of the Invention

[0010] In view of the shortcomings of the prior art, the purpose of this invention is to provide a data governance method and system based on power data middleware business to solve the shortcomings of the prior art. By constructing four mechanisms, namely dynamic semantic modeling, data behavior graph evolution, business semantic consistency verification and trusted data collaboration, the invention realizes the whole process governance of power data from semantic unification, behavior traceability, quality assurance to service collaboration.

[0011] To achieve the above objectives, the present invention adopts the following technical solution;

[0012] A data governance method based on power data middleware business includes the following steps:

[0013] S100. Construct a dynamic semantic modeling mechanism, build an evolvable semantic ontology library based on the domain knowledge system, and form a unified semantic benchmark library through semantic mapping and rule reasoning to give data objects clear business meaning;

[0014] S200. Construct a data behavior graph evolution mechanism, based on a semantic benchmark library, to perform semantic monitoring of the entire data governance process, construct a dynamic behavior graph with business entities as nodes and operation events as edges, and output data lifecycle behavior data with semantic annotations;

[0015] S300. Construct a business semantic consistency verification mechanism. Based on the semantic benchmark library and data lifecycle behavior data, establish a multi-layer verification system to verify the grammatical compliance, logical rationality and business fit of data in the processing process in real time, and output verification results and quality scores.

[0016] S400. Construct a trusted data collaboration mechanism. For data objects whose quality scores in the verification results are higher than a preset threshold, generate a trusted identifier containing their semantic features, governance history, and quality score. Based on dynamic access control policies, provide data services with graded identifiers to business systems.

[0017] As a further aspect of the present invention, step S100 includes:

[0018] S110. Semantic ontology library construction: Establish a core concept system for the power business domain, covering four major business domains: power generation, transmission, distribution, and consumption. Each business domain is further subdivided into specific subclasses such as equipment entities, operating status, and performance indicators. By establishing hierarchical inheritance and combination relationships of concepts, a complete basic framework for the power business knowledge graph is formed.

[0019] The mathematical expression of a semantic ontology library is:

[0020] ;

[0021] It is a collection of concepts, encompassing all core concepts in the power business; A set of relations describes the relationships between concepts; Define the attribute characteristics of a concept for a set of attributes; This is a collection of instances, representing specific instances of business data;

[0022] S120. Multi-type data semantic mapping: Establish a field-level mapping rule library for structured data to achieve accurate matching between data fields and semantic concepts; Develop a template parsing engine for semi-structured data to extract key information and establish semantic associations through predefined parsing templates; Combine natural language processing and deep learning technologies for unstructured data to achieve entity recognition and automatic relationship extraction of unstructured content such as text and images.

[0023] S130. Develop a semantic reasoning engine;

[0024] S140. Establish a semantic evolution mechanism.

[0025] As a further aspect of the present invention, the steps of developing a semantic reasoning engine include:

[0026] Based on descriptive logic, the semantic relationships between concepts are formally defined and axiomatically expressed, constructing a semantic model with a rigorous logical foundation, thus providing a theoretical basis for subsequent automated reasoning; the semantic model is coupled to establish a dynamically configurable business rule base; finally, an inference engine based on table logic or production rules is developed, which automatically derives the implicit semantic relationships in the data through rule matching and logical calculation, enriching the business connotation of the data and mining its potential correlation value.

[0027] The description logic includes equivalence relations, inclusion relations, disjoint relations, etc.

[0028] As a further aspect of the present invention, the step of establishing the semantic evolution mechanism includes:

[0029] First, a version control system is used to record the change history of the semantic ontology to achieve traceability of the change process, and a complete set of semantic evolution operators is designed to atomically process the incremental and change operations of the ontology.

[0030] Then, to ensure logical consistency during the evolution process, higher-level semantic fusion operation rules are defined. These rules are responsible for coordinating conflicts between multiple evolution operators during version merging, thereby ensuring a smooth semantic transition and logical consistency between the old and new versions.

[0031] Finally, through a controlled version iteration process, the changes that have undergone conflict coordination and consistency verification are officially updated to the semantic benchmark library, thereby enabling the semantic ontology to continuously evolve and optimize while maintaining global stability.

[0032] As a further aspect of the present invention, step S200 includes:

[0033] S210. Define the behavioral graph infrastructure, abstract business entities such as data tables, fields, and processing rules as nodes, and abstract operations such as data flow, status change, and permission adjustment as edges, and construct a multi-layered graph structure including a basic operation layer, a business process layer, and a business semantic layer to form a complete data governance behavior model.

[0034] S220. Establish a distributed event acquisition system, adopt a message queue-based architecture to achieve high-concurrency event acquisition, and use stream processing technology to process various state change events in the data governance process in real time to ensure that the behavior graph can reflect the latest data governance status and operation records in a timely manner.

[0035] S230. Develop a semantic annotation service that automatically attaches multi-dimensional semantic tags to each operation event based on a semantic ontology library. The semantic tags include key information such as operation type, execution time, affected objects, and degree of business impact. It also calculates a semantic consistency score to provide rich semantic context information for subsequent data quality analysis.

[0036] Semantic consistency scoring formula:

[0037] ;

[0038] in, For event e A semantic tag, For its weight, This is a function for calculating semantic similarity.

[0039] S240. Design a graph evolution mechanism, adopt an incremental update strategy to handle newly added nodes and edges, establish a graph version management mechanism to support historical behavior tracing, ensure data integrity and logical correctness in the graph evolution process through consistency verification, and realize dynamic maintenance and continuous optimization of the behavior graph.

[0040] As a further aspect of the present invention, the specific implementation steps of the incremental update strategy include:

[0041] S241. Change capture: Real-time monitoring of the operation event stream generated in the data governance process, abstracting each event into an incremental change unit; the incremental change unit includes at least: change operation type, target node / edge identifier, attribute set before and after the change, and timestamp;

[0042] S242. Changeset Construction and Verification: Within a predefined time window or when a certain number of change events are reached, multiple incremental change units are aggregated into an atomic changeset; before application, this changeset undergoes consistency verification, which includes, but is not limited to:

[0043] Structure verification: Ensure that the nodes connected by the newly added edges exist;

[0044] Logical validation: Ensure that the operation sequence conforms to the business logic;

[0045] S243. Versioning: Assign a monotonically increasing version number to each validated change set and apply it sequentially to the behavior graph based on this version number. The current state of the graph is the cumulative effect of all applied change sets.

[0046] The model is expressed as:

[0047] ;

[0048] ;

[0049] Let the state of the behavior graph at time t be... For a set of nodes, Let it be the set of edges; For the change set; The merging operator for the graph is responsible for merging the change set. Incremental updates are merged into the original map. ;

[0050] S244. Establish a graph version management mechanism, which supports the rollback of the behavior graph state to any historical moment by persistently storing each change set and its version number, and provides complete data change audit trails.

[0051] Consistency checks ensure data integrity and logical correctness during the evolution of the behavioral graph, ultimately enabling dynamic maintenance and continuous optimization of the behavioral graph.

[0052] As a further aspect of the present invention, step S300 includes:

[0053] S310. Establish a syntax compliance verification mechanism. Through technical means such as regular expression matching, data type detection, and value range verification, conduct a comprehensive check on the data from basic dimensions such as data structure, format specifications, and value range to ensure that the data meets the predefined technical specifications and standard requirements.

[0054] The formula for grammatical compliance verification is:

[0055] ;

[0056] Data to be verified; A set of syntax rules;

[0057] Data structure check: Verify whether the data table structure, number of fields, primary and foreign key relationships, etc., conform to the model definition;

[0058] Formatting validation: Regular expression matching is used to ensure that the format of fields such as ID number, phone number, and device code conforms to the established rules;

[0059] Value range verification: Checks whether numerical data is within a reasonable range and whether the values ​​of enumerated data are within the preset dictionary entries;

[0060] S320. Logical rationality verification: Based on the business rule base, conduct in-depth analysis of data relationships, state transition logic, and business constraints to identify logical contradictions, state anomalies, and constraint violations in the data, ensuring the correctness and rationality of the data at the business logic level.

[0061] The formula for verifying logical rationality is:

[0062] ;

[0063] It represents a predefined set of rules; Represents data objects In the rule set The total number of verification passes or compliance score; Representative rule set A single rule in; It is an indicator function. When a data object... Satisfy the rules When the data object is in a certain state, the function returns 1; when the data object is in a certain state, the function returns 1 Not satisfied with the rules When this happens, the function returns 0.

[0064] In syntax validation, It is a set of syntax rules, including constraints on data structures, formats, and value ranges;

[0065] In logical verification, It is a set of business rules, including logical rules such as relationships, state transitions, and business constraints; score. The higher the value, the more rules the data object conforms to, and the better its syntactic compliance or logical rationality.

[0066] S330. Business fit assessment employs semantic similarity calculation methods to evaluate the degree of matching between data and specific business scenarios from multiple dimensions, including business meaning, usage scenarios, and application requirements. It uses a vector space model to calculate the similarity between data features and business requirement features, quantifying the business applicability of the data. The formula for quantifying business applicability is:

[0067] ;

[0068] For data feature vectors; For business scenario feature vectors;

[0069] S304. Comprehensive quality assessment: The verification results of dimensions such as grammatical compliance, logical rationality, and business fit are weighted and integrated to generate an objective comprehensive quality score; a quality level classification system is established, and differentiated quality thresholds are set according to the needs of different business scenarios; through the quality report generation engine, a complete report containing problem diagnosis, improvement suggestions, and quality trend analysis is automatically output.

[0070] As a further aspect of the present invention, step S400 includes:

[0071] S401. Establish a trusted identifier generation system to create digital trusted credentials containing data fingerprints, quality scores, governance history and semantic features for data objects that have passed quality verification. Use digital signature and blockchain technology to generate trusted identifiers to ensure the immutability and verifiability of the identifiers and provide authoritative credibility proof for data use.

[0072] Trusted identifier generation formula:

[0073] ;

[0074] Indicates string concatenation; A fingerprint of the data content (such as a SHA-256 hash value); For overall quality scoring; Summary information on the governance process; For timestamps;

[0075] S402. Design fine-grained access control policies based on a multi-dimensional attribute evaluation model, taking into account factors such as data sensitivity level, user permission level, business importance, and access context, to achieve dynamic and context-aware access control decisions and ensure that data is shared and utilized to the maximum extent under secure and controlled conditions.

[0076] The access control decision function is:

[0077] ;

[0078] Data sensitivity level; User permission level; For access context; The data trust level is calculated based on the trusted identifier; It is a policy-based decision logic that outputs a boolean value indicating whether access is authorized.

[0079] S403. Construct a hierarchical data service system, and provide differentiated service levels such as complete data service, summary data service, de-identified data service, and statistical data service according to the security needs, performance requirements and application scenario characteristics of business systems. Guide user requests to appropriate service endpoints through a service routing mechanism.

[0080] S404. Establish a service monitoring system, clarify the quality commitments and performance indicators of each service level through service level agreements, establish a complete monitoring mechanism for service quality measurement, anomaly detection, performance optimization, etc., to ensure the continuous stability of data service quality and form a complete closed loop from data governance to data value realization.

[0081] On the other hand, this application also provides a data governance system based on power data middleware business, including a semantic modeling module, a behavior graph module, a consistency verification module, and a trusted collaboration module;

[0082] Among them, the semantic modeling module is used to construct a dynamic semantic modeling mechanism;

[0083] The Behavior Graph module is used to construct the data behavior graph evolution mechanism;

[0084] The consistency verification module is used to build a business semantic consistency verification mechanism;

[0085] The Trusted Collaboration Module is used to build a trusted data collaboration mechanism.

[0086] In summary, due to the adoption of the above technical solution, the beneficial technical effects of the invention are as follows:

[0087] Compared with existing technologies, the data governance method and system based on power data middleware business provided by this invention brings the following significant benefits by constructing four core mechanisms:

[0088] It achieves unified and dynamically adaptive business semantics: by constructing an evolvable semantic ontology library and semantic evolution mechanism, the business meaning of power data can be smoothly and consistently updated as policies and rules change. This fundamentally solves the problem of static semantic models, ensures real-time synchronization between data semantics and actual business, and provides a unified "language" for cross-system data understanding and exchange.

[0089] It achieves panoramic traceability and transparent management of the data lifecycle: by constructing a data behavior graph evolution mechanism, the entire data governance process is semantically recorded and monitored in graph form. This makes the business intent and impact of every data transfer and every processing clear, solving the "black box" problem of data behavior and providing strong support for data lineage analysis, impact analysis, and problem localization.

[0090] This enhances the accuracy and business relevance of data quality assessment: By constructing a multi-layered verification system covering syntactic, logical, and business fit, and particularly by introducing a business fit assessment based on a vector space model, data quality assessment has evolved from a simple technical compliance check to a quantitative evaluation of the business value of data. This ensures that the data ultimately put into use is not only "correct" but also "useful," directly improving the quality of data-driven decision-making.

[0091] A secure and reliable data sharing and collaboration environment has been built: By constructing a trusted data collaboration mechanism, tamper-proof trusted identifiers are generated for high-quality data, and fine-grained, context-aware access control is implemented based on these identifiers. This solves the problem of business systems being "afraid or unwilling to use" external data, and greatly promotes the smooth flow and value reuse of high-quality data in the business ecosystem while ensuring security, forming a closed loop from data governance to value realization.

[0092] In summary, this invention, through the organic synergy of four major mechanisms, forms a complete and closed-loop power data governance solution, effectively overcoming the shortcomings of existing technologies and significantly improving the data service capabilities, business agility, and overall support level of the power data platform. Attached Figure Description

[0093] Figure 1 This is a flowchart of a data governance method based on power data middleware business;

[0094] Figure 2 This is a flowchart of step S100 in a data governance method based on power data middleware business;

[0095] Figure 3 This is a flowchart of step S200 in a data governance method based on power data middleware business;

[0096] Figure 4 This is a flowchart of step S300 in a data governance method based on power data middleware business;

[0097] Figure 5 This is a flowchart of step S400 in a data governance method based on a power data middleware platform. Detailed Implementation

[0098] To make the objectives, technical solutions, and advantages of the invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0099] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0100] Please see Figure 1 This illustrates an embodiment of the present invention that provides a data governance method based on power data middleware services, including the following steps:

[0101] S100. Construct a dynamic semantic modeling mechanism, build an evolvable semantic ontology library based on the domain knowledge system, and form a unified semantic benchmark library through semantic mapping and rule reasoning to give data objects clear business meaning;

[0102] S200. Construct a data behavior graph evolution mechanism, based on a semantic benchmark library, to perform semantic monitoring of the entire data governance process, construct a dynamic behavior graph with business entities as nodes and operation events as edges, and output data lifecycle behavior data with semantic annotations;

[0103] S300. Construct a business semantic consistency verification mechanism. Based on the semantic benchmark library and data lifecycle behavior data, establish a multi-layer verification system to verify the grammatical compliance, logical rationality and business fit of data in the processing process in real time, and output verification results and quality scores.

[0104] S400. Construct a trusted data collaboration mechanism. For data objects whose quality scores in the verification results are higher than a preset threshold, generate a trusted identifier containing their semantic features, governance history, and quality score. Based on dynamic access control policies, provide data services with graded identifiers to business systems.

[0105] Please refer to Figure 2 The document illustrates a flowchart of step S100 of an exemplary data governance method based on a power data middleware platform, which specifically includes:

[0106] S110. Semantic ontology library construction: Establish a core concept system for the power business domain, covering four major business domains: power generation, transmission, distribution, and consumption. Each business domain is further subdivided into specific subclasses such as equipment entities, operating status, and performance indicators. By establishing hierarchical inheritance and combination relationships of concepts, a complete basic framework for the power business knowledge graph is formed.

[0107] The mathematical expression of a semantic ontology library is:

[0108] ;

[0109] It is a collection of concepts, encompassing all core concepts in the power business; A set of relations describes the relationships between concepts; Define the attribute characteristics of a concept for a set of attributes; This is a collection of instances, representing specific instances of business data;

[0110] S120. Multi-type data semantic mapping: Establish a field-level mapping rule library for structured data to achieve accurate matching between data fields and semantic concepts; Develop a template parsing engine for semi-structured data to extract key information and establish semantic associations through predefined parsing templates; Combine natural language processing and deep learning technologies for unstructured data to achieve entity recognition and automatic relationship extraction of unstructured content such as text and images.

[0111] S130. Develop a semantic reasoning engine, formally define and axiomatically express the semantic relationships between concepts based on descriptive logic, and construct a semantic model with a rigorous logical foundation, thereby providing a theoretical basis for subsequent automated reasoning; on this basis, couple the formal semantic model to establish a dynamically configurable business rule base; furthermore, develop an inference engine based on table logic or production rules, and automatically deduce the implicit semantic relationships in the data through rule matching and logical calculation (e.g., if device A is a transformer and is located in substation B, then its operating data belongs to the asset ledger of substation B), enriching the business connotation of the data and mining its potential associated value;

[0112] The description logic includes equivalence relations, inclusion relations, disjoint relations, etc.

[0113] S140. Establish a semantic evolution mechanism:

[0114] First, a version control system is used to record the change history of the semantic ontology to achieve traceability of the change process. A complete set of semantic evolution operators (such as adding concepts, deleting attributes, merging classes, etc.) is designed to atomically process the incremental and change operations of the ontology.

[0115] Then, to ensure logical consistency during the evolution process, higher-level semantic fusion operation rules are defined. These rules are responsible for coordinating conflicts between multiple evolution operators during version merging (e.g., resolving conflicts such as duplicate concept names and attribute definitions), thereby ensuring a smooth semantic transition and logical consistency between the old and new versions.

[0116] Finally, through a controlled version iteration process, the changes that have undergone conflict coordination and consistency verification are officially updated to the semantic benchmark library, thereby enabling the semantic ontology to continuously evolve and optimize while maintaining global stability.

[0117] Please refer to Figure 3 The document illustrates a flowchart of step S200 of an exemplary data governance method based on a power data middleware platform, which specifically includes:

[0118] S210. Define the behavioral graph infrastructure, abstract business entities such as data tables, fields, and processing rules as nodes, and abstract operations such as data flow, status change, and permission adjustment as edges, and construct a multi-layered graph structure including a basic operation layer, a business process layer, and a business semantic layer to form a complete data governance behavior model.

[0119] S220. Establish a distributed event acquisition system, adopt a message queue-based architecture to achieve high-concurrency event acquisition, and use stream processing technology to process various state change events in the data governance process in real time to ensure that the behavior graph can reflect the latest data governance status and operation records in a timely manner.

[0120] S230. Develop a semantic annotation service that automatically attaches multi-dimensional semantic tags to each operation event based on a semantic ontology library. The semantic tags include key information such as operation type, execution time, affected objects, and degree of business impact. It also calculates a semantic consistency score to provide rich semantic context information for subsequent data quality analysis.

[0121] Semantic consistency scoring formula:

[0122] ;

[0123] in, For event e A semantic tag, For its weight, This is a function for calculating semantic similarity.

[0124] S240. Design a graph evolution mechanism, adopt an incremental update strategy to handle newly added nodes and edges, establish a graph version management mechanism to support historical behavior tracing, ensure data integrity and logical correctness in the graph evolution process through consistency verification, and realize dynamic maintenance and continuous optimization of the behavior graph.

[0125] One possible implementation of the present invention includes the following steps for implementing the incremental update strategy:

[0126] S241. Change capture: Real-time monitoring of the operation event stream generated in the data governance process (such as data access, transformation, desensitization, etc.), and abstracting each event into an incremental change unit; the incremental change unit shall at least include: change operation type (add, delete, modify), target node / edge identifier, attribute set before and after the change, and timestamp;

[0127] S242. Changeset Construction and Verification: Within a predefined time window or when a certain number of change events are reached, multiple incremental change units are aggregated into an atomic changeset; before application, this changeset undergoes consistency verification, which includes, but is not limited to:

[0128] Structure verification: Ensure that the nodes connected by the newly added edges exist;

[0129] Logical validation: Ensure that the operation sequence conforms to the business logic (e.g., the "data deletion" operation will not occur before the "data creation" operation);

[0130] S243. Versioning: Assign a monotonically increasing version number to each validated change set and apply it sequentially to the behavior graph based on this version number. The current state of the graph is the cumulative effect of all applied change sets.

[0131] The model is expressed as:

[0132] ;

[0133] ;

[0134] Let the state of the behavior graph at time t be... For a set of nodes, Let it be the set of edges; For the change set; The merging operator for the graph is responsible for merging the change set. Incremental updates are merged into the original map. ;

[0135] S244. Establish a graph version management mechanism, which supports the rollback of the behavior graph state to any historical moment by persistently storing each change set and its version number, and provides complete data change audit trails.

[0136] Consistency checks ensure data integrity and logical correctness during the evolution of the behavioral graph, ultimately enabling dynamic maintenance and continuous optimization of the behavioral graph.

[0137] Please refer to Figure 4 The document illustrates a flowchart of step S300 of an exemplary data governance method based on a power data middleware platform, which specifically includes:

[0138] S310. Establish a syntax compliance verification mechanism. Through technical means such as regular expression matching, data type detection, and value range verification, conduct a comprehensive check on the data from basic dimensions such as data structure, format specifications, and value range to ensure that the data meets the predefined technical specifications and standard requirements.

[0139] The formula for grammatical compliance verification is:

[0140] ;

[0141] Data to be verified; A set of syntax rules;

[0142] Data structure check: Verify whether the data table structure, number of fields, primary and foreign key relationships, etc., conform to the model definition;

[0143] Formatting validation: Regular expression matching is used to ensure that the format of fields such as ID number, phone number, and device code conforms to the established rules;

[0144] Value range verification: Check whether numerical data is within a reasonable range (e.g., power factor between -1 and 1), and whether the values ​​of enumerated data are in the preset dictionary entries;

[0145] S320. Logical rationality verification: Based on the business rule base, conduct in-depth analysis of data relationships, state transition logic, and business constraints to identify logical contradictions, state anomalies, and constraint violations in the data, ensuring the correctness and rationality of the data at the business logic level.

[0146] The formula for verifying logical rationality is:

[0147] ;

[0148] It represents a predefined set of rules; Represents data objects In the rule set The total number of verification passes or compliance score; Representative rule set A single rule in; It is an indicator function. When a data object... Satisfy the rules When the data object... Not satisfied with the rules When this happens, the function returns 0.

[0149] In syntax validation, It is a set of syntax rules, including constraints on data structures, formats, and value ranges;

[0150] In logical verification, It is a set of business rules, including logical rules such as relationships, state transitions, and business constraints; score. The higher the value, the more rules the data object conforms to, and the better its syntactic compliance or logical rationality.

[0151] S330. Business fit assessment employs semantic similarity calculation methods to evaluate the degree of matching between data and specific business scenarios from multiple dimensions, including business meaning, usage scenarios, and application requirements. It uses a vector space model to calculate the similarity between data features and business requirement features, quantifying the business applicability of the data. The formula for quantifying business applicability is:

[0152] ;

[0153] For data feature vectors; For business scenario feature vectors;

[0154] S340. Comprehensive quality assessment: Weighted integration of verification results from dimensions such as grammatical compliance, logical rationality, and business fit to generate an objective comprehensive quality score; Establishment of a quality level classification system: Setting differentiated quality thresholds based on the needs of different business scenarios; Automatic output of a complete report including problem diagnosis, improvement suggestions, and quality trend analysis through a quality report generation engine.

[0155] Please refer to Figure 5 The document illustrates a flowchart of step S400 of an exemplary data governance method based on a power data middleware platform, which specifically includes:

[0156] S410. Establish a trusted identifier generation system to create digital trusted credentials containing data fingerprints, quality scores, governance history and semantic features for data objects that have passed quality verification. Use digital signature and blockchain technology to generate trusted identifiers to ensure the immutability and verifiability of the identifiers and provide authoritative credibility proof for data use.

[0157] Trusted identifier generation formula:

[0158] ;

[0159] Indicates string concatenation; A fingerprint of the data content (such as a SHA-256 hash value); For overall quality scoring; Summary information on the governance process; For timestamps;

[0160] S420. Design fine-grained access control policies based on a multi-dimensional attribute evaluation model, comprehensively considering factors such as data sensitivity level, user permission level, business importance, and access context, to achieve dynamic and context-aware access control decisions and ensure that data is shared and utilized to the maximum extent under secure and controlled conditions.

[0161] The access control decision function is:

[0162] ;

[0163] Data sensitivity level; User permission level; To access the context (such as time, location, operation type); The data trust level is calculated based on the trusted identifier; It is a policy-based decision logic that outputs a boolean value indicating whether access is authorized.

[0164] S430. Construct a hierarchical data service system, and provide differentiated service levels such as complete data service, summary data service, de-identified data service, and statistical data service according to the security needs, performance requirements and application scenario characteristics of business systems. Guide user requests to appropriate service endpoints through a service routing mechanism.

[0165] S440. Establish a service monitoring system, clarify the quality commitments and performance indicators of each service level through service level agreements, establish a complete monitoring mechanism for service quality measurement, anomaly detection, performance optimization, etc., to ensure the continuous stability of data service quality and form a complete closed loop from data governance to data value realization.

[0166] On the other hand, this application also provides a data governance system based on power data middleware business, including a semantic modeling module, a behavior graph module, a consistency verification module, and a trusted collaboration module;

[0167] Among them, the semantic modeling module is used to construct a dynamic semantic modeling mechanism;

[0168] The Behavior Graph module is used to construct the data behavior graph evolution mechanism;

[0169] The consistency verification module is used to build a business semantic consistency verification mechanism;

[0170] The Trusted Collaboration Module is used to build a trusted data collaboration mechanism.

[0171] The above description is a preferred embodiment of the invention and is not intended to limit the scope of the invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the invention should be included within the scope of protection of the invention.

Claims

1. A data governance method based on power data middleware business, characterized in that, Includes the following steps: S100. Construct a dynamic semantic modeling mechanism, build an evolvable semantic ontology library based on the domain knowledge system, and form a unified semantic benchmark library through semantic mapping and rule reasoning to give data objects clear business meaning; S200. Construct a data behavior graph evolution mechanism, based on a semantic benchmark library, to perform semantic monitoring of the entire data governance process, construct a dynamic behavior graph with business entities as nodes and operation events as edges, and output data lifecycle behavior data with semantic annotations; S300. Construct a business semantic consistency verification mechanism. Based on the semantic benchmark library and data lifecycle behavior data, establish a multi-layer verification system to verify the grammatical compliance, logical rationality and business fit of data in the processing process in real time, and output verification results and quality scores. S400. Construct a trusted data collaboration mechanism. For data objects whose quality scores in the verification results are higher than a preset threshold, generate a trusted identifier containing their semantic features, governance history, and quality score. Based on dynamic access control policies, provide data services with graded identifiers to business systems.

2. The data governance method based on power data middleware business according to claim 1, characterized in that, Step S100 includes: S110. Semantic ontology construction: Establishing a core concept system for the power business domain, covering four major business domains: power generation, transmission, distribution, and consumption. Each business domain is further subdivided into specific subclasses of equipment entities, operating status, and performance indicators. By establishing hierarchical inheritance and composition relationships between concepts, a complete basic framework for a power business knowledge graph is formed; Semantic ontology library. The mathematical expression is: ； It is a collection of concepts, encompassing all core concepts in the power business; A set of relations describes the relationships between concepts; Define the attribute characteristics of a concept for a set of attributes; This is a collection of instances, representing specific instances of business data; S120. Multi-type data semantic mapping: For structured data, a field-level mapping rule library is established to achieve accurate matching between data fields and semantic concepts; For semi-structured data, a template parsing engine is developed to extract key information and establish semantic associations through predefined parsing templates; For unstructured data, natural language processing and deep learning technologies are combined to achieve entity recognition and automatic relationship extraction of unstructured text and image content. S130. Develop a semantic reasoning engine; S140. Establish a semantic evolution mechanism.

3. The data governance method based on power data middleware business according to claim 2, characterized in that, The steps for developing the semantic reasoning engine include: Based on descriptive logic, the semantic relationships between concepts are formally defined and axiomatically expressed, constructing a semantic model with a rigorous logical foundation, thus providing a theoretical basis for subsequent automated reasoning; the semantic model is coupled to establish a dynamically configurable business rule base; finally, an inference engine based on table logic or production rules is developed, which automatically derives the implicit semantic relationships in the data through rule matching and logical calculation, enriching the business connotation of the data and mining its potential correlation value. The description logic includes equivalence relations, inclusion relations, and disjoint relations.

4. A data governance method based on power data middleware business according to claim 2, characterized in that, The steps for establishing the semantic evolution mechanism include: First, a version control system is used to record the change history of the semantic ontology to achieve traceability of the change process, and a complete set of semantic evolution operators is designed to atomically process the incremental and change operations of the ontology. Then, to ensure logical consistency during the evolution process, higher-level semantic fusion operation rules are defined. These rules are responsible for coordinating conflicts between multiple evolution operators during version merging, thereby ensuring a smooth semantic transition and logical consistency between the old and new versions. Finally, through a controlled version iteration process, the changes that have undergone conflict coordination and consistency verification are officially updated to the semantic benchmark library, thereby enabling the semantic ontology to continuously evolve and optimize while maintaining global stability.

5. A data governance method based on power data middleware business according to claim 1, characterized in that, Step S200 includes: S210. Define the behavioral graph infrastructure, abstract data tables, fields, processing rules and business entities as nodes, and abstract data flow, status change and permission adjustment operations as edges, and build a multi-layer graph structure including the basic operation layer, business process layer and business semantic layer to form a complete data governance behavior model. S220. Establish a distributed event acquisition system, adopt a message queue-based architecture to achieve high-concurrency event acquisition, and use stream processing technology to process various state change events in the data governance process in real time to ensure that the behavior graph can reflect the latest data governance status and operation records in a timely manner. S230. Develop a semantic annotation service that automatically attaches multi-dimensional semantic tags to each operation event based on a semantic ontology library and calculates a semantic consistency score, providing rich semantic context information for subsequent data quality analysis; Semantic consistency score formula: ； in, For event e A semantic tag, For its weight, This is a function for calculating semantic similarity. S240. Design a graph evolution mechanism, adopt an incremental update strategy to handle newly added nodes and edges, establish a graph version management mechanism to support historical behavior tracing, ensure data integrity and logical correctness in the graph evolution process through consistency verification, and realize dynamic maintenance and continuous optimization of the behavior graph.

6. A data governance method based on power data middleware business according to claim 5, characterized in that, The specific implementation steps of the incremental update strategy include: S241. Change capture: Real-time monitoring of the operation event stream generated in the data governance process, abstracting each event into an incremental change unit; the incremental change unit includes at least: change operation type, target node / edge identifier, attribute set before and after the change, and timestamp; S242. Changeset Construction and Verification: Within a predefined time window or when a certain number of change events are reached, multiple incremental change units are aggregated into an atomic changeset; before application, this changeset undergoes consistency verification, which includes, but is not limited to: Structure verification: Ensure that the nodes connected by the newly added edges exist; Logical validation: Ensure that the operation sequence conforms to the business logic; S243. Versioning: Assign a monotonically increasing version number to each validated change set and apply it sequentially to the behavior graph based on this version number. The current state of the graph is the cumulative effect of all applied change sets. The model is expressed as: ； ； Let the state of the behavior graph at time t be... For a set of nodes, Let it be the set of edges; For the change set; The merging operator for the graph is responsible for merging the change set. Incremental updates are merged into the original map. ; S244. Establish a graph version management mechanism, which supports the rollback of the behavior graph state to any historical moment by persistently storing each change set and its version number, and provides complete data change audit trails. Consistency checks ensure data integrity and logical correctness during the evolution of the behavioral graph, ultimately enabling dynamic maintenance and continuous optimization of the behavioral graph.

7. The data governance method based on power data middleware business according to claim 1, wherein step S300 includes: S310. Establish a syntax compliance verification mechanism. Through regular expression matching, data type detection, and value range verification techniques, conduct a comprehensive check on the data from the basic dimensions of data structure, format specifications, and value range to ensure that the data meets the predefined technical specifications and standard requirements. The formula for grammatical compliance verification is: ； Data to be verified; A set of syntax rules; S320. Logical rationality verification: Based on the business rule base, conduct in-depth analysis of data relationships, state transition logic, and business constraints to identify logical contradictions, state anomalies, and constraint violations in the data, ensuring the correctness and rationality of the data at the business logic level. The formula for verifying logical rationality is: ； It represents a predefined set of rules; Represents data objects In the rule set The total number of verification passes or compliance score; Representative rule set A single rule in; It is an indicator function; when the data object Satisfy the rules When the data object is in a certain state, the function returns 1; when the data object is in a certain state, the function returns 1 Not satisfied with the rules When this happens, the function returns 0. Score The higher the value, the more rules the data object conforms to, and the better its syntactic compliance or logical rationality. S330. Business fit assessment employs semantic similarity calculation methods to evaluate the degree of matching between data and specific business scenarios from multiple dimensions, including business meaning, usage scenarios, and application requirements. It uses a vector space model to calculate the similarity between data features and business requirement features, quantifying the business applicability of the data. The quantification formula is: ； For data feature vectors; For business scenario feature vectors; S304. Comprehensive quality assessment: The verification results of grammatical compliance, logical rationality, and business fit are weighted and integrated to generate an objective comprehensive quality score; a quality level classification system is established, and differentiated quality thresholds are set according to the needs of different business scenarios; through the quality report generation engine, a complete report including problem diagnosis, improvement suggestions, and quality trend analysis is automatically output.

8. The data governance method based on power data middleware business according to claim 1, wherein step S400 includes: S401. Establish a trusted identifier generation system to create digital trusted credentials containing data fingerprints, quality scores, governance history and semantic features for data objects that have passed quality verification. Use digital signature and blockchain technology to generate trusted identifiers to ensure the immutability and verifiability of the identifiers and provide authoritative credibility proof for data use. S402. Design fine-grained access control policies based on a multi-dimensional attribute evaluation model, comprehensively considering data sensitivity level, user permission level, business importance, and access context factors to achieve dynamic and context-aware access control decisions, ensuring that data is shared and utilized to the maximum extent under secure and controlled conditions. S403. Construct a hierarchical data service system, and provide differentiated service levels such as complete data service, summary data service, de-identified data service, and statistical data service according to the security needs, performance requirements, and application scenario characteristics of the business system. The system will also use a service routing mechanism to direct user requests to the appropriate service endpoints. S404. Establish a service monitoring system, clarify the quality commitments and performance indicators of each service level through service level agreements, establish a complete monitoring mechanism for service quality measurement, anomaly detection, and performance optimization, ensure the continuous stability of data service quality, and form a complete closed loop from data governance to data value realization.

9. The operating system of the data governance method based on power data middleware business according to any one of claims 1-8, characterized in that, The system includes a semantic modeling module, a behavior graph module, a consistency verification module, and a trusted collaboration module; among which, The semantic modeling module is used to build dynamic semantic modeling mechanisms; The Behavior Graph module is used to construct the data behavior graph evolution mechanism; The consistency verification module is used to build a business semantic consistency verification mechanism; The Trusted Collaboration Module is used to build a trusted data collaboration mechanism.

Images