Multi-source data and domain knowledge fusion modeling and retrieval enhancement method and system

Abstract

The application relates to the technical field of data governance and artificial intelligence, in particular to a modeling and retrieval enhancement method and system for multi-source data and field knowledge fusion. The method comprises the following steps: acquiring multi-source heterogeneous metadata and power grid field standardized documents, and performing metadata extraction, cleaning treatment and knowledge structuring to form a standardized data set; based on a power grid field ontology model, the standardized data set is instantiated, mapped and associated to build a knowledge fusion model; the semantic analysis of a user natural language query is performed, business entities and constraint conditions are extracted, and mixed retrieval is performed to obtain candidate data assets; the knowledge fusion model is used for associated expansion and blood relationship tracing, the results are confidence evaluated and structured, retrieval enhancement results are generated and output. The application realizes semantic analysis of a user natural language query, mixed retrieval and associated expansion, thereby improving the accuracy, correlation and result reliability of power grid data retrieval.

Description

Technical Field

[0001] This application relates to the fields of data governance and artificial intelligence technology, and in particular to a modeling and retrieval enhancement method and system that integrates multi-source data and domain knowledge. Background Technology

[0002] In related technologies, with the continuous advancement of digital transformation and smart grid construction in power grid enterprises, various data platforms, monitoring systems, and business management systems are widely used in scenarios such as power grid operation monitoring, equipment maintenance, fault analysis, and dispatch management. During the long-term process of information technology construction, power grid enterprises have accumulated a large amount of data resources. These data originate from multiple data sources, including SCADA monitoring systems, fault recording systems, business management systems, databases, and data warehouses, as well as knowledge information in areas such as equipment ledger standards, operating procedures, and fault judgment rules. By integrating and analyzing the above-mentioned multi-source data, important data support can be provided for power grid operation status assessment, equipment health management, and intelligent data collection and analysis.

[0003] However, existing data retrieval and data governance methods still have certain shortcomings in practical applications. On the one hand, technical metadata is usually scattered across different data systems, while knowledge in the power grid field mostly exists in the form of documents or independent knowledge bases. The lack of effective connection between the two easily leads to a disconnect between technical data and business semantics. On the other hand, existing retrieval methods mostly rely on keyword matching or simple semantic retrieval, making it difficult to simultaneously consider the characteristics of structured metadata, power grid business semantics, and industry rule constraints, resulting in incomplete recall and inaccurate matching of search results. Furthermore, traditional retrieval systems typically only return the data assets themselves, lacking explanations of the data source and the basis of business rules, making it difficult to achieve credible interpretation and source verification of the data results.

[0004] In summary, the technical problems existing in the relevant technologies need to be improved. Summary of the Invention

[0005] The main objective of this application is to propose a modeling and retrieval enhancement method and system that integrates multi-source data and domain knowledge to achieve semantic parsing, hybrid retrieval, and association expansion of user natural language queries, thereby improving the accuracy, relevance, and reliability of power grid data retrieval.

[0006] To achieve the above objectives, one aspect of this application proposes a modeling and retrieval enhancement method that integrates multi-source data and domain knowledge, the method comprising the following steps: Acquire multi-source heterogeneous metadata and standardized documents in the power grid field; Metadata extraction and cleaning are performed on the multi-source heterogeneous metadata to obtain standardized metadata. Document parsing and knowledge structuring are performed on the standardized documents in the power grid field to obtain standard domain knowledge data. The standardized metadata and the standard domain knowledge data are combined to form a standardized dataset. Based on a pre-defined ontology model for the power grid domain, the standardized dataset is instantiated, mapped, and associated to generate a knowledge fusion model. Receive a user's natural language query request, perform semantic parsing on the natural language query request, extract the business entities and constraints involved in the query, and generate parsing results; Based on the analysis results, a hybrid search is performed to obtain candidate data assets; The knowledge fusion model is used to perform association expansion and lineage tracing on the candidate data assets to obtain associated data results. The associated data results are evaluated for confidence and organized in a structured manner to generate and output enhanced search results.

[0007] In some embodiments, the step of extracting and cleaning the multi-source heterogeneous metadata to obtain standardized metadata includes: Technical metadata is extracted from multi-source heterogeneous metadata corresponding to different types of data sources through data source adapters to obtain the original metadata; The original metadata is cleaned to remove null and abnormal data, and duplicate records are deleted by hash comparison. The cleaned metadata is formatted and standardized. Based on a thesaurus of the power grid domain, the terms in the metadata are ambiguously resolved and semantically normalized to obtain standardized metadata.

[0008] In some embodiments, the data source includes a relational database, a non-relational database, a data warehouse, and an ETL tool; the raw metadata includes table structure information, field attribute information, and data lineage information.

[0009] In some embodiments, the step of parsing and structuring the standardized documents in the power grid field to obtain standard domain knowledge data includes: Text parsing is performed on standardized documents in the power grid field to extract original domain knowledge data; the standardized documents include technical specification documents, operating procedure documents, and equipment management documents. Based on natural language processing technology, entity recognition and rule extraction are performed on the original domain knowledge text to obtain power grid domain knowledge information; the power grid domain knowledge information includes power grid business terminology information, equipment coding information, and business rule information. The domain knowledge information is structured and modeled, and attribute annotations are performed on terminology definitions, rule thresholds, and applicable scenarios to generate standard domain knowledge data.

[0010] In some embodiments, the step of instantiating, mapping, and associating the standardized dataset based on a preset power grid domain ontology model to generate a knowledge fusion model includes: The technical metadata, domain knowledge information, and business rule information in the standardized dataset are mapped to corresponding instances in the preset power grid domain ontology model, and a unique identifier is assigned to each instance. The instance is labeled with attributes to generate attribute information; the attribute label includes technology name, data type, storage location, update cycle, terminology definition, applicable scenarios and rule threshold information; Based on the attribute information, semantic similarity is calculated between technical asset instances and business entity instances, and semantic association is established according to a preset similarity threshold. Based on field semantic matching and business scenario consistency verification, a rule binding relationship is established between business rule instances and corresponding technical asset fields; Analyze the data lineage information in the standardized dataset to construct the lineage relationships between data assets; The semantic relationships, rule binding relationships, and bloodline relationships are stored in a unified manner to generate the knowledge fusion model.

[0011] In some embodiments, the step of calculating the semantic similarity between the technical asset instance and the business entity instance based on the attribute information, and establishing a semantic association relationship according to a preset similarity threshold, includes: Based on the attribute information, semantic vectorization processing is performed on the technical asset instance and the business entity instance to obtain the technical asset semantic vector and the business entity semantic vector. Semantic similarity is calculated based on the semantic vector of the technical asset and the semantic vector of the business entity to obtain a semantic similarity value; The semantic similarity value is compared with a preset semantic similarity threshold. When the semantic similarity value is greater than or equal to the semantic similarity threshold, a semantic association relationship is established between the technical asset instance and the business entity instance.

[0012] In some embodiments, receiving a user's natural language query request, performing semantic parsing on the natural language query request, extracting the business entities and constraints involved in the query, and generating parsing results include: Receive user natural language query requests, perform word segmentation on the natural language query requests, and combine them with a power grid domain entity dictionary to perform named entity recognition and extract business entity information; The temporal expression, spatial expression, and business indicator description in the natural language query request are identified, and the identification results are converted into structured constraints. Based on a pre-defined power grid domain ontology model, the business entity information is matched with business entity instances in the ontology, and semantic expansion processing is performed based on synonym relationships and hierarchical relationships. A query expression is constructed based on the business entity information and the structured constraints, and a parsing result is generated.

[0013] In some embodiments, performing a hybrid retrieval based on the parsing results to obtain candidate data assets includes: A search request is constructed based on the business entity information and structured constraints in the parsing results; Based on the search request, keyword matching search is performed on the technical asset name, field annotation and business tag to obtain the first search result; The search request is semantically vectorized and its similarity to the semantic vector of the technology assets in the knowledge fusion model is calculated to obtain a second search result. The first search result and the second search result are merged and deduplicated, and then scored and sorted according to a preset relevance calculation rule; Data assets with a relevance score higher than a preset threshold are selected as candidate data assets.

[0014] In some embodiments, the step of using the knowledge fusion model to perform association expansion and lineage tracing on the candidate data assets to obtain associated data results includes: Starting with the candidate data asset, the business entity instances and business rule instances associated with the candidate data asset are traversed in the knowledge fusion model to obtain semantically associated extended data assets. Based on the rule binding relationship in the knowledge fusion model, determine the business rule information associated with the corresponding field of the candidate data asset; Based on the data lineage relationship in the knowledge fusion model, the candidate data assets are traced back to obtain their upstream data source information and downstream derived data asset information. The extended data assets, the business rule information, and the kinship tracing information are integrated to obtain the associated data results.

[0015] To achieve the above objectives, another aspect of this application proposes a modeling and retrieval enhancement system that integrates multi-source data and domain knowledge, the system comprising: The data source layer is used to collect multi-source heterogeneous metadata and standardized documents in the power grid field. The fusion processing layer is used to perform fusion modeling and retrieval enhancement processing on the data from the data source layer, and to perform the following processing: Metadata extraction and cleaning are performed on the multi-source heterogeneous metadata, and document parsing and knowledge structuring are performed on the standardized documents in the power grid field to obtain a standardized dataset. Based on a pre-defined ontology model for the power grid domain, the standardized dataset is instantiated, mapped, and associated to generate a knowledge fusion model. Receive a user's natural language query request, perform semantic parsing on the natural language query request, extract the business entities and constraints involved in the query, and generate parsing results; Based on the analysis results, a hybrid search is performed to obtain candidate data assets; The knowledge fusion model is used to perform association expansion and lineage tracing on the candidate data assets to obtain associated data results. The associated data results are assessed for confidence and organized in a structured manner to generate enhanced search results; The application service layer is used to provide data services based on the enhanced search results. The data services include an intelligent data query interface, a self-service data retrieval service, a report generation service, and an application programming interface.

[0016] To achieve the above objectives, another aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method.

[0017] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0018] To achieve the above objectives, another aspect of this application provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0019] The embodiments of this application include at least the following beneficial effects: This application provides a modeling and retrieval enhancement method and system for multi-source data and domain knowledge fusion. This solution extracts, cleans, and structures multi-source heterogeneous metadata and standardized documents in the power grid domain in a unified manner, and constructs a knowledge fusion model based on a preset power grid domain ontology model. This achieves semantic association and unified expression between data resources and domain knowledge, thereby effectively solving the problems of complex power grid data sources, large structural differences, and scattered semantic relationships, and improving the standardization and manageability of data organization. Simultaneously, by performing semantic parsing on user natural language query requests, extracting business entities and constraints, and combining a hybrid retrieval strategy to obtain candidate data assets, collaborative optimization can be achieved between structured retrieval and semantic retrieval, thereby improving the accuracy and matching efficiency of data retrieval. Furthermore, by utilizing the knowledge fusion model to extend and trace the lineage of candidate data assets, and by evaluating the confidence level and structuring the retrieval results, the final output retrieval enhancement results not only have stronger semantic association capabilities but also provide clear data sources and relationships, effectively improving the intelligence level and reliability of data retrieval in the power grid domain. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating a modeling and retrieval enhancement method that integrates multi-source data and domain knowledge, as provided in an embodiment of this application. Figure 2 This is a schematic diagram of the multi-source data preprocessing and standardization construction process provided in the embodiments of this application; Figure 3 This is a schematic diagram of the core model of the power grid ontology provided in the embodiments of this application; Figure 4 This is a schematic diagram of the overall architecture of a modeling and retrieval enhancement system that integrates multi-source data and domain knowledge, as provided in an embodiment of this application. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0022] It is understood that the terms “first,” “second,” etc., used in this application may be used herein to describe various concepts, but unless otherwise stated, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the words “if,” “when,” or “in response to a determination” as used herein may be interpreted as “when…” or “when…” or “in response to a determination.”

[0023] As used in this application, the terms "at least one", "multiple", "each", "any", etc., "at least one" includes one, two or more, "multiple" includes two or more, "each" refers to each of the corresponding multiples, and "any" refers to any one of the multiples.

[0024] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0025] Before providing a detailed description of the embodiments of this application, some of the nouns and terms used in the embodiments of this application will be explained first. The nouns and terms used in the embodiments of this application shall be interpreted as follows: AI Flywheel / AI Flywheel Model Update Mechanism: This refers to a closed-loop system that continuously collects user feedback data, automatically analyzes feedback patterns, and drives the algorithm model to perform incremental learning and optimization. Ontology / Domain Ontology: In computer science and information science, it refers to a model that formally and explicitly describes concepts, attributes, relationships, and constraints within a specific domain. In this invention, it specifically refers to the "Power Grid Domain Ontology," used to uniformly describe the business entities, technical assets, rules, and other concepts and their associations within a DC power grid. SCADA (Supervisory Control and Data Acquisition) is a data acquisition and monitoring control system. In power systems, it refers to the core production system used to collect real-time data from field equipment such as converter stations and substations, and to monitor and control them. It is one of the important structured data sources. Wide table: In DC power grid business, a large wide table is a collection of multi-source heterogeneous data (such as equipment ledgers, monitoring data, and fault records) integrated according to business themes. It includes metadata such as field meanings, aggregation rules, and update cycles. Commutation failure: In a high-voltage direct current (HVDC) transmission system, the failure of the converter to transfer current from one valve arm to another during the commutation process is a fault condition and one of the core business entities of the DC power grid. The "Big Watt" model: A pre-trained large-scale model for the power industry released by China Southern Power Grid, possessing core capabilities such as semantic understanding, terminology recognition, and rule reasoning in the power field, providing semantic association computation support for this invention.

[0026] This application provides a modeling and retrieval enhancement method and system for fusing multi-source data and domain knowledge. This solution extracts, cleans, and structures multi-source heterogeneous metadata and standardized documents in the power grid domain, and constructs a knowledge fusion model based on a pre-defined power grid domain ontology model. This achieves semantic association and unified expression between data resources and domain knowledge, effectively solving the problems of complex power grid data sources, large structural differences, and scattered semantic relationships, thus improving the standardization and manageability of data organization. Simultaneously, by semantically parsing user natural language query requests, extracting business entities and constraints, and combining a hybrid retrieval strategy to obtain candidate data assets, collaborative optimization can be achieved between structured retrieval and semantic retrieval, thereby improving the accuracy and matching efficiency of data retrieval. Furthermore, by utilizing the knowledge fusion model to extend and trace the lineage of candidate data assets, and by evaluating the confidence level and structuring the retrieval results, the final enhanced retrieval results not only possess stronger semantic association capabilities but also provide clear data sources and relationships, effectively improving the intelligence level and reliability of power grid domain data retrieval.

[0027] This application provides a modeling and retrieval enhancement method that integrates multi-source data and domain knowledge, relating to the fields of data governance and artificial intelligence. This method can be applied to terminals, servers, or software running on either. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, or in-vehicle terminal, but is not limited to these. The server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network. The software can be an application implementing the modeling and retrieval enhancement method that integrates multi-source data and domain knowledge, but is not limited to the above forms.

[0028] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0029] Figure 1 This is an optional flowchart of a modeling and retrieval enhancement method that integrates multi-source data and domain knowledge, as provided in an embodiment of this application. Figure 1 The method may include, but is not limited to, steps S1 to S7: S1: Obtain multi-source heterogeneous metadata and standardized documents in the power grid field; In this embodiment, the initial data set for subsequent data modeling and retrieval processing is obtained. Specifically, through a pre-configured data access method, multi-source heterogeneous metadata and standardized documents in the power grid field are obtained from multiple business data platforms within the power grid enterprise. The obtained data is then uniformly aggregated to form the basic data source required for subsequent processing.

[0030] Furthermore, during the data acquisition process, data from different sources can be collected through data interface calls, database access, or file reading. The collected data is then identified and recorded by source, ensuring the distinguishability and traceability of data from different sources in subsequent processing. Simultaneously, the acquired data is numbered or labeled according to a unified data identification rule to facilitate subsequent data processing and management.

[0031] Specifically, data related to data asset structure information and power grid business knowledge is acquired from multiple business data platforms within the power grid enterprise. This acquired data is then uniformly stored and managed to form a raw dataset. This raw dataset serves as input data for subsequent metadata extraction, document parsing, and knowledge structuring processing, providing foundational data support for building knowledge fusion models and performing enhanced retrieval processing.

[0032] S2: Extract and clean the multi-source heterogeneous metadata to obtain standardized metadata, and perform document parsing and knowledge structuring on the standardized documents in the power grid field to obtain standard domain knowledge data. Combine the standardized metadata and standard domain knowledge data to form a standardized dataset. refer to Figure 2 As shown, Figure 2 This diagram illustrates the multi-source data preprocessing and standardization construction process in this invention. This process involves identifying, extracting, cleaning, structuring, and quality-verifying data from different sources to form a standardized dataset that can be used for subsequent fusion modeling.

[0033] like Figure 2 As shown, the process first enters the data processing flow, classifying the input data by identifying the data source type. If identified as a metadata source, it enters the metadata processing path, where technical metadata is collected and extracted to obtain raw metadata such as table structure information, field attribute information, and data lineage information. If identified as a domain knowledge source, it enters the domain knowledge processing path, where relevant standardized documents are parsed and business information is extracted. In the metadata processing path, the collected raw metadata is first cleaned and deduplicated to improve data integrity and consistency by removing null values, abnormal data, and duplicate records. The cleaned data is then standardized to ensure consistency in naming conventions, data formats, and terminology across different data sources. In the domain knowledge processing path, raw domain knowledge text is obtained through document parsing and information extraction. Terminology and business rule information within the text are further identified and extracted. The extracted results are then structured to form standardized domain knowledge data.

[0034] After metadata and domain knowledge processing, the two types of data are merged into a unified data pool to form a unified data foundation. Data quality verification is then performed on the merged data, checking its integrity, consistency, and standardization through a data quality verification mechanism. Data that fails the quality check is identified as abnormal or inconsistent and enters the manual review and correction stage. After correction, the data re-enters the data cleaning and deduplication process for further processing, thus forming a closed-loop data quality repair mechanism. Once the data passes all quality checks, it is considered qualified and enters the subsequent fusion modeling process. This process enables the unified organization and standardization of multi-source heterogeneous data and standardized documents in the power grid field, providing a reliable data foundation for subsequent knowledge fusion model construction and data retrieval enhancement.

[0035] This involves extracting and cleaning multi-source heterogeneous metadata to obtain standardized metadata, including: Technical metadata is extracted from heterogeneous metadata from different types of data sources using a data source adapter to obtain raw metadata. These data sources include relational databases, non-relational databases, data warehouses, and ETL tools. The raw metadata includes table structure information, field attribute information, and data lineage information. The original metadata is cleaned to remove null and abnormal data, and duplicate records are deleted by hash comparison. The cleaned metadata is formatted and standardized. Based on a thesaurus of the power grid domain, the terms in the metadata are ambiguously resolved and semantically normalized to obtain standardized metadata.

[0036] In this embodiment, technical metadata extraction is first performed for different types of data sources. For relational databases, data definition statements and system table information can be read from DM, MySQL, Oracle, or SQL Server to extract table names, field names, field types, field lengths, whether they are primary keys, whether they are foreign keys, and field comments. Simultaneously, the logical relationships between the fault event table and the site information table and inspection record table can be identified through primary and foreign key relationships.

[0037] For NoSQL databases, attributes such as collection structure, field names, field value types, and whether fields are required can be read from HBase, MongoDB, or Cassandra, and the logical relationships between data can be reconstructed through key-value pair organization. For example, in a fault waveform data storage scenario, a collection stores waveform number, sampling time, waveform type, and the code of the device to which it belongs. By parsing the key names and nested structures, the association between them and fault event records can be identified.

[0038] For data warehouses, table structure details, field comments, partitioning fields, storage formats, and inter-table dependencies can be extracted from Hive or ClickHouse. For example, after performing a table structure read on the "Switching Failure Analysis Wide Table," its date-based partitioning settings, the meaning of fields for site-based statistics, and its dependencies on the real-time monitoring table and the fault event table can be obtained.

[0039] For ETL tools, processing metadata such as upstream data source paths, field mapping relationships, filtering conditions, aggregation rules, and downstream output locations can be extracted by reading job execution logs, scheduling scripts, and processing configurations, thus forming lineage information. After the above processing, the resulting raw metadata includes at least table structure information, field attribute information, and data lineage information, providing input for subsequent cleaning, deduplication, and standardization processes.

[0040] After extracting the raw metadata, data cleaning is performed. First, null value detection is performed on the raw metadata. Records with missing table names, field names, field types, or empty field descriptions are screened, and information lacking key attributes is removed. Simultaneously, abnormal records that clearly do not conform to business logic or data structure rules are identified, such as mismatched field types and meanings, abnormal data update times, or unclear data sources. Then, duplicate data identification is performed on the remaining records. Hash identifiers are generated based on field names, table names, and field descriptions. Records with the same hash value are compared, and the record with higher information completeness or a clearer source is retained. This process removes duplicate metadata, resulting in cleaned metadata.

[0041] After data cleaning, the cleaned metadata undergoes format standardization. This includes standardizing field names, time formats, character encodings, and field annotation expressions. For example, various naming conventions from different sources are unified into a predefined field naming standard, and date and time formats are standardized into a unified time expression. Simultaneously, based on a thesaurus for the power grid domain, ambiguity resolution and semantic normalization are performed on the terminology in the metadata. Synonyms or aliases used in power grid operations are uniformly mapped; for example, "converter substation" is standardized to "converter station," and "commutation failure" is standardized to "commutation failure," reducing semantic inconsistencies caused by differences in terminology between different data sources. Through these processes, standardized metadata with unified semantics and consistent format is obtained, providing a standardized data foundation for subsequent fusion modeling and correlation analysis.

[0042] Specifically, standardized documents in the power grid field are parsed and processed using knowledge structuring techniques to obtain standard domain knowledge data, including: Text parsing is performed on standardized documents in the power grid field to extract original domain knowledge data; these standardized documents include technical specifications, operating procedures, and equipment management documents. Based on natural language processing technology, entity recognition and rule extraction are performed on the original domain knowledge text to obtain power grid domain knowledge information; this power grid domain knowledge information includes power grid business terminology information, equipment coding information, and business rule information. The domain knowledge information is structured and modeled, and attribute annotations are performed on terminology definitions, rule thresholds, and applicable scenarios to generate standard domain knowledge data.

[0043] In this embodiment, the standardized documents in the power grid field mainly come from internal enterprise and industry-wide standardized documents, including technical specifications, operating procedures, and equipment management documents. First, text parsing is performed on these documents. For PDF documents, main text paragraphs, heading levels, table content, and appendix descriptions can be extracted; for Word documents, chapter titles, definition clauses, rule clauses, and attachment content can be extracted. After text parsing, the original document content can be converted into continuous, processable text fragments, creating conditions for subsequent entity recognition and rule extraction. For example, fault definition clauses can be parsed from the "Power Grid Operation Guidelines," equipment category codes and naming rules can be parsed from the "Equipment Coding Standards," and fault judgment conditions and operational constraints can be parsed from the "Maintenance Management Regulations."

[0044] During the parsing process, different segmentation strategies can be adopted for different document types. For technical specification documents, priority is given to retaining definition clauses, terminology clauses, and parameter description clauses; for operation procedure documents, the focus is on extracting scheduling operation requirements, fault diagnosis procedures, and equipment operation limitations; for equipment management documents, the focus is on extracting equipment classification, equipment codes, ledger attributes, and lifecycle management requirements. For example, in equipment management documents, information such as "converter valves belong to core DC transmission equipment" and "equipment codes consist of category codes, site codes, and sequence codes" can be identified; in operation procedures, rule statements such as "an early warning must be triggered when the arc extinction angle is lower than the specified value" can be identified. By processing documents differently according to their type, the relevance of subsequent extraction results can be significantly improved.

[0045] After document parsing, the original domain knowledge text is obtained. This text contains terminology definitions, equipment attributes, rule thresholds, scope of application, constraints, and processing requirements. For example, it can extract the definition text of "commutation failure," the rule text of "commutation failure warning triggered by arc extinction angle <10°," and equipment classification texts such as "transformer, converter valve, GIS equipment." At this stage, the knowledge content is still in natural language form and requires further entity recognition and rule extraction to be transformed into structured knowledge.

[0046] After parsing standardized documents in the power grid field and obtaining the original domain knowledge text, entity recognition and rule extraction are further performed on the original domain knowledge text using natural language processing technology. Specifically, based on a power grid professional terminology dictionary and industry corpus, entities such as business terms, equipment names, equipment categories, monitoring indicators, and standard numbers in the text are identified and labeled. For example, power grid business entities such as "commutation failure," "arc extinction angle," "converter valve," and "DC transmission system" are identified. At the same time, rule statements in the text are extracted to identify the conditional relationships, indicator thresholds, and applicable scenarios contained therein. For example, judgment conditions or monitoring indicator requirements related to equipment operating status are identified from operating procedure texts. Through the above processing, power grid business terminology information, equipment coding information, and business rule information can be extracted from the original text, transforming the knowledge content, originally existing in natural language form, into knowledge items with clear semantic boundaries.

[0047] After acquiring knowledge information in the power grid domain, this information undergoes structured modeling and key attributes are uniformly labeled. Specifically, different types of knowledge information are organized according to a unified structure. For example, business terms record their name, definition, associated equipment type, and source; equipment codes record their category, coding rules, and corresponding equipment attributes; and business rules record their name, description, applicable scenarios, and source documents. Simultaneously, key information such as term definitions, rule thresholds, and applicable scenarios are labeled with attributes, ensuring that each knowledge record has a clear semantic description and scope of application. Through this structured processing, standard domain knowledge data can be generated, transforming unstructured text into structured and semantically clear knowledge data, providing a foundation for subsequent knowledge fusion modeling and correlation analysis.

[0048] S3: Based on the preset ontology model of the power grid domain, instantiate, map and associate standardized datasets to generate a knowledge fusion model; Among them, based on a pre-defined ontology model of the power grid domain, a standardized dataset is instantiated, mapped, and associated to generate a knowledge fusion model, including: The technical metadata, domain knowledge information, and business rule information in the standardized dataset are mapped to corresponding instances in the preset power grid domain ontology model, and a unique identifier is assigned to each instance. Instances are labeled with attributes to generate attribute information; the attribute label includes technical name, data type, storage location, update cycle, terminology definition, applicable scenarios and rule threshold information; Based on attribute information, semantic similarity is calculated between technical asset instances and business entity instances, and semantic association is established according to a preset similarity threshold. Based on field semantic matching and business scenario consistency verification, a rule binding relationship is established between business rule instances and corresponding technical asset fields; Analyze the data lineage information in standardized datasets and construct the lineage relationships between data assets; Semantic relationships, rule binding relationships, and lineage relationships are stored in a unified manner to generate a knowledge fusion model.

[0049] Based on attribute information, semantic similarity is calculated between technical asset instances and business entity instances, and semantic associations are established according to preset similarity thresholds, including: Based on attribute information, semantic vectorization is performed on technical asset instances and business entity instances to obtain technical asset semantic vectors and business entity semantic vectors. Semantic similarity is calculated based on the semantic vectors of technical assets and business entities to obtain semantic similarity values; The semantic similarity value is compared with a preset semantic similarity threshold. When the semantic similarity value is greater than or equal to the semantic similarity threshold, a semantic association relationship is established between the technical asset instance and the business entity instance.

[0050] In this embodiment, after obtaining the standardized dataset, the data is instantiated, mapped, and associated based on a preset power grid domain ontology model.

[0051] refer to Figure 3 As shown, Figure 3 Centered on the core ontology model of the power grid field, China defines five core concepts: business entities, technical assets, business rules, indicators, and data lineage, and connects them into a whole through five types of relationships.

[0052] Business entities refer to specific objects, events, or states in DC power grid operations, such as commutation failure, abnormal oil chromatography, harmonic exceedance, and protection actions.

[0053] Technical assets refer to the physical or logical units that carry business data, including database tables, fields, wide tables, views, and API interfaces, such as the fault_event table and the station_code field.

[0054] Business rules refer to the judgment logic and operational constraints formed based on domain knowledge, such as the abnormal threshold of oil chromatography for "acetylene content > 5 μL / L" and the risk threshold of commutation failure for "extinction angle < 10°".

[0055] Indicators refer to quantifiable system performance and status parameters, such as line loss rate, converter operating efficiency, and equipment availability.

[0056] Data lineage refers to the processing, derivation, and transmission relationships of data from its generation to consumption, such as the processing chain of "SCADA raw data → minute-level aggregation → fault analysis wide table".

[0057] The five key concepts are connected through specific relationships, forming the model's knowledge network: Semantic association connects business entities and technical assets, representing the semantic correspondence between technical data and business concepts, such as the association between a fault event table (technical asset) and "commutation failure" (business entity).

[0058] Rule binding connects technical assets with business rules, indicating the business rules followed by specific data fields, such as binding the "acetylene" field in an oil chromatogram to the rule "acetylene content > 5 μL / L".

[0059] Bloodline connection refers to the connection between technological assets and data lineage, indicating the origins of data assets in terms of derivation and processing.

[0060] Metrics are related to business entities and metrics, representing the performance metrics that are affected or related to business events.

[0061] Constraint association connects business rules and metrics, representing the constraint relationship between business rules and metric targets.

[0062] Through fusion modeling, the ontology model completes the structured association between technical metadata and business knowledge in the power grid field, providing a unified semantic understanding foundation for subsequent high-precision retrieval enhancement and intelligent data utilization.

[0063] In the implementation process, the technical metadata, domain knowledge information, and business rule information in the standardized dataset are mapped to corresponding instances in the power grid domain ontology model, and each instance is assigned a unique identifier. For example, technical information such as data tables, fields, and data sources are mapped to technical asset instances; power grid business terms, equipment objects, and operational events are mapped to business entity instances; and equipment operation constraints and fault determination rules are mapped to business rule instances. To ensure that each instance can be uniquely identified, a unique identifier can be generated based on data category, source information, and number, thereby forming a unified set of instances.

[0064] After instance mapping is completed, attribute annotation is performed on each instance to generate instance attribute information. This attribute information includes at least the technology name, data type, storage location, update cycle, terminology definition, applicable scenarios, and rule threshold information. For example, for a real-time monitoring data field, its technology name can be labeled as "equipment monitoring indicator," its data type as "numerical data," its storage location as "operational monitoring database," and its update cycle as "real-time update." For business terminology instances, their terminology definition, applicable equipment types, and business application scenarios can be annotated. For business rule instances, their rule description, threshold conditions, and rule source can be annotated. Through attribute annotation, instances acquire a complete semantic information description.

[0065] After obtaining instance attribute information, the semantic relationship between technical asset instances and business entity instances is identified based on this attribute information. First, the textual information in the instance attributes is semantically vectorized, enabling comparison of technical asset instances and business entity instances using a unified semantic representation. Then, a semantic similarity value is obtained based on the semantic similarity between the semantic vectors of the technical assets and the business entities, and this value is compared to a preset semantic similarity threshold. When the semantic similarity value is greater than or equal to the preset threshold, a semantic association is established between the technical asset instance and the business entity instance, thereby achieving automatic association between data assets and power grid business concepts.

[0066] Building upon the established semantic relationships, further rule binding relationships are established between business rule instances and technical asset fields based on field semantic matching and business scenario consistency. For example, when a business rule involves a specific monitoring indicator, the semantic meaning of the technical asset field is identified, and combined with its associated equipment type and application scenario, the corresponding rule is associated with the relevant field. Simultaneously, the data lineage information in the standardized dataset is parsed, and lineage relationships between data assets are constructed based on data source, processing records, and dependencies, thereby clarifying the source relationships and processing paths between different data assets.

[0067] Finally, the aforementioned semantic relationships, rule binding relationships, and lineage relationships are uniformly organized and stored to form a knowledge fusion model. In this model, technical asset instances, business entity instances, and business rule instances are connected through various relationships, enabling data resources, business concepts, and rule information to form a unified knowledge structure. This knowledge fusion model allows for deep integration between technical data and knowledge in the power grid domain, thereby supporting subsequent data retrieval enhancement, business correlation analysis, and data lineage tracing.

[0068] Specifically, this embodiment uses a combination of algorithmic calculation and rule constraints to establish structured relationships among the five major concepts. All relationships are stored in a graph database (such as Neo4j) and recorded in the format of "triples (subject-verb-object)" (such as "acetylene field-rule binding-acetylene content > 5 μL / L rule"). (1) Semantic association: The sentence vector generation function based on the pre-trained "Big Watt" model converts technical asset attributes (such as field annotations and table descriptions) and business entity definitions into 768-dimensional dense vectors. The cosine similarity algorithm is used to calculate vector similarity. The formula is: Similarity = cosθ = (A / B)co ... B) / (|A|×|B|), where A is the technical asset attribute vector and B is the business entity definition vector; Set association thresholds: similarity ≥ 0.8 is "strong association" (directly used for priority matching in retrieval), 0.6-0.8 is "weak association" (needs to be combined with other association dimensions to assist matching), and similarity < 0.6 does not establish an association; Example: "The field annotation 'Converter arc extinction angle monitoring value' (vector A) and the business entity definition 'commutation failure' (vector B) have a similarity of 0.89, establishing a strong association, and the association reason is marked as 'Abnormal arc extinction angle is the core cause of commutation failure'."

[0069] (2) Rule binding: A dual-condition matching logic of field semantic matching and business scenario alignment is adopted. First, the semantic similarity between the business rule description and the technical asset field annotation is calculated through the "big watt" model (threshold ≥ 0.7). Re-verify whether the applicable scenarios of the rules are consistent with the business domain to which the technical assets belong (e.g., the "oil chromatography anomaly threshold rule" only matches fields in tables related to oil chromatography analysis). After binding, mark the associated attributes: binding time, verification personnel, and rule effective date (e.g., "2025-01-15-Zhang San-2025-02-01 effective"). Example: The rule "acetylene content > 5 μL / L" has a semantic similarity of 0.85 with the "acetylene" field in the oil chromatography analysis table (annotated "dissolved acetylene content in transformer oil"), and both belong to the "equipment condition monitoring" business domain, thus completing the rule binding.

[0070] (3) Blood relation: Parse the lineage information in the metadata and extract the mapping relationship between "upstream assets - processing rules - downstream assets"; Use "processing type" as the associated predicate (such as "aggregation generation", "filter derivation", "Join association") to indicate the business meaning (such as "Table A (converter transformer real-time data table) - aggregation generation - Table B (converter transformer daily statistics wide table), used to calculate the daily operating efficiency index of the converter"). Additional processing details: aggregation rules (e.g., "group by station_id, partition by dt date, and calculate transmission power by sum"), processing tools (e.g., DataWorksETL jobs), and processing cycles (e.g., "execute daily at 00:30").

[0071] (4) Source tracing and correlation: Establish a "full-chain traceability chain" for technology assets, recording the following in a linked list structure: data source system → acquisition adapter → preprocessing node → storage location → update node → verification node; Label key traceability attributes: data source name (e.g., "fault recording system"), data acquisition time (e.g., "2025-03-10 14:30:00"), preprocessing rules (e.g., "null value filling, outlier removal"), update time (e.g., "2025-03-10 15:00:00"), verification personnel (e.g., "Li Si"), and verification result (e.g., "qualified"). The traceability chain supports reverse queries (such as tracing back to the original data source through a downstream wide table).

[0072] Dynamic model updates: The system integrates an AI flywheel technology control mechanism to achieve automated iteration of the ontology model. The specific process and triggering conditions are as follows: (1) Triggering conditions: User feedback triggers: The cumulative number of negative feedbacks for a single relationship is ≥3 (e.g., when a user queries "commutation failure", the "extinction angle field" matched by the system is marked as "irrelevant"), or the retrieval accuracy rate for a single business scenario is less than 90% (statistics on a weekly basis). New data access triggers: When a new data source is added (such as adding a new SCADA system for a converter station), a new business rule is added (such as updating the fault judgment threshold), or a new wide table is added (such as adding a "converter transformer status assessment wide table"), incremental updates will be automatically triggered. Scheduled Trigger: A full update is performed every Sunday at 02:00 by default, validating the validity of all relationships.

[0073] (2) Update process: Feedback / Data Collection: Collect user feedback data (relevance scores of associations, suggestions for correcting search results) or add new data; Association evaluation: The semantic similarity is recalculated using the "Big Watt" model, and the effectiveness of existing associations is evaluated by combining the pre-set rules of business experts (such as new business rules that require prioritizing the binding of relevant fields). Incremental update: Delete associations with "≥3 negative feedbacks", establish associations for new instances generated from newly added data, and update the strength level of associations with semantic similarity changes exceeding 0.1 (e.g., strong association → weak association). Review and Release: The updated model must be reviewed by one algorithm engineer and one power grid business expert. After the review is approved, it will be released to the production environment, while retaining historical versions (supporting rollback).

[0074] S4: Receive user natural language query requests, perform semantic parsing on the natural language query requests, extract the business entities and constraints involved in the query, and generate parsing results; This includes receiving user natural language query requests, performing semantic parsing on the natural language query requests, extracting the business entities and constraints involved in the query, and generating parsing results, including: Receive user natural language query requests, perform word segmentation on the natural language query requests, and combine them with a power grid domain entity dictionary to perform named entity recognition and extract business entity information; Identify the temporal, spatial, and business indicator descriptions in natural language query requests, and convert the identification results into structured constraints. Based on a pre-defined ontology model for the power grid domain, business entity information is matched with business entity instances in the ontology, and semantic expansion is performed based on synonym relationships and hierarchical relationships. Query expressions are constructed based on business entity information and structured constraints, and parsed results are generated.

[0075] In this embodiment, the system first receives a natural language query request from the user and performs word segmentation on the query statement. Simultaneously, it combines this with a power grid entity dictionary for named entity recognition to extract business entity information from the query. For example, in the query statement "Characteristics of commutation failure events and associated inspection records at the Suidong Converter Station in the past three months," entities such as "Suidong Converter Station," "commutation failure," and "inspection records" can be identified. "Commutation failure" is a power grid operation event-type business entity, "Suidong Converter Station" is an equipment site entity, and "inspection records" is an operation management record-type entity, thus obtaining the core business object of the query.

[0076] After extracting the business entities, the temporal expressions, spatial expressions, and business indicator descriptions in the natural language query are identified and converted into structured constraints. For example, "the last three months" is converted into the corresponding time interval, and "Suidong Converter Station" is mapped to the corresponding site information. At the same time, the indicator requirements in the query, such as "event characteristics" or "fault information," are identified. Subsequently, based on a pre-set power grid domain ontology model, the identified business entities are matched with business entity instances in the ontology, and semantic expansion is performed through synonym relationships and hierarchical relationships. For example, expressions such as "commutation unsuccessful" are uniformly merged into "commutation failed" to improve the consistency of query semantics.

[0077] After completing entity matching and semantic expansion, a query expression is constructed based on business entity information and structured constraints to generate a parsed result. This parsed result includes information such as core business entities, time constraints, spatial constraints, and indicator requirements. For example, it forms a structured expression such as "business entity is commutation failure, station is Suidong converter station, time range is the past three months, and query content is event characteristics and related inspection records," thereby converting the natural language query into a standardized parsed result that can be used for subsequent retrieval processing.

[0078] S5: Perform a hybrid search based on the parsing results to obtain candidate data assets; Among them, a hybrid retrieval is performed based on the parsing results to obtain candidate data assets, including: Construct a search request based on the business entity information and structured constraints in the parsing results; Based on the search request, keyword matching search is performed on the technical asset name, field annotation and business tag to obtain the first search result; The retrieval request is semantically vectorized and its similarity is calculated with the semantic vector of the technology assets in the knowledge fusion model to obtain the second retrieval result. The first and second search results are merged and deduplicated, and then scored and sorted according to the preset relevance calculation rules. Data assets with relevance scores higher than a preset threshold are selected as candidate data assets.

[0079] In this embodiment, after generating the parsing results, a retrieval request is constructed based on the business entity information and structured constraints in the parsing results. For example, the core business entities, extended entities, and constraints such as time range and spatial range in the query are uniformly organized to form a request content for data asset retrieval. Subsequently, based on the retrieval request, keyword matching retrieval is performed on the technical asset name, field annotations, and business tags to obtain the first retrieval result. For example, based on keywords such as "phase switching failure" and "inspection record," corresponding technical assets such as fault event tables and inspection record tables can be matched. By performing literal matching on the technical asset name, field description, and business tags, the rapid retrieval of basic data assets is achieved.

[0080] After obtaining the first search result, the search request is semantically vectorized and its similarity is calculated with the semantic vectors of technical assets in the knowledge fusion model to obtain the second search result. Semantic retrieval can identify data assets that are semantically related to the query but do not appear directly in the keywords. For example, when searching for "commutation failure event," it can further match real-time monitoring data tables or fault analysis data tables containing relevant monitoring indicators. Combining the semantic relationships in the knowledge fusion model, the retrieval of relevant data assets is expanded, thereby improving the completeness of the search results.

[0081] After obtaining the first and second search results, the two types of results are merged and deduplicated, and then scored and sorted according to preset relevance calculation rules. Data assets with relevance scores higher than a preset threshold are then selected as candidate data assets, such as fault event tables, real-time monitoring data tables, and patrol record tables. By combining keyword retrieval and semantic retrieval, the recall capability of data assets can be improved while ensuring retrieval accuracy, thereby obtaining a more complete set of candidate data assets.

[0082] S6: Use a knowledge fusion model to perform association expansion and lineage tracing on candidate data assets to obtain associated data results; Among these methods, a knowledge fusion model is used to extend and trace the lineage of candidate data assets, resulting in associated data outcomes, including: Starting with candidate data assets, the business entity instances and business rule instances associated with the candidate data assets are traversed in the knowledge fusion model to obtain semantically associated extended data assets. Based on the rule binding relationship in the knowledge fusion model, determine the business rule information associated with the corresponding fields of the candidate data assets; Based on the data lineage relationship in the knowledge fusion model, the lineage of candidate data assets is traced to obtain information on their upstream data sources and downstream derived data assets. The extended data assets, business rule information, and lineage tracing information are integrated to obtain the associated data results.

[0083] In this embodiment, after obtaining candidate data assets, a knowledge fusion model is used to perform association and expansion processing on the candidate data assets. Specifically, taking the candidate data asset as the starting node, the knowledge fusion model traverses the business entity instances and business rule instances associated with it, thereby identifying extended data assets semantically related to the data asset. For example, when the candidate data asset contains a fault event table or a real-time monitoring data table, the data tables or monitoring fields related to the business event can be further identified based on the semantic association between it and business entities such as "phase failure," thereby obtaining semantically associated extended data assets, enabling the originally single data asset to be expanded into a data set related to the business scenario.

[0084] After acquiring the extended data assets, the business rule information corresponding to the candidate data asset fields is further determined based on the rule binding relationships in the knowledge fusion model. Specifically, by identifying key fields in the candidate data assets and combining the matching relationship between field semantics and business rule descriptions, the business rules related to those fields are determined. For example, for monitoring fields related to equipment operating status, the corresponding operation judgment rules or fault identification rules can be identified, thereby obtaining the relevant business rule information and establishing a clear link between data assets and business rules.

[0085] Subsequently, based on the data lineage relationships in the knowledge fusion model, candidate data assets undergo lineage tracing processing. By analyzing the source and processing relationships between data, the upstream data source information of candidate data assets and the downstream derived data assets generated from them can be identified. For example, it can be traced that a certain statistical analysis data table originates from a real-time monitoring data table and a fault event data table, while also identifying the statistical reports or analysis datasets generated by this data table in subsequent analysis processes. Finally, the extended data assets, business rule information, and lineage tracing information are integrated to form a complete set of associated data results, thus providing a foundation for the subsequent organization and output of data results.

[0086] Specifically, in this embodiment, when a user initiates a natural language query (such as "characteristics of commutation failure events and related inspection records of Suidong Converter Station in the past three months"), retrieval enhancement is achieved based on a three-stage process of "query parsing → hybrid retrieval → result assembly". The specific technical details are as follows: 1. In-depth query analysis: Employing a combination of NLP entity recognition and ontology knowledge expansion, it accurately extracts core query information and expands semantic coverage. (1) Concept identification: The pre-trained "Big Watt" model's Named Entity Recognition (NER) module is invoked, and combined with a power grid entity dictionary (including classification tags for business entities, technical assets, constraints, etc.), three types of core information are extracted from the query: Business entities: Phase switching failure, inspection record (identified as "event type" or "document type"); Technical constraints: Suidong Converter Station (spatial constraint, marked "Converter Station Code: SDSL-001"), the past three months (time constraint, automatically converted to a time range "90 days prior to the current date", formatted as "YYYY-MM-DD to YYYY-MM-DD"). Related requirements: Event characteristics (marked "Required indicator data: number of failures, average arc extinction angle, duration"), Related records (marked "Required data tables: inspection record table, fault handling form table"); Regular expressions are used to help extract fuzzy constraints (such as "recent X months" or "top N"). The regular expression formula is: / recent(\d+) months|top(\d+) people|past(\d+) days / , which is automatically converted into quantitative conditions after matching.

[0087] (2) Conceptual expansion: Synonym expansion: Based on the thesaurus of the power grid field (from GB / T 2900 (all parts) Electrical Terminology, Power Terminology (Third Edition) etc. + manual annotation), expand the synonym expressions of core concepts (such as "commutation failure" to "commutation unsuccessful", "commutation interruption", "commutation failure caused by abnormal arc extinction angle"); Expanding hierarchical relationships: Utilizing the "business entity-subclass" association in the ontology model (e.g., the higher-level concept of "commutation failure" is "DC system fault", and the lower-level concepts are "single-phase commutation failure" and "three-phase commutation failure"), the scope of retrieval is expanded; Extended constraints: Only extended concepts related to the query scenario are retained (e.g., the extended concept of "commutation failure" does not include non-DC domain concepts such as "distribution network failure"), to avoid redundancy.

[0088] 2. Hybrid Joint Search: A weighted hybrid strategy combining the BM25 algorithm (weight 0.3) and the vector space model (weight 0.7) is employed. Three types of search tasks are executed in parallel using multiple threads. All search operations are collaboratively completed using a graph database (Neo4j) and a vector database (Milvus). (1) Retrieval execution logic: Parallel triggering: After the query is parsed, a retrieval request is sent to both the graph database (which stores ontology relationships) and the vector database (which stores vector representations of technical assets / business entities) to improve response speed (single query retrieval time ≤ 500ms). Result deduplication: The three types of search results are deduplicated using the "unique asset ID" (such as "Table-T001-Fault Recording Table"), retaining the matching record with the highest confidence.

[0089] (2) Specific implementation of the three types of retrieval: Basic Search: For structured metadata, the BM25 algorithm is used for exact literal matching. The search fields include the name, comments, and business tags of technical assets (such as matching the keyword "fault_waveform table" with the keyword "fault waveform"). For natural language queries, semantic initial matching is performed using a vector database, the cosine similarity between the query vector and the asset vector is calculated (threshold ≥ 0.7), and candidate results are filtered. Example: The query "inspection record" directly matches the technical asset "inspection_record table" (BM25 score 0.85, vector similarity 0.82) and is included in the candidate set.

[0090] Related extended search: Taking the business entities extracted by query parsing as the core, traverse the "association edges" (semantic association, rule binding) in the graph database and recursively query the association nodes of the first and second layers; Semantic association extension: Starting from the "commutation failure" entity, find all strongly related (similarity ≥ 0.8) technical assets (such as the scada_rt_data table and the commutator monitoring table); Rule binding extension: Find the business rules associated with "commutation failure" (such as "arc extinction angle < 10°" or "converter valve trigger angle abnormality"), and then locate the corresponding fields (such as the "extinction_angle" field in the scada_rt_data table) through the rule binding relationship. Extend the results and label the associated paths (e.g., "commutation failure - semantic association - scada_rt_data table").

[0091] Bloodline tracing search: Starting with the technical assets obtained from basic retrieval and related extended retrieval, we trace upstream data sources backward and downstream derivative assets forward along the "bloodline relationship" edge; Reverse tracing: Tracing back to the upstream "scada_raw_data table" through the "fault_waveform table" (processing type: filter derivation, processing rule: extract data during the fault period); Forward lookup: Search the downstream "commutation failure analysis wide table" through the "scada_rt_data table" (processing type: aggregated generation, processing rule: daily statistics of failure counts); The tracing depth is limited to 3 levels to avoid performance degradation caused by an excessively large retrieval range.

[0092] S7: Perform confidence assessment and structured organization on the associated data results, generate enhanced search results, and output them.

[0093] In this embodiment, after obtaining the associated data results, a confidence assessment is first performed on each associated data. This confidence assessment is mainly based on a comprehensive calculation of multiple evaluation factors, such as the degree of association between the data and the query semantics, the frequency of data usage in actual business scenarios, and the authority of the relevant rule source. Specifically, the degree of association indicates the matching degree between the candidate data and the user's queried business entity; the frequency of usage indicates the number of times the data is used in query analysis, report statistics, or data retrieval; and the authority of the rule source indicates whether the relevant rules or knowledge sources come from authoritative materials such as industry standards, corporate specifications, or technical documents. By comprehensively calculating the above factors according to preset weights, a confidence value corresponding to each associated data result can be obtained.

[0094] Specifically, the confidence weighted calculation is as follows: Weighting formula: Confidence score = Association strength score × 0.5 + Data popularity score × 0.1 + Source authority score × 0.4; The scores for each dimension are calculated as follows: Association strength score: strong association (similarity ≥ 0.8) gets 1 point, weak association (0.6-0.8) gets 0.7 points, no association gets 0 points; Data popularity score: 1 point for ≥50 queries in the past 30 days, 0.8 points for 20-49 queries, 0.6 points for 10-19 queries, and 0.4 points for <10 queries; Source authority score: industry standard gets 1 point, internal standard gets 0.8 points, and rule of experience gets 0.6 points; Filtering rules: Core assets with a confidence score of ≥0.9 will be displayed first, and results with a score <0.7 will not be output (to avoid interference from low-confidence data); Example: The scada_rt_data table has a correlation strength score of 1, a data popularity score of 0.8 (35 queries in the last 30 days), a source authority score of 1, and a confidence score of 1×0.5+0.8×0.1+1×0.4=0.98, so it is given priority for display.

[0095] After obtaining the confidence scores, the related data results are sorted and filtered. Specifically, the data is sorted from highest to lowest confidence score, and data results with higher confidence scores are filtered out based on a preset threshold to reduce interference from irrelevant data in the search results. Subsequently, the filtered data is structured and organized, unifying core data asset information, field descriptions, data source information, and relationship information relevant to the user query. For example, the data asset name, field meaning, applicable business scenario, data source, and relationship with business entities are integrated and described, forming a structured data result.

[0096] Furthermore, the processed results are used to generate enhanced search output and returned to the user. The output includes not only data asset information directly related to the user's query, but also descriptions of relevant fields, business rules, and the relationships between data. For example, when a user queries data related to a specific type of power grid fault, the output may include data tables related to that fault, key monitoring fields, fault determination rules, and corresponding data source descriptions, as well as demonstrating the relationships between the data. Through this approach, users can understand the business connections and source basis of the data while obtaining the required information, thereby improving the accuracy and interpretability of the data retrieval results.

[0097] This application also provides a modeling and retrieval enhancement system that integrates multi-source data and domain knowledge, the system comprising: The data source layer is used to collect multi-source heterogeneous metadata and standardized documents in the power grid field. The fusion processing layer is used to perform fusion modeling and retrieval enhancement processing on the data source layer, and performs the following processing: Metadata extraction and cleaning are performed on multi-source heterogeneous metadata, and document parsing and knowledge structuring are performed on standardized documents in the power grid field to obtain a standardized dataset. Based on a pre-defined ontology model for the power grid domain, a knowledge fusion model is generated by instantiating, mapping, and associating standardized datasets. Receive user natural language query requests, perform semantic parsing on the natural language query requests, extract the business entities and constraints involved in the query, and generate parsing results; Perform a hybrid search based on the parsing results to obtain candidate data assets; A knowledge fusion model is used to extend and trace the lineage of candidate data assets to obtain associated data results. The confidence level of the associated data results is assessed and the data is structured to generate enhanced search results. The application service layer is used to provide data services based on enhanced search results. These data services include an intelligent data retrieval interface, self-service data retrieval service, report generation service, and application programming interface.

[0098] refer to Figure 4 As shown, Figure 4 This is a schematic diagram of the overall architecture of the modeling and retrieval enhancement system for multi-source data and domain knowledge fusion according to the present invention. The architecture mainly includes a data source layer, a fusion processing layer, and an application service layer, achieving unified fusion and intelligent retrieval of power grid data resources and domain knowledge through a multi-layered structure.

[0099] like Figure 4 As shown, the data source layer mainly includes structured data, semi-structured data, unstructured data, and a domain knowledge base. Structured data can come from business systems such as operational databases and equipment management databases; semi-structured data includes log data and business messages; unstructured data includes textual materials such as technical documents and operational reports; and the domain knowledge base contains knowledge information such as power grid terminology, business rules, and equipment codes. These data collectively constitute the system's data foundation, providing data sources for upper-layer data processing and knowledge fusion.

[0100] In the fusion processing layer, multi-source data is collected, cleaned, and standardized through metadata processing. Based on the power grid domain ontology model, technical metadata, domain knowledge, and business rules are fused and modeled to form a knowledge fusion model. On this basis, a retrieval enhancement module performs hybrid retrieval on user queries, extracting data assets relevant to the query semantics from the knowledge fusion model and returning the retrieval results to the application service layer. Simultaneously, in the application service layer, data services are provided to users through intelligent data retrieval interfaces, self-service data retrieval services, report generation services, and API interfaces. User feedback mechanisms are used to continuously optimize the model, thereby improving data retrieval and service capabilities.

[0101] It is understood that the content of the above method embodiments is applicable to the present device embodiments. The specific functions implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0102] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0103] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0104] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0105] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0106] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0107] It is understood that the content of the above method embodiments is applicable to the embodiments of this program product. The specific functions implemented by the embodiments of this program product are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0108] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0109] This application provides a modeling and retrieval enhancement method and system for fusing multi-source data and domain knowledge. This solution extracts, cleans, and structures multi-source heterogeneous metadata and standardized documents in the power grid domain, and constructs a knowledge fusion model based on a pre-defined power grid domain ontology model. This achieves semantic association and unified expression between data resources and domain knowledge, effectively solving the problems of complex power grid data sources, large structural differences, and scattered semantic relationships, thus improving the standardization and manageability of data organization. Simultaneously, by semantically parsing user natural language query requests, extracting business entities and constraints, and combining a hybrid retrieval strategy to obtain candidate data assets, collaborative optimization can be achieved between structured retrieval and semantic retrieval, thereby improving the accuracy and matching efficiency of data retrieval. Furthermore, by utilizing the knowledge fusion model to extend and trace the lineage of candidate data assets, and by evaluating the confidence level and structuring the retrieval results, the final enhanced retrieval results not only possess stronger semantic association capabilities but also provide clear data sources and relationships, effectively improving the intelligence level and reliability of power grid domain data retrieval.

[0110] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0111] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0112] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0113] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0114] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A modeling and retrieval enhancement method that integrates multi-source data and domain knowledge, characterized in that, The method includes the following steps: Acquire multi-source heterogeneous metadata and standardized documents in the power grid field; Metadata extraction and cleaning are performed on the multi-source heterogeneous metadata to obtain standardized metadata. Document parsing and knowledge structuring are performed on the standardized documents in the power grid field to obtain standard domain knowledge data. The standardized metadata and the standard domain knowledge data are combined to form a standardized dataset. Based on a pre-defined ontology model for the power grid domain, the standardized dataset is instantiated, mapped, and associated to generate a knowledge fusion model. Receive a user's natural language query request, perform semantic parsing on the natural language query request, extract the business entities and constraints involved in the query, and generate parsing results; Based on the analysis results, a hybrid search is performed to obtain candidate data assets; The knowledge fusion model is used to perform association expansion and lineage tracing on the candidate data assets to obtain associated data results. The associated data results are evaluated for confidence and organized in a structured manner to generate and output enhanced search results.

2. The method according to claim 1, characterized in that, The process of extracting and cleaning the multi-source heterogeneous metadata to obtain standardized metadata includes: Technical metadata is extracted from multi-source heterogeneous metadata corresponding to different types of data sources through the data source adapter to obtain the original metadata; The original metadata is cleaned to remove null and abnormal data, and duplicate records are deleted by hash comparison. The cleaned metadata is formatted and standardized. Based on a thesaurus of the power grid domain, the terms in the metadata are ambiguously resolved and semantically normalized to obtain standardized metadata.

3. The method according to claim 2, characterized in that, The data sources include relational databases, non-relational databases, data warehouses, and ETL tools; the raw metadata includes table structure information, field attribute information, and data lineage information.

4. The method according to claim 1, characterized in that, The process of parsing and structuring standardized documents in the power grid field to obtain standard domain knowledge data includes: Text parsing is performed on standardized documents in the power grid field to extract original domain knowledge data; the standardized documents include technical specification documents, operating procedure documents, and equipment management documents. Based on natural language processing technology, entity recognition and rule extraction are performed on the original domain knowledge text to obtain power grid domain knowledge information; the power grid domain knowledge information includes power grid business terminology information, equipment coding information, and business rule information. The domain knowledge information is structured and modeled, and attribute annotations are performed on terminology definitions, rule thresholds, and applicable scenarios to generate standard domain knowledge data.

5. The method according to claim 1, characterized in that, The method, based on a pre-defined power grid domain ontology model, instantiates, maps, and associates the standardized dataset to generate a knowledge fusion model, including: The technical metadata, domain knowledge information, and business rule information in the standardized dataset are mapped to corresponding instances in the preset power grid domain ontology model, and a unique identifier is assigned to each instance. The instance is labeled with attributes to generate attribute information; the attribute label includes technology name, data type, storage location, update cycle, terminology definition, applicable scenarios and rule threshold information; Based on the attribute information, semantic similarity is calculated between technical asset instances and business entity instances, and semantic association is established according to a preset similarity threshold. Based on field semantic matching and business scenario consistency verification, a rule binding relationship is established between business rule instances and corresponding technical asset fields; Analyze the data lineage information in the standardized dataset to construct the lineage relationships between data assets; The semantic relationships, rule binding relationships, and bloodline relationships are stored in a unified manner to generate the knowledge fusion model.

6. The method according to claim 5, characterized in that, The step of calculating semantic similarity between technical asset instances and business entity instances based on the attribute information, and establishing semantic associations according to a preset similarity threshold, includes: Based on the attribute information, semantic vectorization processing is performed on the technical asset instance and the business entity instance to obtain the technical asset semantic vector and the business entity semantic vector. Semantic similarity is calculated based on the semantic vector of the technical asset and the semantic vector of the business entity to obtain a semantic similarity value; The semantic similarity value is compared with a preset semantic similarity threshold. When the semantic similarity value is greater than or equal to the semantic similarity threshold, a semantic association relationship is established between the technical asset instance and the business entity instance.

7. The method according to claim 1, characterized in that, The process of receiving a user's natural language query request, performing semantic parsing on the natural language query request, extracting the business entities and constraints involved in the query, and generating parsing results includes: Receive user natural language query requests, perform word segmentation on the natural language query requests, and combine them with a power grid domain entity dictionary to perform named entity recognition and extract business entity information; The temporal expression, spatial expression, and business indicator description in the natural language query request are identified, and the identification results are converted into structured constraints. Based on a pre-defined power grid domain ontology model, the business entity information is matched with business entity instances in the ontology, and semantic expansion processing is performed based on synonym relationships and hierarchical relationships. A query expression is constructed based on the business entity information and the structured constraints, and a parsing result is generated.

8. The method according to claim 1, characterized in that, The process of performing a hybrid retrieval based on the parsing results to obtain candidate data assets includes: A search request is constructed based on the business entity information and structured constraints in the parsing results; Based on the search request, keyword matching search is performed on the technical asset name, field annotation and business tag to obtain the first search result; The search request is semantically vectorized and its similarity to the semantic vector of the technology assets in the knowledge fusion model is calculated to obtain a second search result. The first search result and the second search result are merged and deduplicated, and then scored and sorted according to a preset relevance calculation rule; Data assets with a relevance score higher than a preset threshold are selected as candidate data assets.

9. The method according to claim 1, characterized in that, The process of using the knowledge fusion model to perform association expansion and lineage tracing on the candidate data assets to obtain associated data results includes: Starting with the candidate data asset, the business entity instances and business rule instances associated with the candidate data asset are traversed in the knowledge fusion model to obtain semantically associated extended data assets. Based on the rule binding relationship in the knowledge fusion model, determine the business rule information associated with the corresponding field of the candidate data asset; Based on the data lineage relationship in the knowledge fusion model, the candidate data assets are traced back to obtain their upstream data source information and downstream derived data asset information. The extended data assets, the business rule information, and the kinship tracing information are integrated to obtain the associated data results.

10. A modeling and retrieval enhancement system that integrates multi-source data and domain knowledge, characterized in that, The system includes: The data source layer is used to collect multi-source heterogeneous metadata and standardized documents in the power grid field. The fusion processing layer is used to perform fusion modeling and retrieval enhancement processing on the data from the data source layer, and to perform the following processing: Metadata extraction and cleaning are performed on the multi-source heterogeneous metadata, and document parsing and knowledge structuring are performed on the standardized documents in the power grid field to obtain a standardized dataset. Based on a pre-defined ontology model for the power grid domain, the standardized dataset is instantiated, mapped, and associated to generate a knowledge fusion model. Receive a user's natural language query request, perform semantic parsing on the natural language query request, extract the business entities and constraints involved in the query, and generate parsing results; Based on the analysis results, a hybrid search is performed to obtain candidate data assets; The knowledge fusion model is used to perform association expansion and lineage tracing on the candidate data assets to obtain associated data results. The associated data results are assessed for confidence and organized in a structured manner to generate enhanced search results; The application service layer is used to provide data services based on the enhanced search results. The data services include an intelligent data query interface, a self-service data retrieval service, a report generation service, and an application programming interface.

Images