A multimodal retrieval and intelligent question-answering method and system for power distribution equipment operation and maintenance.

By employing retrieval-enhanced generation technology in the operation and maintenance of power distribution equipment, combined with equipment topology constraints and intent awareness, unified retrieval and accurate question answering of unstructured and structured data are achieved. This solves the problems of low efficiency and insufficient security in the acquisition of operation and maintenance knowledge in existing technologies, thereby improving operation and maintenance efficiency and security.

CN123115401APending Publication Date: 2026-11-17POWER RES INST OF STATE GRID SHAANXI ELECTRIC POWER CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611260945.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-08-19
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing technologies for power distribution equipment operation and maintenance suffer from problems such as low efficiency in unstructured knowledge retrieval, high threshold for structured data query, difficulty in unified retrieval of heterogeneous data, and insufficient adaptability of RAG technology in the industrial field, resulting in low efficiency and insufficient security in acquiring operation and maintenance knowledge.

Method used

This system employs Retrieval Augmentation (RAG) technology, integrating hybrid semantic retrieval of unstructured documents with structured operation and maintenance data. Through an intent-aware intelligent routing and scheduling mechanism, it achieves unified retrieval and accurate question answering of heterogeneous knowledge in the field of power distribution equipment operation and maintenance. Specific measures include domain-adaptive segmentation based on equipment topology constraints, vectorized storage, and domain knowledge graph construction. It combines intent recognition and classification, uses hybrid semantic retrieval channels and Text2SQL query channels, and performs heterogeneous alignment fusion and quality self-assessment.

Benefits of technology

It effectively integrates unstructured operation and maintenance documents and structured data, improves the accuracy and efficiency of operation and maintenance knowledge acquisition, lowers the operational threshold, ensures the security and integrity of retrieval, and meets the strong audit requirements of power equipment operation and maintenance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123115401A_ABST
    Figure CN123115401A_ABST
Patent Text Reader

Abstract

This invention provides a multimodal retrieval and intelligent question-answering method and system for power distribution equipment operation and maintenance, belonging to the interdisciplinary field of artificial intelligence and power equipment operation and maintenance technology. The method includes: adaptively partitioning unstructured operation and maintenance documents based on equipment topology constraints, vectorizing storage, and constructing a knowledge graph; accessing structured data sources and establishing a schema semantic index; identifying and classifying user questions, and distributing them to a hybrid semantic retrieval channel or a Text2SQL query channel; performing multi-path recall and fusion reordering on hybrid semantic retrieval to obtain unstructured results; constructing prompts, generating SQL, and self-verifying and repairing Text2SQL queries through dynamic schema pruning, and executing the query to obtain structured results; for mixed-type questions, heterogeneous alignment and fusion of the two types of results to generate traceable answers. This invention integrates the semantic retrieval of unstructured documents with the natural language query capabilities of structured data, achieving unified retrieval and accurate question answering of heterogeneous knowledge in the field of power distribution equipment operation and maintenance.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of artificial intelligence and power equipment operation and maintenance technology, specifically involving a multimodal retrieval and intelligent question-answering method and system for power distribution equipment operation and maintenance. Background Technology

[0002] Power distribution equipment is a crucial component of the power system, and its safe and reliable operation directly impacts power supply quality and grid stability. During the operation and maintenance of power distribution equipment, personnel need to consult a large amount of technical documents, equipment specifications, fault cases, maintenance records, and other relevant materials. They also need to access structured data such as equipment ledgers, operating parameters, and defect statistics. However, the acquisition of operation and maintenance knowledge currently faces the following prominent problems: First, unstructured knowledge retrieval is inefficient. The operation and maintenance of power distribution equipment involves a wide variety of technical documents, including equipment operating procedures, maintenance guidelines, fault handling plans, and typical cases. These documents are usually in unstructured formats such as PDF and Word. Traditional keyword retrieval methods struggle to understand the natural language query intent of maintenance personnel, resulting in low retrieval accuracy and difficulty in returning precisely matching knowledge fragments.

[0003] Secondly, the querying of structured data has a high barrier to entry. The power distribution equipment operation and maintenance management system has accumulated a large amount of structured data, such as equipment ledgers, inspection records, defect work orders, test reports, etc. This data is stored in a relational database. If operation and maintenance personnel need to obtain statistical analysis information, such as the top five 10kV switchgear devices with the most trips in the past three years, they need to write SQL query statements, which poses a high operational barrier for front-line operation and maintenance personnel.

[0004] Third, heterogeneous data is difficult to retrieve in a unified manner. In actual operation and maintenance scenarios, a problem often involves both unstructured knowledge (such as the handling steps in the operating procedure) and structured data (such as the historical defect records of the equipment). Existing systems usually separate document retrieval and database query into two independent functions, requiring operation and maintenance personnel to frequently switch between different systems, lacking a unified intelligent question answering and retrieval entry point.

[0005] Fourth, existing RAG (Retrieval-Augmented Generation) technology is not well adapted to the industrial field. General-purpose RAG systems have been applied in open-domain question answering, but their direct application in the field of power distribution equipment operation and maintenance faces many challenges: inaccurate semantic understanding of domain-specific terminology, document segmentation strategies do not consider the logical structure of operation and maintenance knowledge, search results lack reordering optimization for operation and maintenance scenarios, and they cannot adaptively select the optimal search strategy based on question type.

[0006] Therefore, there is an urgent need for an intelligent question-answering method and system that can unify and integrate unstructured operation and maintenance documents and structured operation and maintenance data, and has intent awareness and adaptive retrieval routing capabilities, in order to lower the threshold for acquiring operation and maintenance knowledge and improve operation and maintenance efficiency and quality. Summary of the Invention

[0007] To address the problems existing in the prior art, this invention provides a multimodal retrieval and intelligent question-answering method and system for power distribution equipment operation and maintenance. It adopts retrieval augmented generation (RAG) technology, integrates hybrid semantic retrieval of unstructured documents with natural language query capabilities of structured operation and maintenance data, and achieves unified retrieval and accurate question answering of heterogeneous knowledge in the field of power distribution equipment operation and maintenance through an intent-aware intelligent routing scheduling mechanism.

[0008] This invention provides a multimodal retrieval and intelligent question-answering method for the operation and maintenance of power distribution equipment, comprising the following steps: Unstructured operation and maintenance documents are subjected to domain-adaptive block partitioning based on device topology constraints, vectorized storage, and domain knowledge graph construction. They are also connected to structured data sources and a schema semantic index is established. The system receives user natural language questions, performs intent recognition and classification, and distributes the questions to the corresponding retrieval channels based on the classification results. The retrieval channels include a hybrid semantic retrieval channel and a Text2SQL query channel. For issues distributed to the hybrid semantic retrieval channel, unstructured operation and maintenance knowledge is retrieved through multiple paths and re-ranked to obtain unstructured retrieval results; for issues distributed to the Text2SQL query channel, suggestions are constructed through dynamic schema pruning, SQL statements are generated and self-verified and repaired, and the query is executed to obtain structured query results. For mixed query questions, the mixed semantic retrieval channel and the Text2SQL query channel are executed in parallel. The unstructured retrieval results and structured query results are heterogeneously aligned and fused to generate traceable answers, and the quality of the answers is self-evaluated.

[0009] Furthermore, the domain-adaptive segmentation based on device topology constraints includes: introducing a power distribution equipment topology hierarchy tree as a priori constraint; for the fault case library, using an identifier-based regular expression matching and semantic boundary detection method to forcibly segment the text into complete fault phenomenon-cause analysis-handling measures knowledge blocks, and encoding and binding them with device nodes in the power distribution equipment topology hierarchy tree; for operation procedure documents, extracting the operation sequence logic features from the text, and performing minimum-granularity segmentation according to uninterrupted operation steps.

[0010] Furthermore, receiving user natural language questions and performing intent recognition and classification includes: Using an intent classification model fine-tuned from a pre-trained language model, questions are categorized into knowledge-based question answering, data querying, casual conversation and refusal to answer, and mixed querying. Knowledge-based question answering questions are routed to the mixed semantic retrieval channel, data querying questions are routed to the Text2SQL query channel, and mixed querying questions are decoupled into knowledge-based question answering sub-questions and data querying sub-questions, which are then routed and processed separately. Casual conversation and refusal to answer questions are responded to with a refusal to answer.

[0011] Furthermore, the multi-path recall of the hybrid semantic retrieval channel includes: vector semantic retrieval, BM25 keyword retrieval and knowledge graph retrieval. The three recall results are deduplicated and then reordered by a cross encoder to output Top-N as the generated context. The vector semantic retrieval adopts a text embedding model fine-tuned by power domain corpus, and the cross encoder adopts a cross encoder fine-tuned by power distribution domain labeled data; the knowledge graph retrieval identifies the equipment entities in the query, performs multi-hop reasoning along the fault mode and component association edges in the knowledge graph, and recalls the associated graph path.

[0012] Furthermore, the dynamic schema pruning includes two levels of pruning: the first level is semantic table recall, which calculates the similarity between the query vector and the table-level description vector of the registered data tables, and selects the top K tables with the highest similarity ranking to include in the candidate table set, where K is the number of registered data tables selected; the second level is topological constraint-based field pruning, which, within the candidate table set, forcibly retains the device association ID field necessary for cross-table join queries based on the foreign key lineage between database tables, removes other redundant fields, and dynamically constructs prompt words; The SQL self-verification and repair includes: performing syntax and existence checks, value range checks, and security checks on the generated SQL statements in sequence. If the checks fail, the error information is fed back to the large language model for retry and correction, with a maximum of 3 retries. The security check uses a regular expression matching mechanism to intercept high-risk operation statements containing DROP, DELETE, UPDATE, and ALTER.

[0013] Furthermore, the heterogeneous alignment and fusion of unstructured retrieval results and structured query results includes: entity alignment, time alignment, and semantic alignment; the entity alignment is based on a pre-built ternary mapping table of device asset codes-standard terms-colloquial aliases, which bidirectionally associates document device descriptions with database device codes; the time alignment adopts a time-series standardization strategy to convert discrete timestamps in structured data and relative times in unstructured documents into absolute time windows; the semantic alignment includes preserving original text fragments on the unstructured side and converting two-dimensional data tables into natural language descriptions on the structured side through a template engine.

[0014] Furthermore, the generation of traceable answers includes: in the aligned unified context, marking the source of each assertion with a special delimiter, the source including document page number or SQL query statement; the quality self-assessment includes using a large language model evaluation mechanism to score the generated answer from three dimensions: factual consistency, answer completeness and traceability completeness, and if it is lower than a preset threshold, the query optimization process is triggered.

[0015] Secondly, this invention provides a multimodal retrieval and intelligent question-answering system for power distribution equipment operation and maintenance, including a knowledge base construction module, an intent recognition and routing scheduling module, a hybrid semantic retrieval module, a Text2SQL query module, and a fusion generation module: The knowledge base construction module is used to collect power distribution equipment operation and maintenance documents, perform domain-adaptive block partitioning based on equipment topology constraints, vectorized storage, and domain knowledge graph construction. It also completes the registration of structured data sources and the construction of schema semantic indexes. The intent recognition and routing module is used to receive user natural language questions, classify the intents, and distribute the questions to the corresponding retrieval channels according to the classification results; the retrieval channels include a hybrid semantic retrieval channel and a Text2SQL query channel; The hybrid semantic retrieval module is used to perform multi-path recall and fusion re-ranking of unstructured operation and maintenance knowledge to obtain unstructured retrieval results; The Text2SQL query module is used to construct hints, generate SQL statements, and perform self-verification and repair for issues distributed to the Text2SQL query channel through dynamic schema pruning, and execute queries to obtain structured query results. The fusion generation module is used to heterogeneously align and fuse unstructured retrieval results and structured query results obtained when performing parallel execution of mixed query types of queries using the mixed semantic retrieval channel and the Text2SQL query channel, generating traceable answers, and performing quality self-evaluation on the answers.

[0016] Thirdly, the present invention also provides a computer device, including a processor and a memory, wherein the memory is used to store a computer executable program, the processor reads the computer executable program from the memory and executes it, and when the processor executes the computer executable program, it implements the multimodal retrieval and intelligent question answering method for the operation and maintenance of power distribution equipment as described in the present invention.

[0017] Simultaneously, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, it can realize the multimodal retrieval and intelligent question-answering method for the operation and maintenance of power distribution equipment as described in this invention.

[0018] Compared with the prior art, the present invention has at least the following beneficial effects: (1) Effectively bridge the semantic gap in heterogeneous data fusion and reduce the probability of illusions during cross-source fusion of large language models; Existing technologies often encounter misattribution issues when handling mixed document and database searches due to inconsistencies between Equipment Asset Management System (PMS) coding and colloquial naming conventions. This invention introduces a ternary mapping table that integrates equipment asset coding, standard terminology, and colloquial aliases, along with relative and absolute time window mapping algorithms. Before generating answers, it performs entity anchoring and time alignment on structured and unstructured maintenance data, breaking down retrieval barriers between maintenance documents and equipment ledgers, improving the consistency between the fusion context and the physical logic of the equipment, and effectively enhancing the retrieval accuracy of complex mixed queries.

[0019] (2) Overcome the technical bias of the general segmentation strategy and ensure the integrity and safety of industrial operation guidance.

[0020] Existing RAG (Random Access Graph) technologies generally employ mechanical segmentation based on word count. When applied to industrial scenarios, this can easily truncate procedural documents with strong temporal logic, leading to omissions in maintenance or operational steps and potentially causing safety incidents. This invention combines the topology hierarchy of power distribution equipment with the uninterrupted timing characteristics of operations to achieve domain-adaptive structured segmentation, such as a three-stage binding of fault phenomenon, cause analysis, and remedial measures. This ensures that the recalled knowledge fragment is a complete industrial guidance unit, enhancing the implementation security of maintenance plans.

[0021] (3) Under the premise of ensuring the security of industrial databases, break through the performance bottleneck of large language models in processing massive schemas.

[0022] Faced with a power distribution network operation and maintenance database containing thousands of tables, full schema injection can lead to large language model memory overflow or severe context forgetting. This invention designs a two-level pruning and filtering mechanism based on semantic and topological constraints to achieve accurate dynamic construction of local schemas. At the same time, it constructs a multi-dimensional SQL self-verification and self-repair closed loop covering the interception of data changes. This mechanism not only empowers front-line personnel with natural language database query capabilities, but also effectively reduces the risks of high-risk write operations and database deletion, strictly adhering to the safety red line of the power production environment.

[0023] (4) A multi-path complementary retrieval system was constructed, which enabled accurate retrieval of scattered and heterogeneous knowledge.

[0024] In response to the complex terminology and non-standardized knowledge structure of power distribution operation and maintenance, this invention not only introduces pre-defined standardized rewriting of domain terminology, but also integrates three methods for multi-path recall: vector semantic retrieval to capture fuzzy intent, BM25 keyword retrieval to ensure accurate hit of proper nouns, and knowledge graph retrieval to provide fault association path reasoning. Furthermore, it employs a domain-fine-tuned cross-encoder for reordering, significantly improving the recall rate and first-place hit rate in a massive heterogeneous operation and maintenance knowledge system.

[0025] (5) Achieve full-chain closed-loop optimization and traceability of answers under strong audit requirements.

[0026] The operation and maintenance of power equipment demands extremely high accuracy of knowledge. This invention mandates the annotation of multimodal source traceability information (accurate to document page numbers or SQL query statements) for each key assertion generated in the final product, ensuring the complete auditability of the answers. Simultaneously, it introduces a fact consistency and completeness self-evaluation mechanism based on LLM-as-Judge, enabling the system to possess self-correcting and iterative optimization capabilities through "retrieval-rewriting-regeneration" without manual intervention, reducing the system's later maintenance costs.

[0027] Furthermore, domain-adaptive segmentation based on device topology constraints avoids the word-based mechanical segmentation in general RAG technology, preventing the truncation of procedural documents with strong temporal logic and eliminating the risk of safety accidents caused by the omission of critical operational steps (such as power outages and grounding). This ensures that any recalled knowledge is a complete industrial guidance unit, enhancing the safety of implementing maintenance plans.

[0028] Furthermore, the system precisely categorizes users' natural language questions into knowledge-based questions, data queries, and mixed queries, and routes the questions to the most appropriate retrieval channel accordingly, employing a combination of semantic retrieval, Text2SQL, or both in parallel. Particularly noteworthy is the decoupling of mixed query types, breaking down complex questions into sub-questions for separate processing before final fusion. This allows the system to uniformly integrate unstructured maintenance documents and structured maintenance data, providing maintenance personnel with a unified intelligent question-and-answer entry point and avoiding frequent switching between different systems.

[0029] Furthermore, vector semantic retrieval captures the user's deep semantics and ambiguous intent; BM25 keyword retrieval ensures accurate targeting of long-tail keywords such as device-specific models and maintenance numbers; and knowledge graph retrieval provides the ability to reason about fault-related paths. These three approaches, combined and complementary, significantly improve the overall recall rate within a massive, heterogeneous maintenance knowledge system. A domain-fine-tuned cross-encoder is used for reordering, improving the first-order hit rate and ensuring that the most relevant knowledge fragments are placed first for the model to generate answers.

[0030] Furthermore, dynamic schema pruning, through two-level pruning, solves the problem of model memory overflow or context forgetting caused by full injection of a large number of data tables, and realizes lightweight and high-precision prompt word construction, enabling large language models to effectively handle Text2SQL tasks of massive databases; the security verification in the SQL self-validation and repair mechanism intercepts any high-risk write operations such as DROP, DELETE, UPDATE and ALTER through regular expressions, completely eliminating the risk of database deletion or modification caused by incorrect SQL generated by the model. At the same time, through syntax and value range verification and a maximum of 3 closed-loop repairs, the success rate and accuracy of SQL generation are improved.

[0031] Furthermore, entity alignment utilizes a ternary mapping table to resolve the inconsistency between database device numbers and colloquial document naming, ensuring that the model correctly associates with the same physical device when fusing text and database information. Time alignment unifies the precise timestamps in the database and the relative times in the documents into absolute time windows, avoiding analysis errors caused by inconsistent time representations. Semantic alignment transforms data from different modalities into a uniformly formatted context that is easy for the model to understand, laying the foundation for generating accurate and logical answers.

[0032] Furthermore, the generation of traceable answers and quality self-assessment meet the industry's stringent auditing requirements. Each key assertion in the answer is forcibly annotated with source information accurate to the document page number or SQL query statement, ensuring the auditability of the answer, which is crucial for industries like power, where accuracy is extremely critical. An LLM-as-Judge mechanism is introduced to automatically assess the quality of answers from three dimensions: factual consistency, completeness, and traceability. This allows for self-correction and continuous improvement without manual intervention, reducing the system's later maintenance costs. Attached Figure Description

[0033] Figure 1 This is a schematic diagram of the overall process of the method described in this invention.

[0034] Figure 2 This is a schematic diagram of the intent recognition and retrieval routing scheduling mechanism.

[0035] Figure 3 This is a schematic diagram of the multi-path recall and fusion re-ranking process for hybrid semantic retrieval.

[0036] Figure 4 This is a schematic diagram of the Text2SQL dynamic query and self-verification repair process.

[0037] Figure 5 This is a schematic diagram of the process for aligning and merging heterogeneous search results and generating answers.

[0038] Figure 6This is a schematic diagram of the overall architecture of the system described in this invention. Detailed Implementation

[0039] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0040] This invention provides a multimodal retrieval and intelligent question-answering method and system for power distribution equipment operation and maintenance. By constructing an adaptive knowledge base based on equipment topology rules and designing an intent-aware routing mechanism, it decouples and parallelizes multi-path hybrid retrieval of unstructured documents with structured Text2SQL queries based on dynamic schema two-level pruning. Furthermore, relying on a pre-defined ternary mapping relationship table and a time window alignment algorithm, it performs forced anchoring and contextual fusion of entities on cross-source retrieval results. Through these methods, this invention not only alleviates the "illusion" and high-risk SQL operation risks caused by heterogeneous data fragmentation in industrial scenarios using large language models, but also helps break down retrieval barriers between operation and maintenance documents and equipment ledgers, providing frontline operation and maintenance personnel with a unified, accurate, and fully traceable natural language question-answering entry point. The multimodal nature of this invention refers to the fusion retrieval of various knowledge representation forms, such as unstructured document knowledge, structured database records, and knowledge graph relationships, in power distribution equipment operation and maintenance scenarios.

[0041] like Figure 6 As shown, Figure 6 This is a schematic diagram of the overall architecture of the system described in this invention. The multimodal retrieval and intelligent question-answering system for power distribution equipment operation and maintenance provided by this invention includes a knowledge base construction module, an intent recognition and routing scheduling module, a hybrid semantic retrieval module, a Text2SQL query module, and a fusion generation module. The knowledge base construction module is used to collect power distribution equipment operation and maintenance documents, perform domain-adaptive block partitioning based on equipment topology constraints, vectorized storage, and domain knowledge graph construction. It also completes the registration of structured data sources and the construction of schema semantic indexes.

[0042] The intent recognition and routing module receives user natural language questions, classifies the intents, and distributes the questions to the corresponding retrieval channels based on the classification results. This module includes an intent recognition unit and a routing unit. The intent recognition unit extracts feature words from the questions based on an intent classification model and classifies the intents according to these feature words. Classifications include knowledge-based question-and-answer, data query, casual conversation and rejection, and mixed query. The routing unit distributes the questions to the corresponding retrieval channels based on the classification results. Specifically, knowledge-based question-and-answer questions are routed to the mixed semantic retrieval channel, data query questions are routed to the Text2SQL query channel, casual conversation and rejection questions are intercepted and rejected, and mixed query questions are decoupled into knowledge-based question-and-answer sub-questions and data query sub-questions, which are then routed and processed separately.

[0043] The hybrid semantic retrieval module is used for multi-path retrieval and fusion reordering of unstructured operation and maintenance knowledge. The hybrid semantic retrieval module includes a query preprocessing and rewriting unit, a vector retrieval unit, a keyword retrieval unit, a knowledge graph retrieval unit, and a fusion reordering unit. The query preprocessing and rewriting unit uses a terminology dictionary for power distribution equipment to rewrite frontline maintenance jargon into standard terminology; the vector retrieval unit captures the deep semantic relationships of user queries and retrieves several closest semantic document fragments; the keyword retrieval unit performs BM25 keyword retrieval, using sparse matrix retrieval to improve the hit rate of long-tail keywords for equipment-specific models and specific maintenance numbers, and retrieves several keyword fragments; the knowledge graph retrieval unit identifies equipment entities in the query, performs multi-hop reasoning along fault modes and component association edges in the knowledge graph, and retrieves association graph paths; the fusion and reordering unit inputs the candidate fragments after the above three deduplications into a cross-encoder finely tuned to the power distribution corpus for query-document fine-grained attention interaction scoring, and outputs Top-N as the generated context.

[0044] The Text2SQL query module is used to convert natural language questions into SQL statements and execute queries. It includes a schema-aware prompt construction unit, an SQL generation unit, a self-validation and repair unit, and a query execution unit. The schema-aware prompt construction unit calculates the similarity between the query vector and the table-level description vector. Based on the foreign key relationships between database tables, it forcibly retains the device association ID field, which has no direct semantic connection to the current question but is necessary for cross-table join queries, while removing other redundant fields. It then generates a lightweight, high-precision prompt. The SQL generation unit uses a large language model to generate the initial SQL statement. The SQL self-validation and repair unit performs syntax checks, field existence checks, value range checks, and the most crucial security checks on the generated SQL statement. If any check fails, the error message is fed back to the large language model for correction, up to three times. The query execution unit queries the target database to obtain the result set after the checks pass.

[0045] The fusion generation module integrates results from different retrieval channels, generates traceable answers, and performs quality assessment. This module includes a heterogeneous result alignment and fusion unit, an answer generation and tracing unit, and a quality self-assessment unit. The heterogeneous result alignment and fusion unit performs entity alignment, temporal alignment, and semantic alignment respectively. The answer generation and tracing unit concatenates the aligned text into a unified context and inputs it into a large language model to generate answers. The quality self-assessment unit uses an LLM-as-Judge mechanism to score the generated answers, evaluating them from three dimensions: factual consistency, answer completeness, and traceability completeness. If the score is below a preset threshold, a query optimization process is triggered.

[0046] Please see Figure 1 , Figure 1 This is a schematic diagram of the overall process of the method described in this invention. The method specifically includes the following steps: Step S1: Construction of the power distribution equipment operation and maintenance knowledge base.

[0047] When constructing the knowledge base for power distribution equipment operation and maintenance, this invention differs from the traditional large language model's mechanical token chunking method, and instead introduces industrial logic constraints: S11, Unstructured Knowledge Construction: A topology hierarchy tree for power distribution equipment is introduced as a priori constraint. This tree covers substations, main transformers, busbars, switchgear, and feeder levels. For the fault case database, identifier-based regular expression matching and semantic boundary detection are used to forcibly segment the database into complete "fault phenomenon—cause analysis—response measures" knowledge blocks, and these knowledge blocks are encoded and bound to the equipment nodes in the power distribution equipment topology hierarchy tree. For operation procedure documents, the operational sequence logic features are extracted from the text, and the database is segmented at the smallest granularity according to uninterrupted operational steps. This method avoids dispersing key operational steps, including power outages, grounding, tagging, and voltage testing, into different knowledge blocks, reducing the risk of missing key steps during subsequent retrieval. After segmentation, the data is converted into vectors for database entry using a text embedding model. This model is based on general embedding models (such as BGE and M3E) and uses a low-rank adaptation (LoRA) method to fine-tune parameters using a power distribution domain corpus, making the model's semantic representation of power industry terminology more accurate. Furthermore, by utilizing a joint model of Named Entity Recognition (NER) and Relation Extraction (RE) based on pre-trained text in the power field, entities such as equipment, components, fault modes, and handling measures are extracted from knowledge blocks, as well as operation and maintenance semantic relationships such as occurrence-phenomenon, cause-leadership, and solution-treatment, to construct a domain knowledge graph.

[0048] S12, Structured Data Source Access and Schema Semantic Index Construction: Connect the structured data source to the operation and maintenance management system database, parse the table structure and build a schema semantic index. The schema semantic index includes table name, fields, enumeration value range and mapping of foreign key lineage between tables.

[0049] Step S2: User intent identification and route retrieval.

[0050] Please see Figure 2After receiving maintenance questions input by users in natural language, the intent classification model is used to classify the intent of the maintenance questions. This intent classification model adopts a text classification architecture based on a pre-trained language model (such as BERT), and is fine-tuned using a power distribution maintenance-specific corpus labeled with four categories: knowledge-based question answering, data querying, casual conversation and rejection, and mixed querying. The model takes the user's original question text as input, encodes it, and outputs the intent category probability through a fully connected classification layer. At the same time, it extracts key feature words from the question through attention weights, such as time range (e.g., the past three years, this month), statistical comparison words (e.g., most, top five, average), equipment number (e.g., 10kV No. 3 main transformer, PMS code), and operation actions (e.g., processing, querying, reason), as auxiliary basis for classification. Finally, the input question is determined to be one of the following four categories: (a) knowledge question and answer category, which only involves the retrieval of unstructured operation and maintenance knowledge, such as "how to handle the failure of vacuum circuit breaker to close", which will be routed to the hybrid semantic retrieval channel; (b) data query category, which only involves the statistical query of structured data, such as "the top five devices with the most trips in the switch cabinet in the past three years", which will be routed to the Text2SQL query channel; (c) hybrid query category, which includes both the retrieval needs of unstructured knowledge and the query needs of structured data, such as the reason for the recent overheating of No. 3 main transformer and the overheating record of the same type of main transformer last year. When the model detects that a question contains both feature words representing statistical queries and feature words representing operation and faults in knowledge retrieval, and the difference in classification probability is lower than a set threshold (e.g., 0.15), it is determined to be a mixed query class. At this time, the routing scheduling module uses a large language model to decouple the complex question into knowledge question-answering sub-questions (e.g., "cause of main transformer overheating") and data query sub-questions (e.g., "records of similar main transformer overheating last year"), and sends them to the mixed semantic retrieval channel and the Text2SQL query channel respectively. After parallel processing, the two channels are handed over to the fusion generation module for alignment and fusion. (d) Casual conversation and refusal to answer type: Inputs that are unrelated to operation and maintenance or violate security policies trigger security interception prompts and execute a refusal to answer response.

[0051] Step S3: Hybrid semantic retrieval for unstructured knowledge. See also... Figure 3 Specifically, it includes the following steps: S31, Query preprocessing: Using a terminology dictionary for the power distribution equipment field, the colloquialisms used by front-line maintenance personnel (for example: transformer humming, failure to trip) are rewritten into standard terms (for example: abnormal transformer noise, failure to operate fault), eliminating the interference of colloquial expressions on the accuracy of retrieval.

[0052] S32, multi-path parallel recall: (i) Vector semantic retrieval: A text embedding model finely tuned with corpus in the power field is used to vectorize the rewritten standard query. Through vector similarity calculation, several unstructured document fragments that are most similar to the query semantics are recalled. (ii) BM25 Keyword Retrieval: Sparse matrix retrieval is adopted. After segmenting the query text, precise keyword matching is performed to improve the hit rate of long-tail keywords such as equipment-specific models and maintenance numbers, and to recall a number of keyword hit fragments. (iii) Knowledge graph retrieval: Identify the equipment entity in the query (such as "10kV vacuum circuit breaker"), perform multi-hop reasoning along the fault mode and component association edges in the constructed domain knowledge graph, and recall the associated fault paths and corresponding text description fragments.

[0053] The results retrieved through the three methods, after deduplication, are collectively referred to as candidate fragments. These three results reflect semantic similarity, keyword matching, and graph correlation, respectively, describing the relationship between the query and the knowledge fragment from different dimensions.

[0054] S33, Fusion Reordering: The candidate segments obtained in S32 are jointly input into a cross-encoder finely tuned with labeled data from the power distribution domain for reordering. This cross-encoder is based on a pre-trained language model and uses a query-document pair approach for joint input. Through a self-attention mechanism, it performs fine-grained semantic interaction calculations between the query and each candidate segment, outputting a relevance score. Finally, it extracts the Top-N segments (N is a preset integer) in descending order of relevance score, which serve as the context for the large language model to generate the answer.

[0055] Step S4, for dynamic Text2SQL queries on structured data, please refer to [link / reference]. Figure 4 This step effectively overcomes the model illusion problem caused by full schema injection: S41, Text2SQL query: The first level is semantic-based table recall: The similarity between the user query vector and the table-level description vectors of all registered data tables is calculated. The tables are sorted in descending order of similarity, and the top K registered data tables with the highest similarity ranking are selected and included in the candidate table set. The remaining registered data tables are pruned and excluded. Here, K is the number of registered data tables selected, and the value of K can be preset according to the database size and model processing capacity. The second level involves field pruning based on topological constraints: Within the candidate table set, based on the foreign key lineage between database tables (i.e., table topological relationships), device association ID fields (such as primary keys and foreign keys) that are not directly semantically related to the current problem but are necessary for cross-table join queries are forcibly retained, while other redundant fields are removed. Subsequently, using the filtered database table and field information, a lightweight, high-precision prompt is dynamically constructed. This prompt only includes the candidate table name, the retained fields and their descriptions, and the range of enumerated values. As an example, a cross-table join query could be performed using a JOIN operation. S42, SQL Generation and Multidimensional Self-Validation: First, the constructed prompt words are input into the large language model to generate an initial version of the SQL statement. Then, the interception engine sequentially performs syntax and existence checks, value range checks, and security checks on the initial version of the SQL. The syntax and existence checks are responsible for checking the correctness of the SQL syntax and whether the table names and field names used actually exist in the database. The value range checks whether the field values ​​in the WHERE condition are within the range of enumerated values ​​specified in the schema. The security checks use a regular expression matching mechanism to intercept high-risk operation statements containing DROP, DELETE, UPDATE, and ALTER that may damage or tamper with the production database to ensure that the query only performs read-only operations. If any check fails, it is marked as an anomaly.

[0056] S43, Closed-loop repair and execution: If any of the above checks fail, the system will assemble the error message and the original generated error SQL according to the prompt template and send it back to the large model for retry and correction, with a maximum of 3 retries; only after all checks pass can the system connect to the target database to execute the SQL and obtain the structured query results.

[0057] Step S5, Heterogeneous Search Results Alignment and Fusion and Answer Generation, please refer to [link / reference]. Figure 5 For mixed query problems, directly concatenating documents and data tables can easily lead to entity mismatches or cross-source semantic association errors. This invention employs a multi-dimensional alignment mechanism, specifically including the following steps: S51, Heterogeneous Multidimensional Alignment: Alignment of unstructured search results and structured query results in terms of entity, time, and semantic dimensions, and establish data connection between the two sides through association mapping and format unification.

[0058] (i) Entity Alignment: On the unstructured side, a pre-built ternary mapping table of equipment asset codes, standard terms, and colloquial aliases is used to bidirectionally associate document equipment descriptions with database equipment codes; on the structured side, equipment codes are mapped to standardized equipment names. As an example, the PMS code "ID_1001A" in the structured ledger query results is mapped to the standardized name "10kV-3 main transformer", while the corresponding description of the equipment is located on the unstructured document side through association mapping.

[0059] (ii) Time alignment: A time-series standardization strategy is adopted to unify the format of discrete timestamps in structured data (e.g., 2023-05-12) and relative times in unstructured documents (e.g., maintenance period, most recent) and convert them into absolute time windows; for example, if the query base date is June 1, 2024, the most recent maintenance will be mapped to the specific timestamp corresponding to the most recent maintenance record.

[0060] (iii) Semantic alignment: The unstructured side retains the original text fragments to preserve the knowledge content of the operation and maintenance documents; the structured side uses a template engine to transform the queried structured data tables into a clear declarative natural language description.

[0061] S52, Generation and Source Tracing: The aligned content above is concatenated into a unified fusion generation context and input into the large language model. In the final answer generated by the large language model, the source of each assertion is marked by a predefined delimiter, as an example: [Source: Operating Procedure V2.0, page 15] or [Source: Ledger Database Equipment Table, corresponding query SQL].

[0062] S53, Quality Self-Assessment: Utilizing the LLM-as-Judge mechanism, the generated answer is scored across three dimensions: factual consistency, answer completeness, and source traceability. If the score is below a preset threshold, the query optimization process is triggered, and steps S3-S5 are re-executed.

[0063] Example 1: Knowledge Question and Answer Category Scenario: Maintenance personnel input the question "How to handle the failure of a 10kV vacuum circuit breaker to close?" Step 1: The intent recognition module determines that the question is a "knowledge question and answer" type and routes it to the hybrid semantic retrieval channel.

[0064] Step 2: The query preprocessing module uses a domain terminology dictionary to standardize the term "refusal to close" in the problem into the standard term "refusal to operate (close)" and expands the synonym "failure to close".

[0065] Step 3, Multi-path Recall Stage: Vector semantic retrieval returns the 5 document fragments closest to the semantics of "vacuum circuit breaker closing fault"; BM25 keyword retrieval returns 5 fragments containing the keywords "vacuum circuit breaker", "failure to close", and "failure to operate"; knowledge graph retrieval starts with "10kV vacuum circuit breaker" as the starting node, finds the "failure to operate" node along the "fault mode" relationship, and then obtains the relevant knowledge subgraph along the "disposal plan" relationship.

[0066] Step 4: The cross encoder reorders the approximately 12 deduplicated candidate segments from the three channels and selects the Top-5 segments as the generated context.

[0067] Step 5: The large language model generates answers based on the Top-5 context fragments, covering the handling steps for operating power supply checks, closing coil detection, and mechanism transmission chain checks, and annotating the specific documents and chapters from which each step originates.

[0068] Step 6: Finally, output the answer containing the source information.

[0069] Example 2: Data Query Class Scenario: Maintenance personnel input the question, "What are some records of transformer oil temperatures exceeding 80 degrees Celsius in 2025?" Step 1: The intent recognition module determines that the problem is a "data query" type and routes it to the Text2SQL query channel.

[0070] Step 2: The schema-aware prompt construction module retrieves the monitoring_data table related to "transformer", "oil temperature" and "record" in the schema semantic index. The relevant fields are device_type (device type), oil_temperature (oil temperature), record_time (record time) and device_name (device name). The module dynamically constructs prompts that include the structure of these tables and descriptions of the fields.

[0071] Step 3: Generate SQL statement using the large language model: SELECT device_name, oil_temperature, record_time FROM monitoring_data WHERE device_type='transformer' AND oil_temperature>80 AND record_time>='2025-01-01' AND record_time<'2026-01-01' ORDER BY oil_temperature DESC.

[0072] Step 4: The SQL self-verification module verifies that the syntax is correct, the fields exist, "transformer" is within the range of device_type enumeration values, and there are no dangerous operations. The verification passes.

[0073] Step 5: Execute the SQL query, obtain the result set, and the large language model will convert the result into a natural language response, along with the corresponding query SQL and data source annotations.

[0074] Step 6: Finally, output the answer containing the source information.

[0075] Example 3: Hybrid Query Class Scenario: Maintenance personnel input the question, "What was the cause of the most recent overheating failure of main transformer No. 3? What are the statistics on overheating failures of similar main transformers?" Step 1: The intent recognition module determines that the question is a "hybrid query". The routing and scheduling module first decouples the question into a knowledge retrieval sub-question and a data query sub-question, and sends them to the hybrid semantic retrieval channel and the Text2SQL query channel respectively.

[0076] Step 2: Use the hybrid semantic retrieval channel to retrieve maintenance documents and fault cases related to "cause of main transformer overheating fault"; use the Text2SQL query channel to generate SQL queries for statistics on the same type of main transformer overheating fault.

[0077] Step 3: The fusion generation module performs entity alignment (associating the device code of "No. 3 main transformer" with the device description in the document), time alignment (unifying the time format), and semantic alignment (converting the statistical results into natural language) on the two results to generate a fusion context.

[0078] Step 4: The large language model generates a comprehensive answer based on the fusion context. The first half is based on the fault case description and cause analysis, and the second half is based on statistical data to present information such as fault frequency and distribution. Each part is labeled with its own source.

[0079] Step 5: Finally, output the answer containing the source information.

[0080] In addition, the present invention can also provide a computer device, including a processor and a memory, wherein the memory is used to store a computer executable program, the processor reads the computer executable program from the memory and executes it, and the processor executes part or all of the computer executable program to realize the multimodal retrieval and intelligent question answering method for power distribution equipment operation and maintenance described in the present invention.

[0081] On the other hand, the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, enables a multimodal retrieval and intelligent question-answering method for the operation and maintenance of power distribution equipment as described in the present invention.

[0082] The computer equipment may be a vehicle-mounted computer, a laptop computer, a desktop computer, or a workstation.

[0083] The processor can be a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), or a field-programmable gate array (FPGA).

[0084] The memory described in this invention can be an internal storage unit of a laptop, desktop computer, or workstation, such as memory or hard disk; or it can be an external storage unit, such as a portable hard disk or flash memory card.

[0085] Computer-readable storage media can include computer storage media and communication media. Computer storage media includes volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer-readable storage media can include: read-only memory (ROM), random access memory (RAM), solid-state drives (SSDs), or optical discs, etc. Random access memory can include resistive random access memory (ReRAM) and dynamic random access memory (DRAM).

[0086] The above content is only for illustrating the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. Any modifications made to the technical solution based on the technical concept proposed in this invention shall fall within the scope of protection of the claims of this invention.

Claims

1. A multimodal retrieval and intelligent question-answering method for the operation and maintenance of power distribution equipment, characterized in that, Includes the following steps: Unstructured operation and maintenance documents are subjected to domain-adaptive block partitioning based on device topology constraints, vectorized storage, and domain knowledge graph construction. They are also connected to structured data sources and a schema semantic index is established. The system receives user natural language questions, performs intent recognition and classification, and distributes the questions to the corresponding retrieval channels based on the classification results. The retrieval channels include a hybrid semantic retrieval channel and a Text2SQL query channel. For the problem of distributing to the hybrid semantic retrieval channel, unstructured operation and maintenance knowledge is retrieved through multiple paths and merged and reordered to obtain unstructured retrieval results; For issues distributed to the Text2SQL query channel, dynamic schema pruning is used to construct hints, generate SQL statements, and perform self-verification and repair, and then the query is executed to obtain structured query results; For mixed query questions, the mixed semantic retrieval channel and the Text2SQL query channel are executed in parallel. The unstructured retrieval results and structured query results are heterogeneously aligned and fused to generate traceable answers, and the quality of the answers is self-evaluated.

2. The multimodal retrieval and intelligent question-answering method for power distribution equipment operation and maintenance according to claim 1, characterized in that, The domain-adaptive segmentation based on device topology constraints includes: introducing a power distribution equipment topology hierarchy tree as a prior constraint; for the fault case library, using an identifier-based regular expression matching and semantic boundary detection method to forcibly segment the text into complete fault phenomenon-cause analysis-handling measures knowledge blocks, and encoding and binding them with device nodes in the power distribution equipment topology hierarchy tree; for operation procedure documents, extracting the operation timing logic features in the text, and performing minimum-granularity segmentation according to uninterrupted operation steps.

3. The multimodal retrieval and intelligent question-answering method for power distribution equipment operation and maintenance according to claim 1, characterized in that, Receiving user natural language questions and performing intent recognition and classification includes: Using an intent classification model fine-tuned from a pre-trained language model, questions are categorized into knowledge-based question answering, data querying, casual conversation and refusal to answer, and mixed querying. Knowledge-based question answering questions are routed to the mixed semantic retrieval channel, data querying questions are routed to the Text2SQL query channel, and mixed querying questions are decoupled into knowledge-based question answering sub-questions and data querying sub-questions, which are then routed and processed separately. Casual conversation and refusal to answer questions are responded to with a refusal to answer.

4. The multimodal retrieval and intelligent question-answering method for power distribution equipment operation and maintenance according to claim 1, characterized in that, The multi-path recall of the hybrid semantic retrieval channel includes: vector semantic retrieval, BM25 keyword retrieval and knowledge graph retrieval. The three recall results are deduplicated and then reordered by a cross encoder to output the Top-N as the generated context. The vector semantic retrieval adopts a text embedding model fine-tuned by power domain corpus, and the cross encoder adopts a cross encoder fine-tuned by power distribution domain labeled data; the knowledge graph retrieval identifies the equipment entities in the query, performs multi-hop reasoning along the fault mode and component association edges in the knowledge graph, and recalls the associated graph path.

5. The multimodal retrieval and intelligent question-answering method for power distribution equipment operation and maintenance according to claim 1, characterized in that, The dynamic schema pruning includes two levels of pruning: The first level is semantic table recall, which calculates the similarity between the query vector and the table-level description vector of the registered data tables, and selects the top K tables with the highest similarity ranking to include in the candidate table set, where K is the number of registered data tables selected. The second level is field pruning based on topological constraints. Within the candidate table set, the device association ID field necessary for cross-table join queries is forcibly retained according to the foreign key lineage between database tables, while other redundant fields are removed, and prompt words are dynamically constructed. The SQL self-verification and repair includes: performing syntax and existence checks, value range checks, and security checks on the generated SQL statements in sequence. If the checks fail, the error information is fed back to the large language model for retry and correction, with a maximum of 3 retries. The security check uses a regular expression matching mechanism to intercept high-risk operation statements containing DROP, DELETE, UPDATE, and ALTER.

6. The multimodal retrieval and intelligent question-answering method for power distribution equipment operation and maintenance according to claim 1, characterized in that, The heterogeneous alignment and fusion of unstructured retrieval results and structured query results includes: entity alignment, time alignment, and semantic alignment. Entity alignment is based on a pre-built ternary mapping table of device asset codes, standard terms, and colloquial aliases, which bidirectionally associates document device descriptions with database device codes. Time alignment adopts a time-series standardization strategy to convert discrete timestamps in structured data and relative times in unstructured documents into absolute time windows. Semantic alignment includes preserving original text fragments on the unstructured side and converting two-dimensional data tables into natural language descriptions on the structured side using a template engine.

7. The multimodal retrieval and intelligent question-answering method for power distribution equipment operation and maintenance according to claim 1, characterized in that, The generation of traceable answers includes: in the aligned unified context, marking the source of each assertion with a special delimiter, the source including document page number or SQL query statement; the quality self-assessment includes using a large language model evaluation mechanism to score the generated answer from three dimensions: factual consistency, answer completeness and traceability completeness. If it is lower than a preset threshold, the query optimization process is triggered, and the retrieval, query and answer generation steps are re-executed.

8. A multimodal retrieval and intelligent question-answering system for the operation and maintenance of power distribution equipment, characterized in that, It includes a knowledge base construction module, an intent recognition and routing module, a hybrid semantic retrieval module, a Text2SQL query module, and a fusion generation module: The knowledge base construction module is used to collect power distribution equipment operation and maintenance documents, perform domain-adaptive block partitioning based on equipment topology constraints, vectorized storage, and domain knowledge graph construction. It also completes the registration of structured data sources and the construction of schema semantic indexes. The intent recognition and routing module is used to receive user natural language questions, classify the intents, and distribute the questions to the corresponding retrieval channels according to the classification results; the retrieval channels include a hybrid semantic retrieval channel and a Text2SQL query channel; The hybrid semantic retrieval module is used to perform multi-path recall and fusion re-ranking of unstructured operation and maintenance knowledge to obtain unstructured retrieval results; The Text2SQL query module is used to construct hints, generate SQL statements, and perform self-verification and repair for issues distributed to the Text2SQL query channel through dynamic schema pruning, and execute queries to obtain structured query results. The fusion generation module is used to heterogeneously align and fuse unstructured retrieval results and structured query results obtained when performing parallel execution of mixed query types of queries using the mixed semantic retrieval channel and the Text2SQL query channel, generating traceable answers, and performing quality self-evaluation on the answers.

9. A computer device comprising a processor and a memory, the memory being used to store a computer-executable program, characterized in that, The processor reads part or all of the computer-executable program from the memory and executes it to implement the multimodal retrieval and intelligent question-answering method for the operation and maintenance of power distribution equipment as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the multimodal retrieval and intelligent question-answering method for the operation and maintenance of power distribution equipment as described in any one of claims 1 to 7.