Question-and-answer processing methods, equipment, and storage media for power operation and maintenance scenarios

By employing a question-and-answer processing method that combines speech recognition and large language model reasoning, the problem of knowledge retrieval for power operation and maintenance personnel in offline environments has been solved. This enables highly efficient and secure operation and maintenance guidance that is conducted entirely offline, thereby improving the efficiency and security of power operation and maintenance.

CN123115362APending Publication Date: 2026-11-17CHINA SOUTHERN POWER GRID DIGITAL GRID GRP CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611622562.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-10-20
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

In power operation and maintenance scenarios, maintenance personnel cannot conduct offline knowledge research in outdoor environments without network connectivity, and it is difficult to touch screen to view documents when wearing insulated gloves, resulting in low operation and maintenance efficiency and poor security.

Method used

This paper provides a question-and-answer processing method for power operation and maintenance scenarios. It generates and broadcasts operation and maintenance answers by voice through speech recognition, bidirectional reference structured knowledge unit retrieval and pre-set large language model reasoning, realizing fully offline interaction and supporting voice-based touchless operation.

Benefits of technology

It enables efficient and accurate operation and maintenance knowledge retrieval in environments without external network access, improves the security and efficiency of operation and maintenance tasks, and adapts to the safety management requirements of power field sites.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123115362A_ABST
    Figure CN123115362A_ABST
Patent Text Reader

Abstract

This application relates to a question-answering processing method, device, and storage medium for power operation and maintenance scenarios, enabling fully offline voice interaction and improving retrieval accuracy and recall. The method includes: in response to a target object's triggered operation for a target operation and maintenance task, acquiring the audio of the question corresponding to the target object; performing speech recognition processing on the question audio to obtain the corresponding audio text; retrieving the audio text based on key operation and maintenance knowledge fragments in multiple structured knowledge units and a first bidirectional reference link between multiple structured knowledge units to obtain the target operation and maintenance knowledge fragment; inferring from the target operation and maintenance knowledge fragment and the audio text based on a preset large language model to generate the target operation and maintenance answer; and performing speech synthesis processing on the target operation and maintenance answer to obtain the target answer audio, which is used to guide the target object in performing the target operation and maintenance task.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a question-and-answer processing method, device and storage medium for power operation and maintenance scenarios. Background Technology

[0002] With the rapid development of technologies such as artificial intelligence, big data, and the Internet of Things, the power industry is gradually transforming from traditional manual operation and maintenance (O&M) models to intelligent and digital transformation. As a core node of the power grid, substations rely heavily on a large amount of structured and semi-structured professional knowledge, such as operating procedures, equipment ledgers, defect records, and emergency plans. O&M personnel need to quickly and accurately consult relevant procedures and clauses when performing SF6 leak handling, equipment inspections, and anomaly response.

[0003] Traditional power operation and maintenance (O&M) solutions rely on cloud-deployed power voice question-and-answer systems or general retrieval augmented generation (RAG) architectures to retrieve O&M operation content. However, cloud deployments depend on network connectivity, and outdoor field scenarios such as power substation O&M typically lack external network coverage or are limited to the secure isolation environment of the power intranet. This makes it difficult to reliably access the large models deployed in the cloud to access O&M-related knowledge, preventing O&M personnel from completing power O&M tasks offline. Secondly, O&M personnel need to touch the terminal interface to trigger voice questions and answers, but wearing insulated gloves and performing real-time O&M operations in bright outdoor or nighttime environments makes it difficult for them to touch the screen to access relevant documents, resulting in low knowledge retrieval efficiency and poor O&M efficiency. Summary of the Invention

[0004] Therefore, it is necessary to provide a question-and-answer processing method, device, and storage medium for power operation and maintenance scenarios to address the above-mentioned technical problems. This method can not only achieve fully offline voice interaction, but also improve retrieval accuracy and recall.

[0005] Firstly, this application provides a question-and-answer processing method for power operation and maintenance scenarios, applied to power operation equipment, the method including:

[0006] In response to the target object's triggered operation on the target operation and maintenance task, obtain the corresponding problem audio of the target object;

[0007] The problematic audio is processed by speech recognition to obtain the corresponding audio text.

[0008] Based on the key operation and maintenance knowledge fragments in each of the multiple structured knowledge units and the first bidirectional reference links between the multiple structured knowledge units, the audio text is retrieved and processed to obtain the target operation and maintenance knowledge fragment. The two ends of the first bidirectional reference link are anchored to two related structured knowledge units respectively. The structured knowledge unit is an independent knowledge carrier that is pre-compiled from multiple power operation and maintenance guidance texts and is used to carry the key operation and maintenance knowledge fragments.

[0009] Based on a pre-defined large language model, reasoning is performed on the target operation and maintenance knowledge fragments and audio text to generate the target operation and maintenance answer;

[0010] The target operation and maintenance answer is processed by speech synthesis to obtain the target answer audio, which is used to guide the target object to perform the target operation and maintenance task.

[0011] In one embodiment, based on key operation and maintenance knowledge fragments within multiple structured knowledge units and a first bidirectional reference link between the multiple structured knowledge units, the audio text is retrieved to obtain the target operation and maintenance knowledge fragment, including:

[0012] The audio text and the aggregated knowledge units are matched to obtain the matching results. The aggregated knowledge units are encapsulated based on the key operation and maintenance knowledge fragments, metadata and corresponding first bidirectional reference links of multiple first knowledge units in the candidate knowledge unit cluster. The candidate knowledge unit cluster is aggregated when the co-occurrence frequency of multiple first knowledge units in multiple structured knowledge units in question answering is greater than the first threshold, and / or the number of links of the first bidirectional reference links between multiple first knowledge units is greater than the second threshold.

[0013] If the matching result indicates that the audio text matches the aggregated knowledge unit, then the audio text is retrieved and processed based on the aggregated knowledge unit to obtain the target operation and maintenance knowledge fragment;

[0014] or,

[0015] If the matching result indicates that the audio text does not match the aggregated knowledge unit, then the audio text is retrieved based on the key operation and maintenance knowledge fragments in each of the multiple structured knowledge units and the first bidirectional reference links between the multiple structured knowledge units to obtain the target operation and maintenance knowledge fragment.

[0016] In one embodiment, the aggregated knowledge unit includes a second bidirectional reference link between the aggregated knowledge unit and each first knowledge unit;

[0017] If the matching result indicates that the audio text matches the aggregated knowledge unit, then the audio text is retrieved and processed based on the aggregated knowledge unit to obtain the target operation and maintenance knowledge fragment, including:

[0018] If the matching result indicates that the audio text matches the aggregated knowledge unit, then the first knowledge unit corresponding to each second bidirectional reference link is determined based on each second bidirectional reference link in the aggregated knowledge unit;

[0019] Based on key operation and maintenance knowledge fragments from multiple first knowledge units, a target operation and maintenance knowledge fragment is generated.

[0020] In one embodiment, each structured knowledge unit corresponds to a first entity; based on the key operation and maintenance knowledge fragments in each of the multiple structured knowledge units and the first bidirectional reference links between the multiple structured knowledge units, the audio text is retrieved to obtain the target operation and maintenance knowledge fragment, including:

[0021] Entity recognition is performed on the audio text to obtain a second entity;

[0022] Based on the second entity, audio text, and the first entity corresponding to each structured knowledge unit, multiple structured knowledge units are filtered to obtain the target structured knowledge unit.

[0023] Based on the first bidirectional reference link corresponding to the target structured knowledge unit, identify the associated structured knowledge units related to the target structured knowledge unit;

[0024] Based on the key operation and maintenance knowledge fragments in the target structured knowledge unit and the key operation and maintenance knowledge fragments in the associated structured knowledge unit, the target operation and maintenance knowledge fragment is generated.

[0025] In one embodiment, each structured knowledge unit further includes metadata corresponding to a first entity, the metadata being used to characterize the power operation and maintenance attributes of the corresponding first entity; based on the second entity, the audio text, and the first entity corresponding to each structured knowledge unit, multiple structured knowledge units are filtered to obtain target structured knowledge units, including:

[0026] The second entity is matched with the first entity corresponding to each of the multiple structured knowledge units, and the matched first entity is taken as the candidate entity.

[0027] Extract operation and maintenance keywords from the audio text. These keywords are used to describe the power operation and maintenance status of the second entity.

[0028] If the operation and maintenance keywords match the attribute fields in the target metadata, then the structured knowledge unit corresponding to the candidate entity is determined as the target structured knowledge unit, and the target metadata is the metadata in the structured knowledge unit corresponding to the candidate entity.

[0029] In one embodiment, the method further includes:

[0030] If the operation and maintenance keywords do not match the attribute fields in the target metadata, then extract the entity name of each candidate entity, the key operation and maintenance knowledge fragments in the structured knowledge unit corresponding to the candidate entity, and the attribute fields in the target metadata.

[0031] The entity matching score for each candidate entity is obtained by matching operation and maintenance keywords with the entity names of each candidate entity.

[0032] The matching of operation and maintenance keywords with key operation and maintenance knowledge fragments in the structured knowledge units corresponding to each candidate entity is performed to obtain the fragment matching score for each candidate entity.

[0033] Based on the matching of operation and maintenance keywords with attribute fields in the target metadata, the tag matching score corresponding to each candidate entity is obtained;

[0034] Based on the entity matching score, fragment matching score and tag matching score corresponding to each candidate entity, the target matching score of the structured knowledge unit corresponding to each candidate entity is calculated.

[0035] Based on the target matching score and preset threshold of the structured knowledge units corresponding to each candidate entity, the structured knowledge units corresponding to multiple candidate entities are filtered to obtain the target structured knowledge units.

[0036] In one embodiment, based on the target matching score and preset threshold of the structured knowledge units corresponding to each candidate entity, the structured knowledge units corresponding to multiple candidate entities are filtered to obtain the target structured knowledge units, including:

[0037] The target matching score of the structured knowledge unit corresponding to each candidate entity is compared with a preset threshold. From the structured knowledge units corresponding to multiple candidate entities, the structured knowledge units corresponding to the target matching score are selected that are greater than or equal to the preset threshold.

[0038] From the structured knowledge units whose target matching scores are greater than or equal to a preset threshold, select a preset number of structured knowledge units as target structured knowledge units.

[0039] In one embodiment, based on the target matching score and preset threshold of the structured knowledge units corresponding to each candidate entity, the structured knowledge units corresponding to multiple candidate entities are filtered to obtain the target structured knowledge units, including:

[0040] If the target matching score of the structured knowledge unit corresponding to each candidate entity is less than the preset threshold, then extract the text feature vector of the audio text and the text feature vector of the key operation and maintenance knowledge fragment in the structured knowledge unit corresponding to each candidate entity.

[0041] Calculate the similarity distance between the text feature vector of the audio text and the text feature vector of the key operation and maintenance knowledge segment in the structured knowledge unit corresponding to each candidate entity, and obtain the text similarity between the audio text and the key operation and maintenance knowledge segment in the structured knowledge unit corresponding to each candidate entity.

[0042] Based on text similarity, the structured knowledge units corresponding to multiple candidate entities are filtered to obtain the target structured knowledge unit.

[0043] In one embodiment, before retrieving the audio text based on key operational knowledge fragments within multiple structured knowledge units and a first bidirectional reference link between the multiple structured knowledge units to obtain the target operational knowledge fragment, the method further includes:

[0044] Entity recognition is performed on multiple power operation and maintenance guidance texts to obtain multiple semantically non-overlapping first entities and entity association relationships;

[0045] Based on each first entity, the content of multiple power operation and maintenance guidance texts is extracted and split to obtain the key operation and maintenance knowledge fragments corresponding to each first entity.

[0046] Based on each first entity and the key operation and maintenance knowledge fragments corresponding to the first entity, a structured knowledge unit corresponding to each first entity is generated. The structured knowledge unit includes the key operation and maintenance knowledge fragments corresponding to the first entity and the metadata corresponding to the first entity.

[0047] Based on entity relationships, a first bidirectional reference link is constructed for the structured knowledge units corresponding to multiple first entities, thereby obtaining the first bidirectional reference link between multiple structured knowledge units.

[0048] Secondly, this application also provides an electrical work equipment, comprising:

[0049] The response and acquisition unit is used to respond to the target object's triggered operation on the target operation and maintenance task and acquire the corresponding problem audio of the target object;

[0050] The speech recognition unit is used to perform speech recognition processing on the problematic audio to obtain the corresponding audio text.

[0051] The retrieval processing unit is used to retrieve and process audio text based on key operation and maintenance knowledge fragments in multiple structured knowledge units and the first bidirectional reference links between multiple structured knowledge units to obtain target operation and maintenance knowledge fragments. The two ends of the first bidirectional reference links are anchored to two related structured knowledge units respectively. The structured knowledge unit is an independent knowledge carrier that is pre-compiled and generated from multiple power operation and maintenance guidance texts to carry key operation and maintenance knowledge fragments.

[0052] The reasoning unit is used to reason about the target operation and maintenance knowledge fragments and audio text based on a preset large language model, and generate the target operation and maintenance answer;

[0053] The speech synthesis unit is used to process the target operation and maintenance answer into speech, and the target answer audio is used to guide the target object to perform the target operation and maintenance task.

[0054] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method described in the first aspect.

[0055] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the first aspect.

[0056] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in the first aspect.

[0057] The aforementioned question-and-answer processing method, equipment, and storage medium for power operation and maintenance scenarios respond to the target object's triggered operation for the target operation and maintenance task through power operation equipment to obtain the corresponding question audio. Then, speech recognition processing is performed on the question audio to obtain the corresponding audio text. Further, based on key operation and maintenance knowledge fragments within multiple structured knowledge units and a first bidirectional reference link between these units, the audio text is retrieved to obtain the target operation and maintenance knowledge fragment. Each end of the first bidirectional reference link is anchored to two related structured knowledge units, which are independent knowledge carriers pre-compiled from multiple power operation and maintenance guidance texts to carry key operation and maintenance knowledge fragments. Then, reasoning is performed on the target operation and maintenance knowledge fragment and the audio text based on a pre-set large language model to generate the target operation and maintenance answer. Finally, speech synthesis processing is performed on the target operation and maintenance answer to obtain the target answer audio, which is used to guide the target object in performing the target operation and maintenance task. Through the above methods, the entire process of speech recognition, retrieval of bidirectional reference structured knowledge units, reasoning of pre-set large language models, and speech synthesis is deployed offline on a local edge computing terminal, i.e., power operation equipment. Accurate retrieval is achieved by relying on pre-compiled, semantically independent structured knowledge units and the first bidirectional reference links anchored between these units. The entire process requires no external network or cloud computing power support, solving the problems of traditional cloud-based power voice question-and-answer systems being unusable in environments without external network isolation and the insufficient computing power of general RAG architectures relying on high-overhead vector retrieval. This results in high query efficiency and accuracy. Furthermore, it supports touchless interaction via voice by maintenance personnel and other target objects while wearing insulated gloves, addressing the pain point of low efficiency in viewing documents on touchscreens during outdoor high-brightness / nighttime operations, and fully adapting to the safety management requirements of power sites. Then, by directly outputting the target answer audio via voice broadcast, the target object is guided to perform the target maintenance task, significantly improving the safety and efficiency of maintenance tasks. Attached Figure Description

[0058] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0059] Figure 1 This is an application environment diagram of a question-and-answer processing method for a power operation and maintenance scenario in one embodiment;

[0060] Figure 2 This is a flowchart illustrating a question-and-answer processing method for a power operation and maintenance scenario in one embodiment.

[0061] Figure 3 This is another flowchart illustrating a question-and-answer processing method for a power operation and maintenance scenario in one embodiment;

[0062] Figure 4 This is another flowchart illustrating a question-and-answer processing method for a power operation and maintenance scenario in one embodiment;

[0063] Figure 5 This is another flowchart illustrating a question-and-answer processing method for a power operation and maintenance scenario in one embodiment;

[0064] Figure 6 This is a structural block diagram of the power work equipment in one embodiment;

[0065] Figure 7 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0066] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0067] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0068] The question-and-answer processing method for power operation and maintenance scenarios provided in this application embodiment can be applied to, for example... Figure 1 The application environment shown. For example... Figure 1As shown, the power operation equipment includes a triggering and acquisition module 101, a power terminology dictionary and speech recognition enhancement module 102, a multi-level semantic retrieval module 103, an edge-side large model answer generation module 104, and a speech synthesis and broadcasting module 105. The triggering and acquisition module 101 can respond to the target object's triggering operation for the target operation and maintenance task to obtain the corresponding question audio. The power terminology dictionary and speech recognition enhancement module 102 can be used to perform speech recognition processing on the question audio to obtain the corresponding audio text. Optionally, the power terminology dictionary and speech recognition enhancement module 102 can also construct a power industry-specific terminology mapping table, and inject this table as a hot word bias list during the speech recognition processing of the question audio, thereby generating more accurate audio text including the terminology. Furthermore, the multi-level semantic retrieval module 103 can be used to retrieve the audio text based on key operation and maintenance knowledge fragments within multiple structured knowledge units and the first bidirectional reference links between multiple structured knowledge units to obtain the target operation and maintenance knowledge fragments. The edge-side large model answer generation module 104 can be used to call a preset large language model to reason about the target operation and maintenance knowledge fragments and audio text, and generate the target operation and maintenance answer. The speech synthesis and broadcasting module 105 can be used to perform speech synthesis processing on the target operation and maintenance answer to obtain the target answer audio. Then, the target answer audio is played back so that it can guide the target object to perform the target operation and maintenance task.

[0069] Optionally, the power operation equipment also includes a knowledge graph construction module 106 and an aggregation point management module 107. The knowledge graph construction module 106 is used to construct a bidirectional reference knowledge graph between structured knowledge units, which includes multiple structured knowledge units and first bidirectional reference links between these units. The aggregation point management module 107 is used to select structured knowledge units that meet preset conditions from these multiple structured knowledge units for aggregation, facilitating direct retrieval from the aggregated knowledge unit when it is matched, thereby improving retrieval efficiency.

[0070] Optionally, the power operation equipment also includes an incremental knowledge base management module 108. The incremental knowledge base management module 108 can be used to incrementally update and dynamically maintain the structured knowledge units within the knowledge graph after constructing a bidirectional reference knowledge graph between structured knowledge units, thereby ensuring the timeliness and accuracy of the knowledge base.

[0071] It should be noted that the power operation equipment mentioned in this application may be an offline edge computing terminal that integrates a central processing unit (CPU) and a neural processing unit (NPU), such as a power operation recorder or an industrial-grade handheld terminal, or other devices with local computing power and supporting offline operation, etc., which are not limited in the embodiments of this application.

[0072] In one exemplary embodiment, such as Figure 2 As shown, a question-answering processing method for power operation and maintenance scenarios is provided, which is applied to... Figure 1 Taking electrical work equipment as an example, the explanation includes the following steps 201 to 205. Wherein:

[0073] 201. In response to the target object's triggered operation on the target operation and maintenance task, obtain the corresponding problem audio of the target object.

[0074] For example, when target objects such as maintenance personnel are performing target maintenance tasks outdoors, they often cannot connect to the cloud server in real time to query due to the lack of network support. Therefore, in order to provide timely and accurate maintenance guidance in an offline environment, the target object can trigger an operation to activate the local Q&A function after starting the power work equipment, thereby enabling the power work equipment to respond to the target object's triggered operation for the target maintenance task.

[0075] Specifically, the target user can wake up the power work equipment using a voice wake-up word to clearly indicate that they need to initiate a voice Q&A request for the target maintenance task. For example, when the target user says a wake-up word (such as "watt watt"), the power work equipment matches the spoken wake-up word with a built-in preset wake-up word. If the match is successful, the power work equipment responds to the trigger operation and enters the voice Q&A mode. Alternatively, the target user can also wake up the power work equipment using gestures. Specifically, the target user can make a preset gesture (such as double-tap, swipe, or long press) on the screen or a specific area of ​​the power work equipment. If the power work equipment detects that the gesture matches the preset gesture, it responds to the trigger operation and enters the voice Q&A mode.

[0076] After entering the voice Q&A mode, the target can continue to ask specific questions about the target maintenance task. The power operation equipment collects the voice signal through the built-in trigger and acquisition module, thereby obtaining the audio of the target's question.

[0077] It should be understood that the question audio refers to the original voice data spoken by the target object in voice question-and-answer mode, which is related to the question content of the target operation and maintenance task.

[0078] For example, the audio of the problem could be a voice message addressing a specific maintenance issue, such as "How to handle excessive transformer oil temperature" or "Precautions for operating disconnect switches." No specific limitations are set in this application.

[0079] 202. Perform speech recognition processing on the problematic audio to obtain the corresponding audio text.

[0080] For example, after obtaining the audio of the problem, the power operation equipment uses a built-in speech recognition model to decode the audio, thereby obtaining the corresponding audio text. It should be understood that the speech recognition model can be an end-to-end speech recognition model fine-tuned from a large corpus of speech in the power operation and maintenance field, capable of converting the acoustic features in the collected audio of the problem into a corresponding text sequence. In this way, the power operation equipment can convert unstructured speech signals into structured audio text for subsequent semantic understanding and knowledge retrieval. For example, when the target subject says "How to handle transformer oil temperature being too high," the power operation equipment accurately converts the speech segment into the text sequence "How to handle transformer oil temperature being too high" using the speech recognition model, thus obtaining audio text consistent with the original speech content.

[0081] In some embodiments, the power operation equipment can also pre-build a power industry-specific terminology mapping table. This mapping table stores the mapping relationships between non-standard terms such as abbreviations, colloquialisms, or easily confused words and standard terms. For example, the abbreviation "GIS" → standard term "gas-insulated switchgear," the colloquial term "knife switch" → standard term "disconnect switch," "CT" → standard term "current transformer," "SF6" → standard term "sulfur hexafluoride," "main transformer" → standard term "110KV GIS gas chamber," etc., which are not limited in this application. Based on this, during the speech recognition processing of the question audio using a speech recognition model, the power operation equipment can also incorporate the power industry-specific terminology mapping table as hot words into the decoding process of the speech recognition model. This allows the speech recognition model to prioritize replacing non-standard words in the question audio with standard terms from the power industry-specific terminology mapping table during decoding, and then generate the final audio text after the replacement.

[0082] By introducing a hot word mechanism, the accuracy of speech recognition models in recognizing electrical engineering terms can be effectively improved, reducing recognition errors caused by homophones or non-standard words, ensuring that the generated audio text accurately reflects the true intent of the target audience, and providing a high-quality data foundation for subsequent semantic understanding and knowledge retrieval.

[0083] 203. Based on the key operation and maintenance knowledge fragments in each of the multiple structured knowledge units and the first bidirectional reference links between the multiple structured knowledge units, the audio text is retrieved and processed to obtain the target operation and maintenance knowledge fragment. The two ends of the first bidirectional reference link are anchored to two related structured knowledge units respectively. The structured knowledge unit is an independent knowledge carrier that is pre-compiled from multiple power operation and maintenance guidance texts and is used to carry the key operation and maintenance knowledge fragments.

[0084] For example, multiple power operation and maintenance (O&M) guidance texts can be pre-acquired and pre-compiled to obtain multiple independent knowledge carriers, i.e., structured knowledge units, that can be used to carry key O&M knowledge fragments. Each structured knowledge unit includes key O&M knowledge fragments corresponding to each independent first entity and metadata corresponding to that first entity. It should be understood that the key O&M knowledge fragments corresponding to each first entity can be understood as O&M knowledge content corresponding to the first entity extracted from these multiple power O&M guidance texts and processed through structure. Compared to the scattered unstructured text in each power O&M guidance text, the key O&M knowledge fragments corresponding to the first entity can aggregate relevant O&M knowledge across documents to form an independent knowledge system, facilitating subsequent rapid retrieval and accurate matching. Furthermore, the first entity can be understood as an independent O&M object extracted from the power O&M guidance text as a carrier for O&M knowledge aggregation, such as specific equipment like transformers, circuit breakers, and disconnect switches, or fault phenomena like winding overheating and SF6 gas leakage, etc., which are not specifically limited in this application.

[0085] In addition, a first bidirectional reference link is pre-constructed between each structured knowledge unit. Each end of the first bidirectional reference link anchors two related structured knowledge units, representing their semantic or logical association. For example, taking bidirectional reference link P as the first bidirectional reference link, if one structured knowledge unit relates to "transformer fault diagnosis" and another relates to "transformer oil temperature anomaly handling," then these two structured knowledge units can be linked through bidirectional reference link P. More specifically, through this bidirectional reference link P, the structured knowledge unit "transformer fault diagnosis" can point to the structured knowledge unit "transformer oil temperature anomaly handling," and vice versa. That is, one can find the related structured knowledge unit "transformer oil temperature anomaly handling" from the structured knowledge unit "transformer fault diagnosis" through bidirectional reference link P, and vice versa.

[0086] Thus, when performing retrieval based on audio text, the audio text is retrieved and processed according to the key operation and maintenance knowledge fragments in each of the multiple structured knowledge units and the first bidirectional reference links between the multiple structured knowledge units to obtain the target operation and maintenance knowledge fragment.

[0087] 204. Based on a pre-defined large language model, reason about the target operation and maintenance knowledge fragments and audio text to generate the target operation and maintenance answer.

[0088] For example, after retrieving the target maintenance knowledge fragment, the query intent corresponding to the target maintenance knowledge fragment and the audio text is input into a preset large language model. The preset large language model utilizes its powerful semantic understanding and logical reasoning capabilities, combined with the structured information in the target maintenance knowledge fragment (such as equipment parameters, fault phenomena, and handling steps) and the key contextual information extracted from the audio text, to perform deep reasoning and analysis, thereby generating accurate, targeted, and realistic target maintenance answers.

[0089] In some embodiments, target operation and maintenance knowledge fragments and audio text can be embedded into a preset prompt word template to construct target prompt information that includes contextual constraints, knowledge reference requirements, and output format specifications. Subsequently, the target prompt information is used as input to a preset large language model, which then processes the target prompt information to generate the target operation and maintenance answer.

[0090] 205. Perform speech synthesis processing on the target operation and maintenance answer to obtain the target answer audio, which is used to guide the target object to perform the target operation and maintenance task.

[0091] For example, after generating the target maintenance answer, a speech synthesis model is used to process the target maintenance answer into speech, thereby converting the text-based target maintenance answer into a natural and fluent target answer audio. The target answer audio can be played through the speech synthesis broadcast module of the power work equipment. In this way, during the target object's performance of the target maintenance task, it can receive and listen to the target answer audio in real time, thus eliminating the need for manual consultation of paper documents and touch operation of power work equipment, achieving hands-free intelligent maintenance assistance, and effectively improving work efficiency and safety.

[0092] In this embodiment, the entire process—from speech recognition and bidirectional reference to structured knowledge units, to reasoning using a pre-defined large language model, and speech synthesis—is deployed offline on a local edge computing terminal, i.e., a power operation device. Accurate retrieval is achieved by relying on pre-compiled, semantically independent structured knowledge units and bidirectionally anchored first bidirectional reference links between these units. The entire process requires no external network or cloud computing power support, solving the problems of traditional cloud-based power voice question-and-answer systems being unusable in environments without external network isolation and the insufficient computing power of general RAG architectures relying on high-overhead vector retrieval. This results in high query efficiency and accuracy. Furthermore, it supports touchless interaction via voice by maintenance personnel while wearing insulated gloves, addressing the low efficiency of touching screen document viewing during outdoor high-brightness / nighttime operations, and fully adapting to the safety management requirements of power sites. Finally, the system directly outputs the target answer audio via voice broadcast to guide the target personnel in performing the target maintenance task, significantly improving the safety and efficiency of the maintenance task.

[0093] In other embodiments, before performing step 203, a bidirectional reference knowledge graph can be pre-constructed. This bidirectional reference knowledge graph is a graph data structure built with structured knowledge units as nodes and first bidirectional reference links as edges. Specifically, before performing step 203—that is, before retrieving and processing the audio text based on the key operational knowledge fragments within multiple structured knowledge units and the first bidirectional reference links between multiple structured knowledge units to obtain the target operational knowledge fragment—the method further includes:

[0094] Step S1: Perform entity recognition on multiple power operation and maintenance guidance texts to obtain multiple semantically non-overlapping first entities and entity association relationships.

[0095] It should be understood that multiple power operation and maintenance guidance texts can be interpreted as textual data related to power operations, such as transformer operation procedures, equipment ledgers, typical fault cases, emergency plans, and operation guidelines. These texts typically contain unstructured information such as equipment parameters, operating procedures, safety regulations, and fault handling methods, serving as the foundational data source for constructing subsequent knowledge graphs and retrieval systems. Based on this, after retrieving these multiple power operation and maintenance guidance texts from the database, entity recognition processing can be performed on these texts to extract multiple initial entities. Then, semantic features are extracted from the initial entities in each guidance text, and semantic similarity between the initial entities is calculated using metrics such as cosine similarity or Euclidean distance. Initial entities with semantic similarity higher than a preset threshold are merged into the same first entity, thus obtaining multiple semantically non-overlapping first entities and entity relationships.

[0096] For example, consider multiple power operation and maintenance guidance documents, including Guidance Document 1, Guidance Document 2, Guidance Document 3, and Guidance Document 4. Guidance Document 1 could be the "Operating Procedures for SF6 Equipment in 110kV Substations," Guidance Document 2 could be the "Instructions for SF6 Leakage Repair in GIS Gas Chambers," Guidance Document 3 could be the "Emergency Handling Manual for SF6 Leakage Faults," and Guidance Document 4 could be the "Power Safety Work Procedures (SF6 Specific Section)." Guidance Document 1 mainly includes content such as "The rated working pressure of the 110kV GIS gas chamber is 0.6MPa, and the alarm pressure threshold is 0.52MPa; the annual humidity of SF6 gas must not exceed 300μL / L; daily inspections must record the pressure gauge readings and ambient temperature correction values ​​for each gas chamber." Guidance Document 2 mainly includes content such as "Before opening the SF6 gas chamber for repair, residual gas must be recovered, and entry is only permitted after the oxygen content is tested to be ≥18%; after repair, an airtightness test must be conducted, with an annual leakage rate ≤0.5%; after replenishing gas, it must be left to stand for 30 minutes before conducting a micro-water test." Guidance text 3 mainly includes the following: "After triggering a low SF6 concentration alarm on-site, the explosion-proof fan must be started immediately for 15 minutes of ventilation; personnel must wear positive pressure breathing apparatus before entering the leak area and are prohibited from acting alone; the isolation switch of adjacent gas chambers must not be operated before the leak point is isolated." Guidance text 4 mainly includes the following: "SF6 leak handling operations must be carried out with a Level 1 work permit, and safety warning fences must be set up on-site; personnel entering the leak handling site must wear gas masks, and qualified safety belts must be used for high-altitude operations (≥2m); the alert can only be lifted after the on-site SF6 concentration is tested to be below 1000μL / L after the handling is completed."

[0097] Thus, the initial entities identified in Guidance Text 1 include the GIS gas chamber rated pressure of 0.6 MPa, the 0.52 MPa alarm threshold, the SF6 humidity threshold of 300 μL / L, the gas chamber pressure gauge reading, and the ambient temperature correction value. The initial entities in Guidance Text 2 include SF6 residual gas recovery, the 18% oxygen content access standard, the airtightness test, the 0.5% annual leakage rate threshold, and the 30-minute settling requirement. The initial entities in Guidance Text 3 include SF6 low concentration alarm, 15-minute ventilation with explosion-proof fans, positive pressure breathing apparatus, two-person operation in the leak area, and the isolation switch between adjacent gas chambers. The initial entities in Guidance Text 4 include SF6 Level 1 work permit, safety warning fence, gas mask, 2-meter high-altitude operation, and the on-site SF6 concentration threshold of 1000 μL / L. Furthermore, after extracting semantic features from the initial entities in these four guidance texts, multiple first entities can be obtained by clustering and merging based on the semantic features of each initial entity. For example, multiple first entities may include a 110kV GIS gas chamber, SF6 leak response operation, SF6 gas detection, SF6 work safety measures, etc. Furthermore, the entity relationships between these first entities can also be obtained. For example, entity relationships may include: "110kV GIS gas chamber ↔ SF6 gas detection," "110kV GIS gas chamber / SF6 gas detection ↔ SF6 leak response operation," "110kV GIS gas chamber / SF6 leak response operation / SF6 gas detection ↔ SF6 work safety measures," etc.

[0098] Through the above clustering and merging process, fragmented information scattered in different guidance texts can be effectively integrated to form a unified and standardized entity system, laying the foundation for the subsequent construction of structured knowledge units.

[0099] Step S2: Based on each first entity, extract and split the content of multiple power operation and maintenance guidance texts to obtain the key operation and maintenance knowledge fragments corresponding to each first entity.

[0100] It should be understood that after obtaining the first entity, it is necessary to extract key operation and maintenance knowledge fragments related to each first entity from these multiple power operation and maintenance guidance texts. Specifically, multiple power operation and maintenance guidance texts can be traversed to identify and extract sentences or paragraphs containing each first entity as the key operation and maintenance knowledge fragments corresponding to the first entity.

[0101] For example, taking the first entity, the "110kV GIS gas chamber," as an example, the following content can be extracted from the guidance text 1, "Operation Procedures for SF6 Equipment in 110kV Substations": "The rated working pressure of the 110kV GIS gas chamber is 0.6MPa, and the alarm pressure threshold is 0.52MPa; the annual humidity of SF6 gas must not exceed 300μL / L; daily inspections must record the pressure gauge readings and ambient temperature correction values ​​of each gas chamber"; and from the guidance text 2, "Instructions for SF6 Leakage Repair and Maintenance of GIS Gas Chambers," the following content can be extracted: "Before opening the SF6 gas chamber for maintenance, residual gas must be recovered, and the oxygen content must be tested to be ≥18% before entering; after maintenance, an airtightness test must be conducted, with an annual leakage rate ≤0.5%; after replenishing gas, it must be left to stand for 30 minutes before conducting a micro-water test." However, the guidance text 3, "Emergency Handling Manual for SF6 Leakage Faults," and the guidance text 4, "Power Safety Work Procedures (SF6 Special Part)," do not include the relevant content on the routine operation and maintenance verification of the first entity (110kV GIS gas chamber). Therefore, the key operation and maintenance knowledge segments corresponding to the first entity "110kV GIS gas chamber" can be integrated as follows: "Rated working pressure 0.6MPa, alarm threshold 0.52MPa; annual SF6 humidity detection ≤300μL / L; daily inspection records pressure gauge readings and ambient temperature correction values; recover residual gas before opening the cover for maintenance, and oxygen content ≥18% is allowed to enter; after maintenance, the annual leakage rate of the airtightness test is ≤0.5%, and after replenishing gas, let it stand for 30 minutes before measuring trace moisture."

[0102] Similarly, for the remaining first entities, key operation and maintenance knowledge fragments were also obtained by extracting and deduplicating them across the guidance text.

[0103] Through the above content extraction and decomposition, it can be ensured that each primary entity is associated with specific and complementary operation and maintenance details from different source texts. These key operation and maintenance knowledge fragments lay a solid data foundation for the subsequent construction of a structured and searchable power operation and maintenance knowledge base.

[0104] Step S3: Based on each first entity and the key operation and maintenance knowledge fragments corresponding to the first entity, generate a structured knowledge unit corresponding to each first entity. The structured knowledge unit includes the key operation and maintenance knowledge fragments corresponding to the first entity and the metadata corresponding to the first entity.

[0105] Specifically, after identifying each first entity and its corresponding key operation and maintenance knowledge fragments, metadata for each first entity can be extracted. This metadata characterizes the power operation and maintenance attributes of the corresponding first entity, such as equipment type, voltage level, and the substation to which it belongs.

[0106] Based on this, the first entity is combined with its corresponding key operation and maintenance knowledge fragments and metadata to generate a structured knowledge unit corresponding to the first entity. For example, for the first entity "110kV GIS gas chamber", the generated structured knowledge unit can be represented as: {Entity name: "110kV GIS gas chamber", Key operation and maintenance knowledge fragments: "Rated working pressure 0.6MPa, alarm threshold 0.52MPa; annual SF6 humidity detection ≤300μL / L; daily inspection records pressure gauge readings and ambient temperature correction values; residual gas is recovered before opening the cover for maintenance, and entry is only allowed if the oxygen content is ≥18%; annual leakage rate of air tightness test after maintenance ≤0.5%, and micro-water is measured again after standing for 30 minutes after gas replenishment", Metadata: {Equipment type: "Switchgear", Voltage level: "110kV", Substation: "XX Substation"}}.

[0107] Step S4: Based on entity association, construct the first bidirectional reference link for the structured knowledge units corresponding to multiple first entities, and obtain the first bidirectional reference link between multiple structured knowledge units.

[0108] For example, the reference points between different structured knowledge units can be determined based on the entity association relationships between multiple first entities. This allows the establishment of first bidirectional reference links between the structured knowledge units corresponding to these multiple first entities, thereby forming a closely connected power operation and maintenance knowledge network. Specifically, based on the entity association relationships, bidirectional reference identifiers pointing to structured knowledge units corresponding to other entities can be added to the structured knowledge units corresponding to each first entity, thus forming a bidirectional reference knowledge graph between multiple structured knowledge units.

[0109] In other words, this bidirectional reference knowledge graph is a knowledge graph structure built with structured knowledge units as nodes and the first bidirectional reference links between structured knowledge units as edges.

[0110] For example, taking entity relationships as an example, such as: 110kV GIS gas chamber ↔ SF6 gas detection, 110kV GIS gas chamber / SF6 gas detection ↔ SF6 leak handling operation, 110kV GIS gas chamber / SF6 leak handling operation / SF6 gas detection ↔ SF6 work safety measures, a first bidirectional reference link A can be established between the structured knowledge units corresponding to "110kV GIS gas chamber" and "SF6 gas detection"; and a first bidirectional reference link B can be established between the structured knowledge units corresponding to "110kV GIS gas chamber" and "SF6 leak handling operation"; similarly, a first bidirectional reference link C can be established between the structured knowledge units corresponding to "110kV GIS gas chamber" and "SF6 work safety measures". Furthermore, in the structured knowledge unit corresponding to "110KV GIS gas chamber," reference identifiers pointing to the structured knowledge units corresponding to "SF6 gas detection," "SF6 leak handling operation," and "SF6 work safety measures" are added, such as "{'SF6 gas detection':[link ID_A],'SF6 leak handling operation': [link ID_B], 'SF6 work safety measures': [link ID_C]}". Here, link ID_A, link ID_B, and link ID_C represent the indices used to uniquely identify each first bidirectional reference link. Additionally, reference identifiers pointing to "110KV GIS gas chamber" are also added to the structured knowledge units corresponding to "SF6 gas detection," "SF6 leak handling operation," and "SF6 work safety measures," such as link ID_A, link ID_B, and link ID_C, respectively, thereby achieving bidirectional communication. In this way, when maintenance personnel view the structured knowledge unit of "110KV GIS Gas Chamber," it automatically links to the structured knowledge units corresponding to "SF6 Gas Detection," "SF6 Leakage Handling Operations," and "SF6 Work Safety Measures." Similarly, for other structured knowledge units, the same process is used, establishing first bidirectional reference links to other related entities in the structured knowledge units corresponding to "SF6 Gas Detection," "SF6 Leakage Handling Operations," and "SF6 Work Safety Measures," thereby ensuring interconnectivity between nodes in the entire power maintenance knowledge network.

[0111] In other embodiments, the structured knowledge units of this application can also be represented in Markdown format. For example, the structured knowledge unit corresponding to the first entity "110KV GIS gas chamber" can be represented in Markdown format as follows:

[0112] ```markdown

[0113] #110KV GIS Gas Chamber

[0114] ##Key Operations and Maintenance Knowledge Snippets

[0115] - Rated operating pressure 0.6MPa, alarm threshold 0.52MPa

[0116] -Annual SF6 humidity test ≤300μL / L

[0117] -Daily inspection records of pressure gauge readings and ambient temperature correction values

[0118] - Before opening the cover for maintenance, recover residual gas; only gas with an oxygen content ≥18% is allowed to enter.

[0119] - After maintenance, the annual leakage rate should be ≤0.5% in the airtightness test. After replenishing the air, let it stand for 30 minutes and then test for trace moisture.

[0120] ##Metadata

[0121] - Equipment type: Switchgear

[0122] - Voltage level: 1100kV

[0123] Substation in question: XX Substation

[0124] ## First Two-Way Reference Link

[0125] Related references: [{'SF6 gas detection': 'link ID_A'},{'SF6 leak handling procedures': 'link ID_B'}, {'SF6 work safety measures': 'link ID_C'}]

[0126] ```markdown

[0127] By constructing the first bidirectional reference link mentioned above, the power operation and maintenance knowledge is transformed from isolated fragments to a connected network. This not only improves the accuracy and completeness of knowledge retrieval, but also helps operation and maintenance personnel quickly locate relevant operating procedures and safety specifications when dealing with specific equipment failures, thereby effectively reducing the risk of misoperation and improving operation and maintenance efficiency and safety.

[0128] In other embodiments, after constructing multiple structured knowledge units and the first bidirectional reference links between them, the content of any structured knowledge unit can be dynamically updated. Specifically, taking any one of the multiple structured knowledge units (e.g., the first structured knowledge unit) as an example, when a change is detected in the first entity or key operation and maintenance knowledge fragment in the first structured knowledge unit, the changed first entity or key operation and maintenance knowledge fragment can be synchronously updated to the first structured knowledge unit to ensure the real-time nature and consistency of knowledge. Furthermore, it is also necessary to check whether the changed first entity or key operation and maintenance knowledge fragment has logical conflicts or semantic contradictions with the key operation and maintenance knowledge fragments in other structured knowledge units (e.g., the second structured knowledge unit) associated with the first bidirectional reference link corresponding to the first structured knowledge unit. If a conflict or semantic contradiction exists, the first bidirectional reference link between the first structured knowledge unit and the second structured knowledge unit is disconnected, and a conflict alarm message is generated to prompt the operation and maintenance personnel to manually review or correct the relevant knowledge points; if there is no conflict or semantic contradiction, the first bidirectional reference link between the first structured knowledge unit and the second structured knowledge unit remains unchanged, and an update log is recorded to complete the dynamic update of the first structured knowledge unit.

[0129] This method enables the automatic detection of logical consistency among related knowledge units when modifying key operation and maintenance knowledge fragments of any structured knowledge unit. This ensures the dynamic evolution of the knowledge base while maintaining the accuracy and reliability of the entire power operation and maintenance knowledge network.

[0130] In other embodiments, after constructing the bidirectional reference knowledge graph, i.e., obtaining multiple structured knowledge units and the first bidirectional reference links between these structured knowledge units, these multiple structured knowledge units can be filtered to construct converged knowledge units. This can be understood in the following way:

[0131] First, we detect the co-occurrence frequency of each structured knowledge unit in the question-answering process, as well as the number of links in the corresponding first bidirectional reference link.

[0132] It should be understood that co-occurrence frequency can be interpreted as the number of times multiple structured knowledge units are simultaneously hit or invoked in the same question-and-answer request, used to measure the degree of collaborative association between structured knowledge units. Link count, on the other hand, can be understood as the number of links between other structured knowledge units directly associated with a structured knowledge unit in the bidirectional reference knowledge graph.

[0133] Subsequently, if the co-occurrence frequency of multiple first knowledge units among these multiple structured knowledge units in question-answering processing is greater than a first threshold within a preset time window, and / or the number of first bidirectional reference links between these multiple first knowledge units is greater than a second threshold, then these multiple first knowledge units are aggregated to obtain a candidate knowledge unit cluster. It should be understood that these multiple first knowledge units are structured knowledge units whose co-occurrence frequency is greater than the first threshold and / or whose number of links is greater than the second threshold; that is, these multiple first knowledge units are structured knowledge units that satisfy the threshold conditions of co-occurrence frequency and / or number of links.

[0134] Next, the key operational knowledge fragments, metadata, and corresponding first bidirectional reference links of each first knowledge unit within the candidate knowledge unit cluster are assembled to obtain a converged summary. This converged summary is then encapsulated to obtain the converged knowledge unit. Furthermore, a second bidirectional reference link is constructed between the converged knowledge unit and each first knowledge unit to achieve a hierarchical association between the converged knowledge unit and the original structured knowledge unit.

[0135] It should be understood that the aggregated knowledge unit includes a second bidirectional reference link between the aggregated knowledge unit and each first knowledge unit, as well as the unit identification information of each first knowledge unit, which is used to quickly locate the key operation and maintenance knowledge fragments, metadata and corresponding first bidirectional reference links of the multiple first knowledge units covered by the aggregated knowledge unit during question and answer retrieval.

[0136] For example, taking the aforementioned structured knowledge units including "110KV GIS gas chamber" knowledge unit 1, "SF6 gas detection" knowledge unit 2, "SF6 leak handling operation" knowledge unit 3, and "SF6 operation safety measures" knowledge unit 4, it can be detected that the co-occurrence frequency of these four structured knowledge units within a preset time window is 5 times, and the number of first bidirectional reference links between these four structured knowledge units is 6. If the first threshold is 3 times and the second threshold is 4 links, then the co-occurrence frequency can be detected as 5 times, which is greater than the first threshold of 3 times, and / or the number of links is 6, which is greater than the second threshold of 4 links. Based on this, these four structured knowledge units can be aggregated to obtain a candidate knowledge unit cluster, and then the key operation and maintenance knowledge fragments, metadata, and corresponding first bidirectional reference links of each first knowledge unit in the candidate knowledge unit cluster can be assembled to obtain a converged summary, such as "alarm confirmation → gas chamber pressure verification → ventilation and protection → leak point location and power restoration". Further, this converged summary is encapsulated into a converged knowledge unit "SF6 leak handling converged point". Thus, the aggregation type can be marked as "emergency response" in the metadata of the aggregated knowledge unit "SF6 Leakage Handling Aggregation Point". Furthermore, it inherits the common attribute fields (equipment type, voltage level) within the cluster and adds an aggregation hierarchy field. This aggregation hierarchy field records the unit identifiers of the covered knowledge units, such as the unit identifiers of knowledge units 1 to 4. Simultaneously, a second bidirectional reference link is constructed between the aggregated knowledge unit and knowledge units 1 to 4, and stored in the bidirectional reference knowledge graph, forming a hierarchical bidirectional index structure from the original knowledge unit to the aggregated knowledge unit.

[0137] In other embodiments, semantic deduplication and redundancy pruning can be performed on the candidate knowledge unit cluster to remove duplicate or low-value segments.

[0138] In other embodiments, such as Figure 3 As shown, step 203 includes steps 301 to 303. Wherein:

[0139] 301. Match the audio text with the aggregated knowledge units to obtain the matching results; the aggregated knowledge units are encapsulated based on the key operation and maintenance knowledge fragments, metadata and corresponding first bidirectional reference links of multiple first knowledge units in the candidate knowledge unit cluster. The candidate knowledge unit cluster is aggregated when the co-occurrence frequency of multiple first knowledge units in multiple structured knowledge units in question-answering processing is greater than the first threshold within a preset time window, and / or the number of links of the first bidirectional reference links between multiple first knowledge units is greater than the second threshold.

[0140] For example, after constructing the aforementioned bidirectional reference knowledge graph and aggregated knowledge units, a preset large language model deployed in the power operation equipment can be invoked to perform intent recognition on the audio text, thereby obtaining the question-and-answer intent of the audio text. If the question-and-answer intent of the audio text belongs to the operation and maintenance question-and-answer intent covered by the bidirectional reference knowledge graph, then the audio text is further matched with the aggregated knowledge unit to determine the matching result.

[0141] In some examples, semantic feature vectors of the audio text and the converged summary in the converged knowledge unit can be extracted. Then, using algorithms such as cosine similarity, the similarity distance between the semantic feature vector of the audio text and the semantic feature vector of the converged summary is calculated to obtain the semantic similarity between the audio text and the converged summary in the converged knowledge unit. The matching result is then determined based on this semantic similarity and a preset semantic threshold.

[0142] Specifically, if the semantic similarity is greater than or equal to a preset semantic threshold, the matching result is determined to be a match between the audio text and the aggregated knowledge unit. In this case, when the matching result is a match between the audio text and the aggregated knowledge unit, the aggregated knowledge unit can be retrieved first, i.e., step 302 is executed. Conversely, if the semantic similarity is less than the preset semantic threshold, the matching result is determined to be a mismatch between the audio text and the aggregated knowledge unit. In this case, the process reverts to the original structured knowledge unit for fine-grained retrieval, i.e., step 303 is executed.

[0143] In other examples, if the question-and-answer intent of the audio text does not fall under the operation and maintenance question-and-answer intent covered by the bidirectional reference knowledge graph, guidance information is generated to indicate that the current round of question-and-answer processing has not retrieved an operation and maintenance knowledge fragment that matches the audio text.

[0144] In other embodiments, to determine whether the question-and-answer intent of the audio text belongs to the maintenance question-and-answer intent covered by the bidirectional reference knowledge graph, it can be done by extracting key entities from the audio text and comparing them with the first entity of each structured knowledge unit in the bidirectional reference knowledge graph. If the comparison result shows that the overlap between the key entity and the first entity reaches a preset matching threshold, the question-and-answer intent is determined to belong to the maintenance question-and-answer intent covered by the bidirectional reference knowledge graph, and subsequent retrieval operations are performed. Conversely, if the overlap between the key entity and the first entity does not reach the preset matching standard, the question-and-answer intent is determined to be outside the maintenance question-and-answer intent covered by the bidirectional reference knowledge graph, and guidance information is directly generated.

[0145] 302. If the matching result indicates that the audio text matches the aggregated knowledge unit, then the audio text is retrieved and processed based on the aggregated knowledge unit to obtain the target operation and maintenance knowledge fragment.

[0146] For example, if the matching result indicates that the audio text matches the aggregated knowledge unit, the aggregated knowledge unit can be used first to retrieve and process the audio text, thereby obtaining the target operation and maintenance knowledge fragment.

[0147] Specifically, since the aggregated knowledge unit is encapsulated based on key operational knowledge fragments, metadata, and corresponding first bidirectional reference links of multiple first knowledge units within the candidate knowledge unit cluster, and since there are corresponding second bidirectional reference links between the aggregated knowledge unit and these multiple first knowledge units, the aggregated knowledge unit can carry richer semantic context information. Therefore, when retrieving and processing audio text based on this aggregated knowledge unit, multiple first knowledge units that have second bidirectional reference links with the aggregated knowledge unit can be identified based on each second bidirectional reference link. Subsequently, based on the key operational knowledge fragments in these multiple first knowledge units, target operational knowledge fragments are generated, thereby improving retrieval efficiency and semantic coverage while retaining fine-grained information.

[0148] For example, taking the aforementioned knowledge unit "SF6 Leakage Handling Hub" as an example, when maintenance personnel ask "How to handle the low SF6 pressure alarm of main transformer No. 3" via voice, after the aforementioned voice recognition, intent judgment, and knowledge unit matching, if it is determined that the audio text "How to handle the low SF6 pressure alarm of main transformer No. 3" matches the hub summary in the knowledge unit "SF6 Leakage Handling Hub," then the associated first knowledge unit can be found based on the second bidirectional reference link corresponding to the knowledge unit "SF6 Leakage Handling Hub." Examples include "110KV GIS Gas Chamber" knowledge unit 1, "SF6 Gas Detection" knowledge unit 2, "SF6 Leakage Handling Operation" knowledge unit 3, and "SF6 Operation Safety Measures" knowledge unit 4. In this way, the key maintenance knowledge fragments of these four knowledge units can be aggregated to generate the target maintenance knowledge fragment.

[0149] By employing the above method, when a converged knowledge unit is hit, the first knowledge unit, which is associated with the second bidirectional reference link, can be effectively aggregated, thereby improving the semantic coverage and fine-grained information retention capabilities of the retrieval. Furthermore, using converged knowledge units for retrieval allows the retrieval scope to be narrowed to a limited set of converged knowledge units, even when the computing power of the edge NPU is constrained. This significantly reduces the computational load and power consumption of the edge NPU, avoids a full traversal of massive amounts of original knowledge units, further compressing retrieval time, and results in more complete semantics for the target operation and maintenance knowledge fragments retrieved through cross-knowledge point associations.

[0150] Alternatively, 303, if the matching result indicates that the audio text does not match the aggregated knowledge unit, then based on the key operation and maintenance knowledge fragments in each of the multiple structured knowledge units and the first bidirectional reference links between the multiple structured knowledge units, the audio text is retrieved and processed to obtain the target operation and maintenance knowledge fragment.

[0151] For example, if the matching result indicates that the audio text does not match the aggregated knowledge unit, the aggregated retrieval path is skipped, and instead, a fine-grained retrieval is performed based on multiple structured knowledge units and their first bidirectional reference links. Specifically, the audio text is retrieved based on the key operation and maintenance knowledge fragments within each of the multiple structured knowledge units, as well as the first bidirectional reference links between the multiple structured knowledge units. The associated knowledge units are expanded using the first bidirectional reference links, thereby accurately locating and generating the target operation and maintenance knowledge fragment. In this way, even if the aggregated retrieval fails, cross-knowledge unit association expansion can be achieved through the first bidirectional reference links, ensuring that the target operation and maintenance knowledge fragment can still be accurately located through fine-grained association expansion even when the aggregated retrieval fails, balancing retrieval accuracy and response efficiency.

[0152] In this embodiment, the matching results between audio text and aggregated knowledge units are used to dynamically select either the aggregated retrieval path or the fine-grained retrieval path based on the original structured knowledge units. This ensures both the breadth of semantic coverage and the accuracy of fine-grained information retrieval and response efficiency.

[0153] In other embodiments, such as Figure 4 As shown, step 303 may include steps 401 to 404. Wherein:

[0154] 401. Perform entity recognition on the audio text to obtain the second entity.

[0155] For example, a pre-trained named entity recognition (NER) model is used to perform entity recognition on audio text to obtain a second entity in the audio text.

[0156] 402. Based on the second entity, audio text, and the first entity corresponding to each structured knowledge unit, multiple structured knowledge units are filtered to obtain the target structured knowledge unit.

[0157] For example, after obtaining the second entity, the multiple structured knowledge units can be filtered based on the second entity, the audio text, and the first entity corresponding to each structured knowledge unit, thereby obtaining the target structured knowledge unit.

[0158] Specifically, the second entity can be matched with the unique first entity of each of these multiple structured knowledge units, and the matched first entities can be used as candidate entities. It should be understood that the number of candidate entities can be one or more. Since each structured knowledge unit also includes metadata corresponding to the first entity, the metadata can be used to characterize the power operation and maintenance attributes of the corresponding first entity. Therefore, after determining the candidate entities, operation and maintenance keywords can be further extracted from the audio text. These keywords are used to describe the power operation and maintenance status of the second entity. Subsequently, the metadata in the structured knowledge unit corresponding to the candidate entity is used as the target metadata, and then the operation and maintenance keywords are matched with the attribute fields in the target metadata. If the operation and maintenance keywords successfully match the attribute fields in the target metadata, the structured knowledge unit corresponding to the candidate entity is determined as the target structured knowledge unit.

[0159] Conversely, if the operation and maintenance keywords fail to match the attribute fields in the target metadata, the entity name of each candidate entity, the key operation and maintenance knowledge fragments in the structured knowledge unit corresponding to the candidate entity, and the corresponding attribute fields in the target metadata are extracted to reassess the matching degree between the candidate entity and the operation and maintenance context in the audio text. It should be understood that the entity name of a candidate entity can be understood as the title of the structured knowledge unit, used to quickly locate potentially relevant knowledge scope in the initial screening stage; the key operation and maintenance knowledge fragments in the structured knowledge unit corresponding to the candidate entity can be understood as the main text content of the structured knowledge unit. Next, the operation and maintenance keywords are matched with the entity names of each candidate entity to obtain the entity matching score for each candidate entity. For example, the cosine similarity algorithm can be used to calculate the similarity between the vector of the operation and maintenance keywords and the vector of the entity name as the entity matching score. Similarly, the operation and maintenance keywords are also matched with the key operation and maintenance knowledge fragments in the structured knowledge unit corresponding to each candidate entity to obtain the fragment matching score for each candidate entity. For example, the cosine similarity algorithm can be used to calculate the similarity between the semantic vector of the operation and maintenance keywords and the semantic vector of the key operation and maintenance knowledge fragment as the fragment matching score. Furthermore, by matching operational keywords with attribute fields in the target metadata, a tag matching score is obtained for each candidate entity. For example, a cosine similarity algorithm can be used to calculate the similarity between the vector of operational keywords and the vector of attribute fields in the target metadata, serving as the tag matching score. Then, based on the entity matching score, fragment matching score, and tag matching score for each candidate entity, a target matching score for the structured knowledge unit corresponding to each candidate entity is calculated. For example, a weighted summation method can be used, employing preset weight coefficients to weight and sum the entity matching score, fragment matching score, and tag matching score to obtain the target matching score. Finally, based on the target matching score of the structured knowledge unit corresponding to each candidate entity and a preset threshold, the structured knowledge units corresponding to multiple candidate entities are filtered to obtain the target structured knowledge unit.

[0160] In other embodiments, during the process of filtering structured knowledge units corresponding to multiple candidate entities based on the target matching score and a preset threshold for each candidate entity, the target matching score of each candidate entity's structured knowledge unit can be compared with the preset threshold first. From the structured knowledge units corresponding to multiple candidate entities, structured knowledge units with target matching scores greater than or equal to the preset threshold are selected. Subsequently, from the structured knowledge units with target matching scores greater than or equal to the preset threshold, a preset number of structured knowledge units are selected as the target structured knowledge units.

[0161] For example, when the preset quantity is 3, if there are 5 structured knowledge units with a target matching score greater than or equal to the preset threshold, these 5 structured knowledge units are sorted in descending order of their target matching scores, and the top 3 structured knowledge units are selected as target structured knowledge units. If there are fewer than 3 structured knowledge units with a target matching score greater than or equal to the preset threshold, for example, only 2, these 2 structured knowledge units are directly selected as target structured knowledge units; alternatively, based on actual business needs, structured knowledge units corresponding to other candidate entities with target matching scores closest to the preset threshold can be selected until the preset quantity is met. Alternatively, structured knowledge units that meet the preset quantity can be randomly selected from those structured knowledge units with target matching scores greater than or equal to the preset threshold as target structured knowledge units. This approach ensures that the final selected target structured knowledge units meet both relevance requirements and quantity limits, thus providing a high-quality data foundation for generating accurate target operation and maintenance knowledge fragments.

[0162] In other embodiments, if the target matching score of the structured knowledge unit corresponding to each candidate entity is less than a preset threshold, then the text feature vector of the audio text and the text feature vector of the key operation and maintenance knowledge fragments in the structured knowledge unit corresponding to each candidate entity are extracted. Subsequently, the similarity distance between the text feature vector of the audio text and the text feature vector of the key operation and maintenance knowledge fragments in the structured knowledge unit corresponding to each candidate entity is calculated to obtain the text similarity between the audio text and the key operation and maintenance knowledge fragments in the structured knowledge unit corresponding to each candidate entity. Then, based on the text similarity, the structured knowledge units corresponding to multiple candidate entities are filtered to obtain the target structured knowledge unit.

[0163] Specifically, structured knowledge units corresponding to multiple candidate entities can be sorted according to text similarity from high to low, and the top-ranked structured knowledge units can be selected as target structured knowledge units. Alternatively, structured knowledge units with text similarity greater than or equal to a preset similarity threshold can be selected as target structured knowledge units. By introducing similarity calculation of text feature vectors, the deep semantic relationships between audio text and candidate knowledge units can be further explored even when the initial matching score is low. This improves the accuracy and recall of target structured knowledge unit selection and avoids the omission of relevant operational knowledge due to a low initial matching score.

[0164] 403. Based on the first bidirectional reference link corresponding to the target structured knowledge unit, determine the associated structured knowledge units related to the target structured knowledge unit.

[0165] For example, after identifying the target structured knowledge unit, one can use the target structured knowledge unit as a starting point and traverse or search along the first bidirectional reference link to find related structured knowledge units that have a bidirectional reference relationship with the target structured knowledge unit. The target structured knowledge unit is one end anchor of the first bidirectional reference link, and its corresponding related structured knowledge unit can be understood as the other end anchor of the first bidirectional reference link.

[0166] 404. Generate target operation and maintenance knowledge fragments based on key operation and maintenance knowledge fragments in the target structured knowledge unit and key operation and maintenance knowledge fragments in the associated structured knowledge unit.

[0167] For example, after determining the associated structured knowledge units, the key operation and maintenance knowledge fragments in the target structured knowledge unit can be integrated with the key operation and maintenance knowledge fragments in the associated structured knowledge units to form a complete target operation and maintenance knowledge fragment.

[0168] By employing the aforementioned method, the knowledge integration mechanism based on the first bidirectional reference link can fully leverage the bidirectional reference relationship between structured knowledge units. This overcomes the limitations of traditional solutions that rely on original fragments from the original operation and maintenance guidance text during the retrieval process. It effectively integrates and supplements key operation and maintenance knowledge fragments corresponding to first entities with different semantic boundaries, thereby constructing target operation and maintenance knowledge fragments with more complete semantics and stronger contextual relevance. This significantly improves the comprehensiveness and accuracy of the final generated target operation and maintenance knowledge fragments, laying the foundation for subsequent reasoning of more accurate target operation and maintenance answers.

[0169] In other embodiments, Figure 5 Another flowchart of the question-and-answer processing method for power operation and maintenance scenarios provided in this application is shown. Figure 5 As shown, taking a page with structured knowledge units represented in Markdown format as an example, the question-and-answer processing method for power operation and maintenance scenarios can include the following steps:

[0170] First, in step 501, if a trigger operation (e.g., a voice wake-up word) is detected for the target object targeting the target maintenance task, the corresponding question audio is collected. In step 502, the question audio is processed using a power industry-specific terminology mapping table to obtain the corresponding audio text. In step 503, if the question-and-answer intent of the audio text belongs to the maintenance question-and-answer intent covered by the bidirectional reference knowledge graph, it is determined whether the audio text matches the aggregated knowledge unit. If they match, steps 504 and 519-520 are executed. Otherwise, if they do not match, steps 505-520 are executed.

[0171] Specifically, in step 504, if the audio text matches the aggregated knowledge unit, then based on each second bidirectional reference link, the first knowledge unit that has a second bidirectional reference link with the aggregated knowledge unit is determined, and based on the key operation and maintenance knowledge fragments in multiple first knowledge units, the target operation and maintenance knowledge fragment is generated.

[0172] Specifically, in step 505, entity recognition is performed on the audio text to obtain a second entity. In step 506, the second entity is matched with the first entities corresponding to each of the multiple pages, and the matched first entities are used as candidate entities. In step 507, maintenance keywords are extracted from the audio text; these maintenance keywords describe the power maintenance status of the second entity. In step 508, it is determined whether the maintenance keywords match the attribute fields in the target metadata of the page corresponding to the candidate entity. If they match, steps 509 and 518-520 are executed; if they do not match, steps 510 to 520 are executed.

[0173] Step 509: If the operation and maintenance keywords match the attribute fields in the target metadata, then the page corresponding to the candidate entity is determined as the target page.

[0174] Step 510: If the operation and maintenance keywords do not match the attribute fields in the target metadata, extract the entity name of each candidate entity, the key operation and maintenance knowledge fragments in the page corresponding to the candidate entity, and the attribute fields in the target metadata. Furthermore, in Step 511, calculate the entity matching score, fragment matching score, and tag matching score. Specifically, match the operation and maintenance keywords with the entity names of each candidate entity to obtain the entity matching score for each candidate entity. Also, match the operation and maintenance keywords with the key operation and maintenance knowledge fragments in the page corresponding to each candidate entity to obtain the fragment matching score for each candidate entity. And, match the operation and maintenance keywords with the attribute fields in the target metadata to obtain the tag matching score for each candidate entity. In Step 512, calculate the target matching score for the page corresponding to each candidate entity based on the entity matching score, fragment matching score, and tag matching score. In Step 513, determine whether the target matching score for the page corresponding to each candidate entity is greater than or equal to a preset threshold. If there is a target matching score greater than or equal to the preset threshold, then proceed to step 514 and steps 518-520; if all target matching scores are less than the preset threshold, then proceed to steps 515 to 520.

[0175] Step 514: Select a preset number of pages from the pages whose target matching score is greater than or equal to the preset threshold as target pages.

[0176] Step 515: If the target matching scores are all less than the preset threshold, extract the text feature vector of the audio text and the text feature vector of the key operation and maintenance knowledge fragments in the pages corresponding to each candidate entity. Next, execute step 516: Calculate the similarity distance between the text feature vector of the audio text and the text feature vector of the key operation and maintenance knowledge fragments in the pages corresponding to each candidate entity, obtaining the text similarity between the audio text and the key operation and maintenance knowledge fragments in the pages corresponding to each candidate entity. In step 517, based on the text similarity, filter the pages corresponding to multiple candidate entities to obtain the target pages.

[0177] Furthermore, after obtaining the target page, in step 518, the audio text is retrieved based on the key operation and maintenance knowledge fragments in each of the multiple target pages and the first bidirectional reference links between the multiple target pages to obtain the target operation and maintenance knowledge fragments.

[0178] Finally, in step 519, reasoning is performed on the target operation and maintenance knowledge fragments and audio text based on a preset large language model to generate the target operation and maintenance answer. In step 520, the target operation and maintenance answer is processed by speech synthesis to obtain the target answer audio, which is used to guide the target object to perform the target operation and maintenance task.

[0179] Through the above method, the entire process from wake-up → questioning → retrieval → answering → broadcasting is completed locally. This involves full voice interaction, requiring no network or screen touch, thus solving the problems of cloud inaccessibility and the impracticality of screen interaction. It also enables low-power offline deployment. Furthermore, by utilizing a power industry-specific terminology mapping table and multi-level structured retrieval, it not only significantly improves the accuracy of technical terminology recognition but also balances retrieval precision and recall. Compared to the general RAG architecture, this embodiment significantly reduces errors and illusions in understanding technical terms.

[0180] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0181] Based on the same inventive concept, this application also provides a power operation device for implementing the question-and-answer processing method for power operation and maintenance scenarios described above. The solution provided by the device is similar to the solution described in the above method; therefore, the specific limitations in one or more power operation device embodiments provided below can be found in the limitations of the question-and-answer processing method for power operation and maintenance scenarios described above, and will not be repeated here.

[0182] In one exemplary embodiment, such as Figure 6 As shown, a power work device is provided, comprising:

[0183] The response and acquisition unit 601 is used to respond to the trigger operation of the target object for the target operation and maintenance task and acquire the problem audio corresponding to the target object;

[0184] The speech recognition unit 602 is used to perform speech recognition processing on the problematic audio to obtain the audio text corresponding to the problematic audio.

[0185] The retrieval processing unit 603 is used to perform retrieval processing on audio text based on key operation and maintenance knowledge fragments in multiple structured knowledge units and the first bidirectional reference link between multiple structured knowledge units to obtain the target operation and maintenance knowledge fragment. The two ends of the first bidirectional reference link are anchored to two related structured knowledge units respectively. The structured knowledge unit is an independent knowledge carrier that is pre-compiled and generated from multiple power operation and maintenance guidance texts to carry key operation and maintenance knowledge fragments.

[0186] The reasoning unit 604 is used to reason about the target operation and maintenance knowledge fragments and audio text based on a preset large language model, and generate the target operation and maintenance answer.

[0187] The speech synthesis unit 605 is used to perform speech synthesis processing on the target operation and maintenance answer to obtain the target answer audio, which is used to guide the target object to perform the target operation and maintenance task.

[0188] In other embodiments, the retrieval processing unit 603 is specifically used for:

[0189] The audio text and the aggregated knowledge units are matched to obtain the matching results. The aggregated knowledge units are encapsulated based on the key operation and maintenance knowledge fragments, metadata and corresponding first bidirectional reference links of multiple first knowledge units in the candidate knowledge unit cluster. The candidate knowledge unit cluster is aggregated when the co-occurrence frequency of multiple first knowledge units in multiple structured knowledge units in question answering processing is greater than the first threshold within a preset time window, and / or the number of links of the first bidirectional reference links between multiple first knowledge units is greater than the second threshold.

[0190] If the matching result indicates that the audio text matches the aggregated knowledge unit, then the audio text is retrieved and processed based on the aggregated knowledge unit to obtain the target operation and maintenance knowledge fragment;

[0191] or,

[0192] If the matching result indicates that the audio text does not match the aggregated knowledge unit, then the audio text is retrieved based on the key operation and maintenance knowledge fragments in each of the multiple structured knowledge units and the first bidirectional reference links between the multiple structured knowledge units to obtain the target operation and maintenance knowledge fragment.

[0193] In other embodiments, a second bidirectional reference link exists between the aggregated knowledge unit and each first knowledge unit; the retrieval processing unit 603 is specifically used for:

[0194] If the matching result indicates that the audio text matches the converged knowledge unit, then based on each second bidirectional reference link, determine the first knowledge unit that has a second bidirectional reference link with the converged knowledge unit;

[0195] Based on key operation and maintenance knowledge fragments from multiple first knowledge units, a target operation and maintenance knowledge fragment is generated.

[0196] In other embodiments, each structured knowledge unit corresponds to a first entity; the retrieval processing unit 603 is specifically used for:

[0197] Entity recognition is performed on the audio text to obtain a second entity;

[0198] Based on the second entity, audio text, and the first entity corresponding to each structured knowledge unit, multiple structured knowledge units are filtered to obtain the target structured knowledge unit.

[0199] Based on the first bidirectional reference link corresponding to the target structured knowledge unit, identify the associated structured knowledge units related to the target structured knowledge unit;

[0200] Based on the key operation and maintenance knowledge fragments in the target structured knowledge unit and the key operation and maintenance knowledge fragments in the associated structured knowledge unit, the target operation and maintenance knowledge fragment is generated.

[0201] In one embodiment, each structured knowledge unit further includes metadata corresponding to the first entity, the metadata being used to characterize the power operation and maintenance attributes of the corresponding first entity; the retrieval processing unit 603 is specifically used for:

[0202] The second entity is matched with the first entity corresponding to each of the multiple structured knowledge units, and the matched first entity is taken as the candidate entity.

[0203] Extract operation and maintenance keywords from the audio text. These keywords are used to describe the power operation and maintenance status of the second entity.

[0204] If the operation and maintenance keywords match the attribute fields in the target metadata, then the structured knowledge unit corresponding to the candidate entity is determined as the target structured knowledge unit, and the target metadata is the metadata in the structured knowledge unit corresponding to the candidate entity.

[0205] In one embodiment, the retrieval processing unit 603 is further configured to:

[0206] If the operation and maintenance keywords do not match the attribute fields in the target metadata, then extract the entity name of each candidate entity, the key operation and maintenance knowledge fragments in the structured knowledge unit corresponding to the candidate entity, and the attribute fields in the target metadata.

[0207] The entity matching score for each candidate entity is obtained by matching operation and maintenance keywords with the entity names of each candidate entity.

[0208] The matching of operation and maintenance keywords with key operation and maintenance knowledge fragments in the structured knowledge units corresponding to each candidate entity is performed to obtain the fragment matching score for each candidate entity.

[0209] Based on the matching of operation and maintenance keywords with attribute fields in the target metadata, the tag matching score corresponding to each candidate entity is obtained;

[0210] Based on the entity matching score, fragment matching score and tag matching score corresponding to each candidate entity, the target matching score of the structured knowledge unit corresponding to each candidate entity is calculated.

[0211] Based on the target matching score and preset threshold of the structured knowledge units corresponding to each candidate entity, the structured knowledge units corresponding to multiple candidate entities are filtered to obtain the target structured knowledge units.

[0212] In one embodiment, the retrieval processing unit 603 is specifically used for:

[0213] The target matching score of the structured knowledge unit corresponding to each candidate entity is compared with a preset threshold. From the structured knowledge units corresponding to multiple candidate entities, the structured knowledge units corresponding to the target matching score are selected that are greater than or equal to the preset threshold.

[0214] From the structured knowledge units whose target matching scores are greater than or equal to a preset threshold, select a preset number of structured knowledge units as target structured knowledge units.

[0215] In one embodiment, based on the target matching score and preset threshold of the structured knowledge units corresponding to each candidate entity, the structured knowledge units corresponding to multiple candidate entities are filtered to obtain the target structured knowledge units, including:

[0216] If the target matching score of the structured knowledge unit corresponding to each candidate entity is less than the preset threshold, then extract the text feature vector of the audio text and the text feature vector of the key operation and maintenance knowledge fragment in the structured knowledge unit corresponding to each candidate entity.

[0217] Calculate the similarity distance between the text feature vector of the audio text and the text feature vector of the key operation and maintenance knowledge segment in the structured knowledge unit corresponding to each candidate entity, and obtain the text similarity between the audio text and the key operation and maintenance knowledge segment in the structured knowledge unit corresponding to each candidate entity.

[0218] Based on text similarity, the structured knowledge units corresponding to multiple candidate entities are filtered to obtain the target structured knowledge unit.

[0219] In one embodiment, the retrieval processing unit 603 is further configured to:

[0220] Before retrieving and processing the audio text to obtain the target operation and maintenance knowledge fragment, based on the key operation and maintenance knowledge fragments in each of the multiple structured knowledge units and the first bidirectional reference links between the multiple structured knowledge units, entity recognition is performed on multiple power operation and maintenance guidance texts to obtain multiple semantically non-overlapping first entities and entity association relationships.

[0221] Based on each first entity, the content of multiple power operation and maintenance guidance texts is extracted and split to obtain the key operation and maintenance knowledge fragments corresponding to each first entity.

[0222] Based on each first entity and the key operation and maintenance knowledge fragments corresponding to the first entity, a structured knowledge unit corresponding to each first entity is generated. The structured knowledge unit includes the key operation and maintenance knowledge fragments corresponding to the first entity and the metadata corresponding to the first entity.

[0223] Based on entity relationships, a first bidirectional reference link is constructed for the structured knowledge units corresponding to multiple first entities, thereby obtaining the first bidirectional reference link between multiple structured knowledge units.

[0224] Each module in the aforementioned power operation equipment can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the computer device's memory as software, so that the processor can call and execute the operations corresponding to each module.

[0225] In one exemplary embodiment, a computer device is provided, the internal structure of which can be as shown in the figure. Figure 7 As shown, the computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and databases. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores data such as power operation and maintenance guidance texts. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a question-and-answer processing method for power operation and maintenance scenarios.

[0226] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0227] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method described above.

[0228] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0229] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the method described above.

[0230] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0231] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0232] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0233] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for question and answer processing for a power operation and maintenance scene, characterized in that, Applied to electrical work equipment, the method includes: In response to a trigger operation by the target object targeting a specific maintenance task, obtain the audio of the problem corresponding to the target object; The problematic audio is subjected to speech recognition processing to obtain the audio text corresponding to the problematic audio. Based on the key operation and maintenance knowledge fragments in each of the multiple structured knowledge units and the first bidirectional reference links between the multiple structured knowledge units, the audio text is retrieved and processed to obtain the target operation and maintenance knowledge fragment. The two ends of the first bidirectional reference link are respectively anchored to two associated structured knowledge units. The structured knowledge unit is an independent knowledge carrier that is pre-compiled from multiple power operation and maintenance guidance texts to carry the key operation and maintenance knowledge fragment. Based on a pre-defined large language model, reasoning is performed on the target operation and maintenance knowledge fragments and the audio text to generate the target operation and maintenance answer; The target operation and maintenance answer is processed by speech synthesis to obtain the target answer audio, which is used to guide the target object to perform the target operation and maintenance task.

2. The method of claim 1, wherein, The audio text is retrieved based on key operation and maintenance knowledge fragments within multiple structured knowledge units and a first bidirectional reference link between these structured knowledge units to obtain the target operation and maintenance knowledge fragment, including: The audio text and the aggregated knowledge units are matched to obtain the matching result. The aggregated knowledge units are encapsulated based on the key operation and maintenance knowledge fragments, metadata, and corresponding first bidirectional reference links of multiple first knowledge units within the candidate knowledge unit cluster. The candidate knowledge unit cluster is aggregated when the co-occurrence frequency of multiple first knowledge units in the multiple structured knowledge units in question-answering processing is greater than a first threshold within a preset time window, and / or the number of links of the first bidirectional reference links between the multiple first knowledge units is greater than a second threshold. If the matching result indicates that the audio text matches the aggregated knowledge unit, then the audio text is retrieved based on the aggregated knowledge unit to obtain the target operation and maintenance knowledge fragment; or, If the matching result indicates that the audio text does not match the aggregated knowledge unit, then the audio text is retrieved based on the key operation and maintenance knowledge fragments in each of the multiple structured knowledge units and the first bidirectional reference links between the multiple structured knowledge units to obtain the target operation and maintenance knowledge fragment.

3. The method of claim 2, wherein, Each of the aggregated knowledge units has a second bidirectional reference link with each of the first knowledge units; if the matching result indicates that the audio text matches the aggregated knowledge unit, then the audio text is retrieved based on the aggregated knowledge unit to obtain the target operation and maintenance knowledge fragment, including: If the matching result indicates that the audio text matches the aggregated knowledge unit, then based on each of the second bidirectional reference links, a first knowledge unit that has a second bidirectional reference link with the aggregated knowledge unit is determined; Based on multiple key operation and maintenance knowledge fragments from the first knowledge unit, a target operation and maintenance knowledge fragment is generated.

4. The method of claim 2, wherein, Each of the structured knowledge units corresponds to a first entity; the audio text is retrieved based on the key operation and maintenance knowledge fragments in each of the multiple structured knowledge units and the first bidirectional reference links between the multiple structured knowledge units to obtain the target operation and maintenance knowledge fragment, including: Entity recognition is performed on the audio text to obtain a second entity; Based on the second entity, the audio text, and the first entity corresponding to each structured knowledge unit, multiple structured knowledge units are filtered to obtain the target structured knowledge unit; Based on the first bidirectional reference link corresponding to the target structured knowledge unit, the associated structured knowledge units related to the target structured knowledge unit are determined; Based on the key operation and maintenance knowledge fragments in the target structured knowledge unit and the key operation and maintenance knowledge fragments in the associated structured knowledge unit, a target operation and maintenance knowledge fragment is generated.

5. The method of claim 4, wherein, Each of the structured knowledge units further includes metadata corresponding to the first entity, the metadata being used to characterize the power operation and maintenance attributes corresponding to the first entity; the step of filtering multiple structured knowledge units based on the second entity, the audio text, and the first entity corresponding to each structured knowledge unit to obtain a target structured knowledge unit includes: The second entity is matched with the first entity corresponding to each of the multiple structured knowledge units, and the matched first entity is taken as the candidate entity. Extract maintenance keywords from the audio text; these maintenance keywords are used to describe the power maintenance status of the second entity. If the operation and maintenance keywords match the attribute fields in the target metadata, then the structured knowledge unit corresponding to the candidate entity is determined as the target structured knowledge unit, and the target metadata is the metadata in the structured knowledge unit corresponding to the candidate entity.

6. The method of claim 5, wherein, The method further includes: If the operation and maintenance keywords do not match the attribute fields in the target metadata, then extract the entity name of each candidate entity, the key operation and maintenance knowledge fragments in the structured knowledge unit corresponding to the candidate entity, and the attribute fields in the target metadata. The entity matching score for each candidate entity is obtained by matching the operation and maintenance keywords with the entity names of each candidate entity. The operation and maintenance keywords are matched with the key operation and maintenance knowledge fragments in the structured knowledge units corresponding to each candidate entity to obtain the fragment matching score corresponding to each candidate entity. Based on the matching of the operation and maintenance keywords with the attribute fields in the target metadata, the tag matching score corresponding to each candidate entity is obtained; Based on the entity matching score, fragment matching score and tag matching score corresponding to each candidate entity, calculate the target matching score of the structured knowledge unit corresponding to each candidate entity; Based on the target matching score and preset threshold of the structured knowledge units corresponding to each candidate entity, the structured knowledge units corresponding to multiple candidate entities are filtered to obtain the target structured knowledge units.

7. The method according to claim 6, characterized in that, The step of filtering the structured knowledge units corresponding to multiple candidate entities based on the target matching score and preset threshold of each candidate entity to obtain the target structured knowledge unit includes: The target matching score of the structured knowledge unit corresponding to each candidate entity is compared with the preset threshold, and the structured knowledge unit corresponding to the target matching score is selected from the structured knowledge units corresponding to multiple candidate entities. From the structured knowledge units whose target matching scores are greater than or equal to the preset threshold, select a preset number of structured knowledge units as target structured knowledge units.

8. The method according to claim 6, characterized in that, The step of filtering the structured knowledge units corresponding to multiple candidate entities based on the target matching score and preset threshold of each candidate entity to obtain the target structured knowledge unit includes: If the target matching score of the structured knowledge unit corresponding to each candidate entity is less than the preset threshold, then extract the text feature vector of the audio text and the text feature vector of the key operation and maintenance knowledge fragment in the structured knowledge unit corresponding to each candidate entity. Calculate the similarity distance between the text feature vector of the audio text and the text feature vector of the key operation and maintenance knowledge segment in the structured knowledge unit corresponding to each candidate entity, and obtain the text similarity between the audio text and the key operation and maintenance knowledge segment in the structured knowledge unit corresponding to each candidate entity. Based on the text similarity, the structured knowledge units corresponding to multiple candidate entities are filtered to obtain the target structured knowledge unit.

9. The method according to claim 1, characterized in that, Before retrieving the audio text based on key operation and maintenance knowledge fragments from multiple structured knowledge units and first bidirectional reference links between these structured knowledge units to obtain the target operation and maintenance knowledge fragment, the method further includes: Entity recognition is performed on the multiple power operation and maintenance guidance texts to obtain multiple semantically non-overlapping first entities and entity association relationships; Based on each first entity, the content of the multiple power operation and maintenance guidance texts is extracted and split to obtain the key operation and maintenance knowledge fragments corresponding to each first entity. Based on each first entity and the key operation and maintenance knowledge fragments corresponding to the first entity, a structured knowledge unit corresponding to each first entity is generated. The structured knowledge unit includes the key operation and maintenance knowledge fragments corresponding to the first entity and the metadata corresponding to the first entity. Based on the entity association relationship, a first bidirectional reference link is constructed for the structured knowledge units corresponding to the multiple first entities, thereby obtaining a first bidirectional reference link between the multiple structured knowledge units.

10. A power work equipment, characterized in that, The power operation equipment includes: The response and acquisition unit is used to respond to the trigger operation of the target object for the target operation and maintenance task and acquire the problem audio corresponding to the target object; A speech recognition unit is used to perform speech recognition processing on the question audio to obtain the audio text corresponding to the question audio. The retrieval processing unit is used to perform retrieval processing on the audio text based on the key operation and maintenance knowledge fragments in each of the multiple structured knowledge units and the first bidirectional reference links between the multiple structured knowledge units to obtain the target operation and maintenance knowledge fragment. The two ends of the first bidirectional reference link are respectively anchored to two associated structured knowledge units. The structured knowledge unit is an independent knowledge carrier that is pre-compiled from multiple power operation and maintenance guidance texts to carry the key operation and maintenance knowledge fragment. The reasoning unit is used to reason about the target operation and maintenance knowledge fragment and the audio text based on a preset large language model, and generate the target operation and maintenance answer; A speech synthesis unit is used to perform speech synthesis processing on the target operation and maintenance answer to obtain the target answer audio, which is used to guide the target object to perform the target operation and maintenance task.

11. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 9.