Power knowledge base question answering method and system based on improved RAG

Abstract

The invention provides an improved RAG-based power knowledge base question answering method and system, and the method comprises the steps: employing a mode of combining structured and unstructured data, carrying out the analysis and storage of various content types of drawings, tables and texts, and injecting the domain knowledge in a pre-training stage into a multi-modal Transform structure; according to the method, fragment correlation enhancement, optimization before retrieval and optimization after retrieval are applied to a model operation process of a multi-modal Transform structure, and an optimization result is obtained. According to the scheme, the intelligence, the adaptability and the response capability of the system are improved.

Description

Technical Field

[0001] This invention relates to the field of automation, and more specifically, to a question-answering method and system for an electrical knowledge base based on an improved RAG. Background Technology

[0002] With the rise of ChatGPT, large language models have demonstrated impressive language recognition, understanding, and reasoning capabilities in the field of Natural Language Processing (NLP), even surpassing human performance on multiple task metrics. However, these models also face numerous limitations, including limited knowledge coverage, lag in updates, generational illusions, and data security issues. To address these challenges, a semi-parametric approach—Retrieval-Augmented Generation (RAG)—has gradually become a research hotspot. This approach combines non-parametric corpus retrieval with parametric generative models, providing more reliable knowledge support for the models.

[0003] In the power industry, diverse work scenarios further highlight the applicability and potential challenges of RAG technology. For example, during substation inspections in remote areas, handling sudden faults, or emergency situations, poor network conditions or lack of coverage prevent staff from accessing central servers or cloud resources in real time. If the question-answering system relies entirely on an online model, it may lead to service interruptions and fail to meet actual needs. Furthermore, if the question-answering system cannot dynamically optimize based on user behavior and feedback, the model will struggle to adapt to changing needs, and its understanding of user habits and domain-specific questions will gradually become rigid. Current mainstream RAG systems typically extract the most relevant documents through a retrieval model and then process them through a generative model. However, this design faces inherent problems, such as insufficient retrieval accuracy; simple similarity retrieval methods can easily miss key information or introduce noisy data. Simultaneously, generation efficiency is low; the presence of low-quality or irrelevant context further increases the generation burden, reducing the quality and speed of responses. As the scale of data continues to expand, RAG systems are facing the dual challenges of declining retrieval efficiency and a drop in the quality of generated content.

[0004] To address the aforementioned issues, there is an urgent need for a question-and-answer method and system for the power knowledge base based on an improved RAG. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a question-answering method and system for an improved RAG-based power knowledge base.

[0006] The present invention adopts the following technical solution.

[0007] A question-answering method for an electricity knowledge base based on an improved RAG (Research Aggregator), characterized by the following steps:

[0008] Step 1: Using a combination of structured and unstructured data, we parse and store various content types such as drawings, tables, and texts, and inject the domain knowledge from the pre-training stage into the multimodal Transformer structure.

[0009] Step 2: Apply the sharding relevance enhancement, pre-retrieval optimization, and post-retrieval optimization to the model operation process of the multimodal Transformer structure to obtain the optimization results.

[0010] The present invention further employs the following optional solutions.

[0011] The method employs a combination of structured and unstructured data to parse and store various content types, including drawings, tables, and text. Domain knowledge from the pre-training phase is injected into the multimodal Transformer structure, including:

[0012] Multimodal data is preprocessed, semantic embeddings are generated using the BERT language model for the preprocessed text data, visual features are extracted from the preprocessed image data using a convolutional neural network, audio representations are generated from the preprocessed audio data using spectrograms, and global video dynamic features are extracted from the preprocessed video data by combining keyframe features with a time series model.

[0013] A multimodal pre-trained model is used to map image and text features to a shared vector space, and the similarity between positive and negative samples is optimized through contrastive learning.

[0014] Data is processed, cleaned, and labeled using a large language model.

[0015] The preprocessing of multimodal data includes:

[0016] Clean the text data, remove special characters, perform word segmentation, remove stop words, and standardize the data;

[0017] Image data is resized, normalized, and augmented.

[0018] Noise reduction, segmentation, and MFCC feature extraction are performed on the audio data;

[0019] Extract keyframes or decompose video data into image frames.

[0020] The process of processing, cleaning, and labeling data using a large language model includes:

[0021] The large language model adopts a multimodal Transformer structure, including an embedding layer, multiple underlying Transformers, a cross-modal fusion layer, and a task header.

[0022] The training of the large language model is as follows:

[0023] Through large-scale public corpora The weight parameters θ of the large language model are obtained through training. base ,

[0024] Utilizing domain data Update the model weights to optimize the weight parameters of the large language model to θ. domain ;

[0025] The objective function is to minimize the language modeling loss.

[0026] The large language model is fine-tuned after training. The fine-tuning process includes the following steps:

[0027] Freeze the weights of some domain knowledge and only update the weights of task-related layers;

[0028] Adjust the input / output format and loss function according to the task requirements;

[0029] Through the regularization term λ||θ-θ domain || 2 To limit excessive shifts in model parameters, the fine-tuning objective is to minimize cross-entropy loss.

[0030] The further includes:

[0031] Continuously collect implicit behavioral feedback signals, including click-through rate, dwell time, frequency of subsequent interactions, and swipe and back actions;

[0032] Evaluate the dialogue quality of a multi-turn question-answering system by utilizing implicit behavioral feedback signals;

[0033] Positive samples R extracted based on dialogue quality + And negative samples R-, to achieve model optimization using Triplet Loss or a modified Triplet Loss.

[0034] Another aspect of the present invention provides a power knowledge base question answering system based on an improved RAG, implemented using the aforementioned power knowledge base question answering method based on an improved RAG, the system including a pre-training module and a retrieval module;

[0035] The pre-training module uses a combination of structured and unstructured data to parse and store various content types such as drawings, tables, and text, and injects the domain knowledge from the pre-training stage into the multimodal Transformer structure.

[0036] The retrieval module applies fragment relevance enhancement, pre-retrieval optimization, and post-retrieval optimization to the model computation process of the multimodal Transformer structure to obtain optimized results.

[0037] The beneficial effects of this invention lie in its improved RAG-based question-answering method and system for power knowledge bases. Compared with existing technologies, this invention focuses on the core problems that current Retrieval-Enhanced Generation (RAG) systems urgently need to solve, seeking the optimal balance between data redundancy, retrieval efficiency, and the relevance of generated content. By introducing domain knowledge increments during the pre-training stage, the model can initially grasp the semantic structure and core concepts of the industry. During the fine-tuning stage, the model further improves its capabilities for specific tasks, such as its performance in question-answering dialogues. Finally, by combining reinforcement learning with real user interaction data, the model weights are dynamically optimized, making the generated content more aligned with actual needs. In terms of system architecture design, a hybrid mode compatible with both offline and online environments is adopted, ensuring continuous availability of functions not only in network-constrained environments but also supporting automatic synchronous updates of the knowledge base and model parameters when network conditions permit. This solution effectively improves the system's intelligence, adaptability, and responsiveness, providing an efficient, reliable, and sustainably optimized intelligent solution for specific fields such as the power industry.

[0038] The beneficial effects of the present invention also include:

[0039] Information integration and comprehensiveness: By fusing data from multiple modalities such as text, images, audio, and video, it can comprehensively capture knowledge in the power industry and make up for the limitations of single-modal data.

[0040] Improve retrieval efficiency and accuracy: By preprocessing and extracting features from multimodal data, high-quality indexes can be generated, which can significantly improve the efficiency and accuracy of the retrieval process and meet the needs of rapid response.

[0041] Improved Question-Answer Performance: By learning from domain-specific question-answering datasets, the model can more accurately understand common problems and coping strategies in the power industry. Through deep processing and feature extraction of multimodal data, the model can not only understand the semantics of text but also enhance the accuracy of answers using information such as images, audio, and video.

[0042] Supports compatibility between offline and online functions: Through a hybrid architecture design, it achieves compatibility between offline and online functions, ensuring normal use in environments with limited network access, and automatically synchronizing and updating the knowledge base and model weights when network conditions permit, thereby improving the intelligence and adaptability of the system.

[0043] In summary, the application of these methods can significantly improve the construction quality and system performance of multimodal power knowledge bases, providing the power industry with efficient, reliable, and sustainably optimized intelligent solutions. Attached Figure Description

[0044] Figure 1 A diagram illustrating a question-and-answer method for an electricity knowledge base based on an improved RAG. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of this invention clearer and more accurate, the technical solutions of this invention are described in detail below through several specific embodiments. The embodiments used in this invention are merely illustrative and are not intended to limit the scope of this invention.

[0046] Figure 1 This is a diagram of a question-answering method for an electricity knowledge base based on an improved RAG. (Example:) Figure 1 The first aspect of this invention relates to a question-answering method for an electricity knowledge base based on an improved RAG, the method comprising:

[0047] Step 1: Using a combination of structured and unstructured data, we parse and store various content types such as drawings, tables, and texts, and inject the domain knowledge from the pre-training stage into the multimodal Transformer structure.

[0048] In constructing the knowledge base for the power industry, a combination of structured and unstructured data was adopted to support the parsing and storage of various content types, including drawings, tables, and text. This approach better adapts to the diversity of documents in the power industry and provides a foundation for subsequent data processing and knowledge extraction.

[0049] Preprocessing of multimodal data is a crucial step in index generation, requiring specific processing for different data modalities. Text data is first cleaned, removing special characters, and then segmented, stop words removed, and standardized. Image data is improved for consistency and model robustness through methods such as resizing, normalization, and data augmentation. Audio data is standardized through noise reduction, segmentation, and feature extraction (such as MFCC). Video data is processed by extracting keyframes or decomposing it into image frames to remove irrelevant information and reduce computational complexity.

[0050] To extract effective embedding features from various modalities, deep learning models are employed. Text data uses language models such as BERT to generate semantic embeddings; image data uses convolutional neural networks (such as ResNet) to extract visual features; audio data uses spectrograms or other specific tools to generate audio representations; and video data combines keyframe features with time-series models (such as LSTM) to extract global video dynamic features. This ensures that each modality has a high-dimensional, dense representation, facilitating subsequent processing and analysis.

[0051] To achieve effective alignment of multimodal features, a multimodal pre-trained model (such as CLIP) is employed to map image and text features to a shared vector space, and contrastive learning is used to optimize the similarity between positive and negative samples. Simultaneously, a cross-modal attention mechanism is used to capture implicit associations between different modalities. Furthermore, dimensionality reduction techniques are employed to reduce the dimensionality of feature vectors, thereby lowering storage and computational costs.

[0052] Before storing the data, it is processed, cleaned, and labeled using a Large Language Model (LLM) to increase information density. This improvement is because unstructured data from heterogeneous data sources is often not specifically designed for RAG systems and may have problems such as low information density, noisy data, and information duplication.

[0053] The methods for generating summaries and retrieving these summaries using LLM can significantly improve retrieval efficiency. When dealing with large-scale datasets, two indexes are created: one consisting of summaries and the other of document blocks. The search process consists of two steps: first, relevant documents are filtered out using the summary index, and then a detailed search is performed on these relevant documents.

[0054] Furthermore, a series of hypothetical question-and-answer pairs are generated for each document using a large model, and these generated questions are used as blocks for embedding retrieval. During the retrieval process, the system provides the retrieved questions and their corresponding answers to the LLM, thereby increasing the cosine similarity between the query embeddings and the embeddings of the generated questions, further improving the accuracy and efficiency of retrieval.

[0055] Integrating domain knowledge into model training is a phased, multi-layered optimization process, including domain knowledge injection in the pre-training phase, task adaptation in the fine-tuning phase, and dynamic optimization in the reinforcement learning phase. These three phases progress from generalized knowledge to domain-specific capabilities, forming a systematic knowledge transfer and optimization process.

[0056] During the pre-training phase, domain knowledge increments need to be injected into the pre-trained multimodal Transformer. The multimodal Transformer structure is as follows:

[0057] The pre-trained multimodal Transformer architecture includes an embedding layer, multiple low-level Transformers, a cross-modal fusion layer, and a task header;

[0058] text:

[0059]

[0060] image:

[0061] x p =W projflatten(patch) + e pos (patch)+e mod (image)

[0062] The flatten(patch) function flattens the patch of the image into a one-dimensional vector, W. proj The weight matrix is ​​a learnable linearly varying matrix, epos is the position encoding of the patch, emod is the modality identifier vector, and x is the position of the patch. p This is the final embedding vector.

[0063] Audio:

[0064] x f =W frame spec(f)+e pos (f)+e mod ("audio")

[0065] Among them W frame It is a learnable linearly changing weight matrix, spec(f) refers to the spectral features for frame f, epos is the position code, and emod is the modality identifier vector.

[0066] video:

[0067] x p,t =W vt patch p,t +e pos_spa (p)+e pos_tem (t)+e mod ("video")W vt The p-th patch in frame t p,t The learnable linearly varying weight matrix, e pos_spa (p) is the spatial location encoding of the patch within a frame of an image, e pos_tem (t) represents the time position encoding of frame t.

[0068] In the underlying Transformer, each modality is stacked independently.

[0069] Each block layer includes:

[0070] Multi-head self-attention:

[0071]

[0072] Feed-Forward Network:

[0073] FFN(x) = max(0, xW1+b1)W2+b2

[0074] Residual connectivity + Layer Normalization (LayerNorm).

[0075] The cross-modal fusion layer concatenates the outputs of the last layer of each modality and inputs several layers of cross-modal self-attention (CrossAttention) + FFN to capture the semantic associations between different modalities through cross-modal attention.

[0076] The task header includes:

[0077] Language Modeling Head

[0078]

[0079] Retrieval Head (Retrieval Head)

[0080] z = W r Pool(H), score(q,d) = cos(z) q ,z d )

[0081] Classification Head

[0082] logits = W c h [CLS] +b c

[0083] The training of the pre-trained language model is as follows:

[0084] pre-trained language models Its weighting parameter is θ base Through large-scale public corpora Training is achieved. To incorporate domain knowledge, incremental pre-training aims to leverage domain data. Update the model weights to optimize their parameters to θ. domain .

[0085] The objective function is to minimize the language modeling loss.

[0086]

[0087] Where x and y represent the input and target output, respectively, and P(y|x;θ) is the conditional probability distribution based on the current parameter estimates.

[0088] Through Continue training This allows the model to retain general knowledge while possessing domain-specific semantic modeling capabilities. The focus at this stage is on adjusting parameters such as W in the embedding layer and the context modeling layer. proj W vt This makes the model's representation of domain-specific vocabulary and sentence structures more accurate, while preventing catastrophic forgetting.

[0089] The fine-tuning process includes the following steps:

[0090] Freeze some domain knowledge weights (such as W in the embedding layer or lower-level Transformer) proj W vt Only update the weights of task-related layers (such as the task header layer θ). head This preserves the domain characteristics;

[0091] Adjust the input / output format and loss function according to the task requirements (e.g., sequence labeling or question answering) to ensure the fine-tuning process is highly targeted. If the data size is limited, regularization terms λ||θ-θ can be used. domain || 2 Limit excessive shifts in model parameters to avoid overfitting. Through fine-tuning, the model is further adapted from general domain knowledge to specific tasks, making its output highly relevant to task requirements.

[0092] In practical multi-turn question-answering systems, users rarely use explicit rating to label the quality of the model-generated results. However, the system can continuously collect a large number of implicit behavioral feedback signals, such as click-through rate, dwell time, frequency of subsequent interactions, swipe and back operations, etc. These behaviors can serve as indirect reflections of the quality of the dialogue.

[0093] During the fine-tuning phase, the model is transferred from domain-general knowledge to the specific task. Assume the task dataset is... The goal of a model is to optimize the performance metrics for a specific task. Taking classification as an example, the fine-tuning objective is to minimize the cross-entropy loss:

[0094]

[0095] Where c represents the target category of input x, θ task These are the fine-tuned model parameters.

[0096] Through fine-tuning, the model is further adapted from general domain knowledge to specific tasks, making its output highly relevant to task requirements.

[0097] This invention proposes a novel optimization path that does not rely on policy gradient methods—reinforcement-based contrastive learning. By automatically constructing contrastive samples from these implicit feedbacks, it achieves reinforcement optimization of the language model while maintaining the stability and efficiency of the training process.

[0098] This method is based on the query context Q of real users and the corresponding responses generated by multiple models, and constructs a ranking score based on behavioral feedback signals. Responses with stronger user preferences are extracted as positive samples R. + The relatively inferior samples are used as negative samples R-, and (Q,R) is constructed. + The process begins by generating triples h_Q, h_+, and h_-. Next, a high-level semantic representation of Q and each response (such as [CLS] vectors or mean-pooled sentence vectors) is extracted using a language model and fed into a projection layer to generate embeddings h_Q, h_+, and h_-. Based on these embedding vectors, a contrastive learning objective is introduced, optimized using Triplet Loss or a modified Triplet Loss approach, with the goal of making Q in the semantic space closer to R. + By moving away from R-, the model is implicitly guided to learn a more accurate alignment with user preferences. Its key loss function is as follows:

[0099]

[0100] Or a softened variant:

[0101]

[0102] Where sim is the cosine similarity, m is the minimum semantic distance threshold between positive and negative samples, and τ is the temperature coefficient. Unlike methods such as policy gradient, this method does not require building an explicit reward model or performing a high-variance sampling process. Instead, it achieves optimization through an end-to-end differentiable structure, resulting in higher stability.

[0103] To improve generalization ability, the bottom embedding layer and some lower-level Transformer layers can be frozen, with only the mid-to-high-level Transformer modules and the projection head being updated, thus preserving the language understanding ability of the pre-trained language model. In engineering implementation, user behavior data is extracted from production system logs at regular intervals (e.g., daily / weekly) to build a feedback scoring model (e.g., a weighted linear model or a lightweight regression network), automatically generating training triples. The projection head is typically a 1-2 layer MLP, and the output vector is normalized before being used for similarity calculation. By continuously acquiring implicit feedback and transforming it into contrast optimization signals, the language model can gradually align with users' true preferences, achieving the goal of reinforcement learning, while possessing strong practicality and scalability.

[0104] In the pre-training phase, domain knowledge is injected through incremental optimization to establish a domain semantic foundation for the model. In the fine-tuning phase, task data is used to optimize for specific application scenarios, achieving task-level adaptation. In the reinforcement learning phase, user feedback is used to further dynamically optimize the generated content, making the model more practical and adaptable in real-world applications. This formal modeling and layer-by-layer optimization process provides theoretical support and practical guidance for the efficient application of large language models in domain-specific scenarios.

[0105] Step 2: Apply the sharding relevance enhancement, pre-retrieval optimization, and post-retrieval optimization to the model operation process of the multimodal Transformer structure to obtain the optimization results.

[0106] Enhanced fragment relevance in current embedding retrieval-based question answering systems, while capable of efficiently matching document fragments relevant to user queries through semantic matching, is often hampered by the "fragment fragmentation" problem: the retrieval system only returns a few fragments most similar to the query itself, ignoring the contextual relationships of these matched fragments within the original document, as well as potential connections between these semantically closely related but not directly matched fragments. This results in the generative model lacking a global context when constructing answers, leading to problems such as incomplete information, logical fragmentation, or irrelevant answers.

[0107] To address this issue, a storage and retrieval mechanism called "fragment relevance enhancement" is introduced. During document preprocessing, in addition to calculating the semantic vector for each fragment individually, a "semantic adjacency graph" or "contextual association graph" is explicitly built between fragments. In this graph structure, each node represents a fragment, and each edge represents the semantic relevance or structural connection between that fragment and other fragments. Edge weights can be weighted and fused based on multiple factors such as semantic similarity (e.g., cosine similarity), positional distance, and syntactic dependencies. For example, two adjacent fragments that are originally consecutive in the document, or whose content exhibits syntactic subject-predicate relationships or coreference, can be considered to have a high "intra-text relevance weight."

[0108] In the actual retrieval process, the system first uses conventional vector retrieval to find the top-k segments most similar to the query. Then, using these segments as "anchors," it performs k-hop expansion on the segmented relevance graph, selecting adjacent segments with high relevance to the anchor segments (e.g., through threshold filtering or the largest of the top N edge weights) to construct a local context subgraph. The final context set returned to the large model generator is no longer a patchwork of isolated segments, but a "context cluster" that is more semantically coherent and has a wider coverage, thus significantly improving the accuracy and completeness of question answering generation.

[0109] By storing and retrieving the fragment relevance between queries and knowledge fragments, the semantic asymmetry problem is mitigated. When a query fails to directly find the answer, the system retrieves content that might contain the answer from "semantic neighbor" fragments (i.e., fragments semantically similar to the query). A graph structure is used to model the relationships between fragments, thereby enhancing the understanding of cross-paragraph and cross-sentence logic. In multi-turn question-answering scenarios, by maintaining the graph structure between fragments, the dependency paths of historical context are explicitly recorded. This allows the model to locate potential logical extension points based on existing context, thus better supporting follow-up question processing. The storage and retrieval mechanism based on fragment relevance provides structured semantic support for the generation stage.

[0110] Pre-retrieval optimization employs Hypothetical Document Embedding (HDE) to generate hypothetical answers to replace the original query, addressing the asymmetry between the query and document embedding and improving semantic similarity. Hypothetical document embedding can be expressed using a similarity formula:

[0111] Use cosine similarity to calculate the matching degree between hypothetical document embeddings and actual document embeddings: in It is the embedding vector of the hypothetical document. This is the embedding vector of the actual document. The higher the similarity, the better the match between the hypothetical document and the search target.

[0112] Post-retrieval optimization suggestion compression can incorporate the formula for attention weights:

[0113] Assuming the attention matrix of LLM is A, representing the degree of attention each token pays to other tokens, the weights of important tokens after compression can be expressed as: Among them W i A is the importance weight of the i-th token. ij It is the attention weight of the i-th token to the j-th token.

[0114] Reverse decompression can be measured based on the information entropy formula:

[0115] Assuming the semantic distribution after compression is p(x) and the distribution after decompression is q(x), the similarity between the two can be calculated using cross-entropy: H(p,q)=-∑ x p(x)logq(x) minimizes the cross-entropy, ensuring that the distribution after inverse decompression is as close as possible to the original distribution.

[0116] The context relevance scoring mechanism can be modeled using Bayes' theorem:

[0117] Given a context C and a retrieval content R, the context relevance P(C|R) can be expressed as: where P(R|C) is the probability of the retrieval content in the current context, P(C) is the prior probability of the context, and P(R) is the overall probability of the retrieval content.

[0118] The unknown response mechanism is based on a confidence threshold. If the confidence of the retrieval result is S, when S < Threshold, the system returns an "unknown" response. For example, the confidence calculation can be based on embedding similarity or the output probability of a classification model.

[0119] Engineering application optimization includes:

[0120] 1) Hybrid mode

[0121] In the system architecture design, a hybrid mode that is compatible with offline and online is adopted to ensure the continuous availability of functions, support the automatic synchronization and update of the knowledge base and model parameters.

[0122] (a) Offline function design

[0123] The offline mode needs to ensure that the question-and-answer system can operate independently in a network-free environment, while providing sufficient knowledge coverage and service quality.

[0124] Local knowledge base construction: In the offline scenario, the local knowledge base is the core resource.

[0125] Lightweight processing: Through knowledge compression technology, the huge knowledge base is optimized into a small version for local storage, such as only retaining key technical documents, common operation guides, and typical fault cases.

[0126] Hierarchical storage: For different devices or scenarios, the knowledge base is stored hierarchically according to priority to ensure that frequently used knowledge is available at any time, and low-frequency knowledge is supplemented by calling remote resources.

[0127] Embedded model deployment: Adopt lightweight models (such as distilled small language models or quantized 4-bit models) to ensure that the model can run on resource-constrained devices (such as industrial-grade tablets or embedded devices).

[0128] Basic question-and-answer function: The offline question-and-answer system supports basic retrieval and generation capabilities, covering common question answers, device operation instructions, and safety specifications.

[0129] (b) Online function design

[0130] The online mode is mainly used for the dynamic update of the knowledge base and the cloud computing of complex problems.

[0131] Real-time knowledge updates: Once the network connection is restored, the system can synchronize with the central server to obtain the latest knowledge base content, including new cases, updated industry standards, or troubleshooting methods.

[0132] Cloud computing support: For problems that require higher computing power (such as complex multimodal question answering or reasoning tasks), the system can send the request to the cloud, where a more powerful server model will perform the calculation and return the result.

[0133] (c) Offline and online collaboration mechanism

[0134] To achieve seamless integration of offline and online functions, the system requires an efficient collaboration mechanism:

[0135] Status switching: The system automatically switches between offline and online modes based on network status, avoiding manual intervention. For example, when the system detects that the network has recovered, it can proactively synchronize the knowledge base.

[0136] Local caching strategy: In online mode, the system can pre-cache the content recently queried by users and related information to ensure that relatively complete services can still be provided when the network is interrupted.

[0137] Conflict resolution: When a user updates local data or the knowledge base in offline mode (e.g., marking certain content as high priority), the system must use a version control mechanism to merge the conflicts in online mode to ensure data consistency.

[0138] The application scenarios are as follows:

[0139] Equipment Inspection: Inspection personnel can obtain equipment operation specifications or common fault handling methods in offline mode. After returning to a network environment, the system will automatically upload the inspection data and synchronize the knowledge base.

[0140] Emergency Command: In emergency situations, such as power equipment failures or power grid outages caused by natural disasters, the offline mode ensures that command personnel can quickly access emergency plans and procedures.

[0141] Operation and maintenance in remote areas: For remote areas with insufficient network coverage, the offline Q&A function significantly improves the convenience of operation and maintenance work.

[0142] 2) User feedback optimization

[0143] (a) Collection of user feedback

[0144] To effectively utilize user feedback, the system needs to design multiple data collection mechanisms:

[0145] Explicit feedback: After each response, the system can prompt the user to rate the answer (e.g., satisfied, dissatisfied) or provide suggestions. This direct feedback method helps the model quickly locate problems.

[0146] Implicit feedback: By analyzing user behavior data (such as the number of question restates, query frequency, and result click-through rate), we can infer user satisfaction with the answer. For example, if a user frequently modifies their question after receiving an answer, it may indicate that the generated result does not meet their expectations.

[0147] Error correction feedback: When users find that their answers are incorrect, they can manually mark the correct answer, which the system will record for future optimization.

[0148] (b) Dynamic adjustment mechanism

[0149] The collected user feedback can be used to dynamically optimize system performance in the following ways:

[0150] Model weight updates: For answers repeatedly marked as incorrect or unsatisfactory in user feedback, the reasons for the model's errors are analyzed in conjunction with the context and knowledge points, and the model weights are optimized through a fine-tuning mechanism. For example, if the terminology of a certain type of equipment is repeatedly misunderstood, the question-answering ability in that domain can be strengthened first.

[0151] Knowledge base revision: Based on user feedback on error correction, the knowledge base content is regularly updated and corrected to ensure the accuracy of the answers.

[0152] Priority adjustment: For frequently queried questions, the system can dynamically increase the priority of relevant knowledge to optimize retrieval efficiency.

[0153] (c) Continuous process optimization

[0154] Data-driven iterative optimization: User feedback is used as part of the training data to continuously expand the capabilities of the question-answering model. For example, after multiple interactions and feedback optimizations, some complex questions can generate more complete answers.

[0155] Personalized services: By analyzing user habits and building user profiles, we can optimize the model's adaptability to different user groups. For example, beginners may need more detailed operating instructions, while professional engineers prefer concise technical answers.

[0156] Feedback system integration: Integrate the feedback module into the user interface to ensure that users can easily provide feedback and record relevant data for analysis.

[0157] Feedback analysis model: This model uses machine learning methods to classify and analyze the collected feedback data, and extract the key points that need to be optimized.

[0158] Automated optimization tools: Develop an automated toolchain that forms a closed-loop optimization process, from data collection and model adjustment to knowledge base updates.

[0159] A second aspect of this invention relates to a power knowledge base question-answering system based on an improved RAG (Research Aggregator). The system is implemented using the method described in the first aspect of this invention. The system includes a pre-training module and a retrieval module. The pre-training module uses a combination of structured and unstructured data to parse and store various content types, such as drawings, tables, and text, and injects the domain knowledge from the pre-training stage into a multimodal Transformer structure. The retrieval module applies fragment relevance enhancement, pre-retrieval optimization, and post-retrieval optimization to the model computation process of the multimodal Transformer structure to obtain optimized results.

[0160] A third aspect of the present invention relates to a terminal, including a processor and a storage medium; the storage medium is used to store instructions; the processor is used to operate according to the instructions to execute the steps of the method described in the first aspect of the present invention.

[0161] A fourth aspect of the present invention relates to a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of the method described in the first aspect of the present invention.

[0162] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions of the present invention still include content that can be modified or equivalently replaced in the specific implementation of the present invention. Any modifications or equivalent replacements that do not depart from the spirit and scope of the present invention should be covered within the protection scope of the claims of the present invention.

Claims

1. A question-answering method for an electricity knowledge base based on an improved RAG, characterized in that, The method includes the following steps: Step 1: Using a combination of structured and unstructured data, we parse and store various content types such as drawings, tables, and texts, and inject the domain knowledge from the pre-training stage into the multimodal Transformer structure. Step 2: Apply the sharding relevance enhancement, pre-retrieval optimization, and post-retrieval optimization to the model operation process of the multimodal Transformer structure to obtain the optimization results.

2. The power knowledge base question-answering method based on improved RAG according to claim 1, characterized in that: The method employs a combination of structured and unstructured data to parse and store various content types, including drawings, tables, and text. Domain knowledge from the pre-training phase is injected into the multimodal Transformer structure, including: Multimodal data is preprocessed, semantic embeddings are generated using the BERT language model for the preprocessed text data, visual features are extracted from the preprocessed image data using a convolutional neural network, audio representations are generated from the preprocessed audio data using spectrograms, and global video dynamic features are extracted from the preprocessed video data by combining keyframe features with a time series model. A multimodal pre-trained model is used to map image and text features to a shared vector space, and the similarity between positive and negative samples is optimized through contrastive learning. Data is processed, cleaned, and labeled using a large language model.

3. The power knowledge base question-answering method based on improved RAG according to claim 2, characterized in that: Preprocessing of multimodal data includes: Clean the text data, remove special characters, perform word segmentation, remove stop words, and standardize the data; Image data is resized, normalized, and augmented. Noise reduction, segmentation, and MFCC feature extraction are performed on the audio data; Extract keyframes or decompose video data into image frames.

4. The power knowledge base question-answering method based on improved RAG according to claim 3, characterized in that: Data is processed, cleaned, and labeled using a large language model, including: The large language model adopts a multimodal Transformer structure, including an embedding layer, multiple underlying Transformers, a cross-modal fusion layer, and a task header.

5. A question-answering method for an electrical knowledge base based on an improved RAG, as described in any one of claims 2-4, characterized in that: The training of the large language model is as follows: Through large-scale public corpora The weight parameters θ of the large language model are obtained through training. base , Utilizing domain data Update the model weights to optimize the weight parameters of the large language model to θ. domain ; The objective function is to minimize the language modeling loss.

6. A question-answering method for an electrical knowledge base based on an improved RAG, as described in any one of claims 2-4, characterized in that: After training, the large language model undergoes fine-tuning, which includes the following steps: Freeze the weights of some domain knowledge and only update the weights of task-related layers; Adjust the input / output format and loss function according to the task requirements; By regularization term λ||θ-θ domain || 2 To limit excessive shifts in model parameters, the fine-tuning objective is to minimize cross-entropy loss.

7. A question-answering method for an electrical knowledge base based on an improved RAG, as described in any one of claims 6, characterized in that, Further includes: Continuously collect implicit behavioral feedback signals, including click-through rate, dwell time, frequency of subsequent interactions, and swipe and back actions; Evaluate the dialogue quality of a multi-turn question-answering system by utilizing implicit behavioral feedback signals; Positive samples R are extracted based on dialogue quality. + And negative samples R-, to achieve model optimization using Triplet Loss or a modified Triplet Loss.

8. A power knowledge base question-answering system based on an improved RAG, implemented using the power knowledge base question-answering method based on an improved RAG as described in any one of claims 1-7, characterized in that, The system includes a pre-training module and a retrieval module; The pre-training module uses a combination of structured and unstructured data to parse and store various content types such as drawings, tables, and text, and injects the domain knowledge from the pre-training stage into the multimodal Transformer structure. The retrieval module applies fragment relevance enhancement, pre-retrieval optimization, and post-retrieval optimization to the model computation process of the multimodal Transformer structure to obtain optimized results.

9. A terminal, comprising a processor and a storage medium; characterized in that: The storage medium is used to store instructions; The processor is configured to operate according to the instructions to perform the steps of the method according to any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the method according to any one of claims 1-7.

Images