A power knowledge retrieval system based on a large language model

Abstract

The application discloses a power knowledge retrieval system based on a large language model, and relates to the technical field of data processing.The system comprises a training data collection module, which is used to collect professional data in the power field and cross-field general data; a model training module, which is used to perform internal-external balance training on a hierarchical memory network architecture based on the training data, and to construct a power knowledge retrieval large model; a retrieval instruction recognition module, which is used to receive retrieval requirements of a user end, and to analyze and generate disassembled user requirements; and a power knowledge retrieval module, which is used to receive the disassembled user requirements, and to input the power knowledge retrieval large model to perform power knowledge retrieval.The application solves the technical problem that the existing power knowledge retrieval system has low efficiency and poor accuracy when processing complex documents and cross-field knowledge, and achieves the technical effect of improving the power knowledge retrieval efficiency and accuracy through the hierarchical memory network architecture based on the large language model and the internal-external balance training.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and specifically to a power knowledge retrieval system based on a large language model. Background Technology

[0002] With the rapid development of the power industry, the accumulation and complexity of power knowledge are constantly increasing, placing higher demands on power knowledge retrieval systems. Existing power knowledge retrieval systems typically rely on keyword-based matching or rule-driven methods. These methods often suffer from low retrieval efficiency and poor accuracy when faced with complex technical documents, real-time data, and cross-domain knowledge in the power field, especially when processing long technical documents, where key information is easily lost. Furthermore, the intersection of the power field with other fields is becoming increasingly frequent, and existing systems have significant shortcomings in integrating cross-domain knowledge, limiting the accuracy and practicality of retrieval results. Summary of the Invention

[0003] This application provides a power knowledge retrieval system based on a large language model, which addresses the technical problems of low efficiency and poor accuracy in existing power knowledge retrieval systems when processing complex documents and cross-domain knowledge.

[0004] This application provides a power knowledge retrieval system based on a large language model. The system includes: a training data acquisition module for collecting a dedicated training dataset in the power field through big data, the dedicated training dataset including professional data in the power field and general data across fields; a model training module for performing internal and external balance training on a preset hierarchical memory network architecture based on the professional data in the power field and general data across fields to construct a large power knowledge retrieval model; a retrieval instruction recognition module for receiving user retrieval requests from the knowledge retrieval user terminal, performing request identification and analysis, and generating a breakdown of user requests; and a power knowledge retrieval module, which embeds the large power knowledge retrieval model, for receiving the breakdown of user requests, inputting them into the large power knowledge retrieval model to perform power knowledge retrieval, and outputting knowledge retrieval results.

[0005] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0006] This application provides a power knowledge retrieval system based on a large language model, relating to the field of data processing technology. It collects professional data in the power field and general cross-domain data through a training data acquisition module, supports internal and external balance training in a model training module to construct a large power knowledge retrieval model, and analyzes user retrieval needs through a retrieval command recognition module. The power knowledge retrieval module receives user needs, inputs them into the large retrieval model for knowledge retrieval, and outputs accurate retrieval results. This solves the technical problems of low efficiency and poor accuracy in existing power knowledge retrieval systems when processing complex documents and cross-domain knowledge. It achieves the technical effect of improving the efficiency and accuracy of power knowledge retrieval through a hierarchical memory network architecture based on a large language model and internal and external balance training. Attached Figure Description

[0007] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0008] Figure 1 A schematic diagram of the structure of a power knowledge retrieval system based on a large language model, provided for an embodiment of this application;

[0009] Figure 2 This is a schematic diagram of the hierarchical memory network architecture in a power knowledge retrieval system based on a large language model, provided as an embodiment of this application.

[0010] Figure labeling: Training data acquisition module 10, model training module 20, retrieval command recognition module 30, power knowledge retrieval module 40, domain expert feedback module 50. Detailed Implementation

[0011] This application provides a power knowledge retrieval system based on a large language model, which addresses the technical problems of low efficiency and poor accuracy in existing power knowledge retrieval systems when processing complex documents and cross-domain knowledge.

[0012] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0013] It should be noted that the terms "first," "second," etc., in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or modules not explicitly listed or inherent to such processes, methods, products, or devices.

[0014] Example 1, as Figure 1 As shown, this application provides a power knowledge retrieval system based on a large language model, the system comprising:

[0015] The training data acquisition module 10 is used to collect a dedicated training dataset for the power industry through big data. The dedicated training dataset includes professional data in the power industry and general data across different fields.

[0016] Specifically, the core function of the training data acquisition module 10 in this application is to collect a specialized training dataset in the power industry through big data technology. This dataset includes professional data in the power industry and general data across different fields.

[0017] First, to ensure the breadth and comprehensiveness of the data, the training data acquisition module 10 utilizes big data technologies (such as distributed data acquisition and processing frameworks) to automatically collect data from various power-related resources. These resources include, but are not limited to, power equipment operation data, maintenance records, technical reports, standard documents, academic papers, and various sensor data. Power-related specialized data refers to various types of data directly related to the power system, such as power load, generation efficiency, distribution network status, and equipment fault records. This data typically comes from various stages of power production, transmission, distribution, and consumption, and forms the basis for power system management and optimization.

[0018] To enhance the model's versatility and adaptability, the training data acquisition module 10 also collects cross-domain general data. Cross-domain data refers to domain data that is indirectly related to the power system but has reference value in a certain context, such as meteorological data (weather conditions affecting power load), economic indicators (such as the relationship between energy price fluctuations and demand changes), and social behavior data (such as changes in consumption patterns). By combining this external data with power-related professional data, the model can better handle complex multi-factor problems, thereby improving its accuracy and generalization ability in practical applications.

[0019] After data collection, the module preprocesses and integrates the data, removing redundancy and irrelevant information through cleaning, standardization, and denoising techniques to ensure data quality. Feature engineering is also performed during this step, including time-series analysis of sensor data, fault detection, and prediction techniques to extract highly informative features. With the support of these technologies, the collected power system data not only reflects the operating status of the power system but also uncovers potential patterns and trends, providing a solid data foundation for subsequent model training.

[0020] Finally, the multi-source data needs to be fused and modeled. Data from different sources (such as power data, meteorological data, and socioeconomic data) are fused according to certain rules to generate a comprehensive dataset suitable for training large language models. In this process, the balance and representativeness of the dataset are crucial. Therefore, the module selects and collects data based on the needs and future development trends of the power industry, ensuring that the collected data covers all important areas of the power system and avoids data bias towards a particular direction or field, thus providing a rich and high-value data source for the power knowledge retrieval system.

[0021] The model training module 20 is used to perform internal and external balance training on a preset hierarchical memory network architecture based on the professional data in the power field and cross-domain general data, so as to build a large power knowledge retrieval model.

[0022] Furthermore, such as Figure 2 As shown, the model training module 20 is also used to perform the following steps:

[0023] P21: Design a hierarchical memory network architecture, which includes a document classification layer, a long text segmentation layer, and a document extraction layer. The document extraction layer adopts a dual extraction verification mechanism, including a feature extraction unit and an information verification unit. P22: Based on the professional data in the power field and cross-domain general data, perform internal and external balance training on the document classification layer, long text segmentation layer, and document extraction layer of the hierarchical memory network architecture to generate the large-scale power knowledge retrieval model.

[0024] It should be understood that the model training module 20 of this application is responsible for constructing a large-scale power knowledge retrieval model through internal and external balancing training on professional data in the power field and general cross-domain data. In this module, the first step is to design a hierarchical memory network architecture adapted to the needs of power knowledge retrieval. This architecture includes a document classification layer, a long text segmentation layer, and a document extraction layer. During the design process, the main task of the document classification layer is to perform preliminary classification based on the input text data, dividing power field documents into different categories, such as power load, equipment failure, and operation logs. Classified data facilitates subsequent processing, ensuring that the model can more efficiently analyze different categories of data in subsequent steps.

[0025] The purpose of the long text segmentation layer is to cut and break down the input long text, avoiding information loss due to excessive input length. This layer breaks the document down into chapters or paragraphs, allowing the model to analyze each smaller part in depth, thus improving the accuracy and efficiency of long text processing. The segmented data provides the model with a more refined contextual understanding, especially in technical literature in the power industry, where long reports and standard documents are common; refined processing can effectively reduce information omissions.

[0026] The document extraction layer's task is to extract key information from segmented documents, including the operating status of power equipment, load forecasting, and fault detection. This layer employs a dual extraction and verification mechanism to improve extraction accuracy. First, the feature extraction unit identifies key features from the text, such as equipment operating parameters, fault types, or technical details. Next, the information verification unit verifies these extracted features to ensure the accuracy and consistency of the information. This verification process can be performed by comparing the extracted information with an existing power industry knowledge base to confirm its reasonableness, thereby avoiding the impact of erroneous data on model training.

[0027] After completing the design of the hierarchical memory network architecture, the model training module will next undergo internal and external balancing training. In this stage, specialized data from the power sector and cross-domain general data are introduced into the training process. Specialized data from the power sector includes power load data, power equipment operation logs, and technical reports. This data enables the model to deeply learn the core technologies of power system operation, equipment operation, and fault diagnosis. Cross-domain general data, including meteorological data, economic data, and social behavioral data, are also incorporated into the training process. The aim is to allow the model to learn from external factors, such as the impact of weather changes on power load and the impact of energy price fluctuations on electricity demand. In this way, the model can not only focus on core knowledge in the power sector but also effectively absorb knowledge from other fields, enhancing its ability to handle complex situations and interdisciplinary problems.

[0028] Furthermore, the internal and external balance training dynamically adjusts the weights of the loss function to balance the training weights of power industry data and cross-domain data. Specifically, during training, the model adjusts the training process based on the contributions of different data sources, ensuring that while deeply learning the power industry domain, it does not neglect knowledge from other domains, thus making the training process more adaptable and generalizable. Internal training primarily targets power industry data, using supervised learning to train each layer of the model using labeled power industry data. For example, in the document classification layer, the model learns how to categorize documents into the correct topic categories based on their titles and summaries; in the long document segmentation layer, the model learns how to reasonably break down long documents into chapters, paragraphs, and sentences; and in the document extraction layer, the model learns how to accurately extract key information from text units, and uses an information verification unit to ensure the accuracy and consistency of the extracted information.

[0029] External training utilizes cross-domain general data, employing unsupervised or semi-supervised learning methods to enhance the model's generalization ability and flexibility. For example, in the document classification layer, the model learns how to handle cross-domain documents related to the power industry; in the long document segmentation layer, the model learns how to handle structural differences between documents from different domains; and in the document extraction layer, the model learns how to extract general information from cross-domain documents. Through balanced internal and external training, the model can fully leverage its domain expertise while flexibly addressing general cross-domain problems when dealing with power industry issues.

[0030] Ultimately, through this series of training exercises, the model training module successfully generated a large-scale power knowledge retrieval model. This model can not only accurately understand the professional terminology and technical details in the power field, but also effectively process knowledge from other fields, becoming the core driving force of the power knowledge retrieval system.

[0031] Furthermore, during the internal and external balance training, the model training module 20 is also used to perform the following steps:

[0032] P22-1: Perform dataset balancing on the power industry-specific data and cross-domain general data, and define the training weights for each data type; P22-2: Based on the training weights, perform classification training using the power industry-specific data and cross-domain general data to generate a document classification layer; P22-3: Extract long power technology documents and perform multi-level encoding training to generate a long document segmentation layer; P22-4: Use power industry-specific data for feature extraction training, and combine it with cross-domain general data for parallel information verification training to generate a document extraction layer.

[0033] Optionally, during the internal and external balance training, the model training module 20 further optimizes the training process to ensure that specialized data in the power sector and general cross-domain data can be reasonably processed and integrated in the model.

[0034] First, a dataset balancing process is performed on specialized power sector data and cross-domain general data to ensure that the two types of data receive reasonable weight allocation during training. Specifically, training weights for each type of data are defined based on expert experience or industry data experience. This ensures that specialized power sector data can fully train the model's power knowledge, while cross-domain general data helps improve the model's generalization ability and avoids overfitting to a specific domain.

[0035] Building upon this foundation, predefined training weights are used to classify and train power-related specialized data and cross-domain general data. The generation process of the document classification layer relies on the guidance of these weights. By classifying the input data, the model can automatically identify different categories in power-related documents, such as power equipment, load dispatching, and fault detection. In this process, the model learns how to categorize power-related documents into different topic categories based on information such as document titles, abstracts, and keywords. For example, documents on "power system stability analysis" are classified into one category, while documents on "smart grid technology applications" are classified into another. This classification helps subsequent layers process and extract information more accurately, improving the overall performance of the model. Simultaneously, by combining training with cross-domain general data, the model can better understand the commonalities and differences between documents from different domains, further enhancing its generalization ability.

[0036] Next, multi-level encoding training is performed on long power technology documents to generate long-text segmentation layers. Long power technology documents are typically lengthy and complex, containing a large amount of technical details, formula derivations, and experimental data. Through multi-level encoding training, the model can decompose long documents into smaller text units, such as chapters, paragraphs, and sentences, and encode and aggregate text units at each level. This process not only helps the model better understand and extract key information from long documents but also avoids information loss or misunderstanding when processing long texts. For example, when processing a long paper on power system stability analysis, the model can first encode each chapter to extract the core content of each chapter; then further encode the paragraphs within each chapter to extract key information from the paragraphs; and finally encode the sentences to capture sentence-level details. Through this multi-level encoding and aggregation process, the model can better grasp the overall structure and key information of long documents, thereby optimizing the contextual understanding and information extraction capabilities of long documents.

[0037] Finally, in generating the document extraction layer, feature extraction training is performed using specialized data from the power industry, and parallel information verification training is conducted using cross-domain general data. The document extraction layer is a crucial part of the hierarchical memory network architecture, employing a dual extraction verification mechanism, including a feature extraction unit and an information verification unit. The feature extraction unit is responsible for extracting key information from text units, such as technical terms, parameters, and formulas. For example, when extracting "methods for diagnosing power equipment faults," the feature extraction unit identifies the names, principles, and steps of various fault diagnosis techniques. The information verification unit verifies and corrects the extracted information, ensuring its accuracy and consistency. In this process, the model not only utilizes specialized data from the power industry for feature extraction training to ensure accurate extraction of key information from the power industry, but also combines cross-domain general data for parallel information verification training. By comparing information from multiple sources and checking the logical relationships between information, the extracted information is verified and corrected. For example, the model can check whether the extracted fault diagnosis techniques are consistent with known fault diagnosis theories and practices, and whether they are consistent across different literature, thereby improving the quality of information extraction.

[0038] Through these steps, the model training module 20 can effectively perform internal and external balancing training on the hierarchical memory network architecture, generating a large-scale power knowledge retrieval model with efficient long text understanding and information extraction capabilities. When handling complex knowledge retrieval tasks in the power field, this model can accurately understand power-related terminology and concepts, and flexibly address general cross-domain issues.

[0039] Furthermore, when training the document extraction layer, the model training module 20 is also used to perform the following steps:

[0040] P22-41: Using professional data in the power field, extract long text hierarchical coding samples and short text coding samples, and conduct long text feature extraction training and short text feature extraction training respectively to construct feature extraction units; P22-42: Using cross-domain general data, extract cross-domain general feature samples, conduct information verification training, and construct information verification units.

[0041] In one possible embodiment of this application, when training the document extraction layer, the model training module 20 further refines the training process of feature extraction and information verification to ensure that key information can be extracted efficiently and accurately from both long and short documents, while effectively verifying the extracted information.

[0042] First, we extract hierarchical coding samples of long and short documents using specialized data from the power industry. Long-text hierarchical coding samples typically originate from technical reports, industry standards, and equipment manuals in the power sector. These documents usually contain complex contextual information and are quite lengthy. The model needs to be trained on these long documents using hierarchical coding techniques to ensure it can understand each level (e.g., chapters, paragraphs) and extract key information from each level. Short-text coding samples mainly come from shorter texts such as power equipment fault reports and load dispatch records. These documents typically focus more on describing specific problems, containing less information but still possessing significant value.

[0043] By training the model to extract features from long and short documents separately, the model can be optimized for different document types. Long document feature extraction training emphasizes contextual understanding and hierarchical information extraction, while short document feature extraction training focuses on quickly and accurately extracting key data from concise documents. During training, the model utilizes professional data from the power industry to learn how to extract important features such as power equipment status and system operating parameters from both long and short documents. A feature extraction unit was constructed, capable of handling various information types within power documents, ensuring the rapid and accurate extraction of key information in subsequent information extraction processes.

[0044] Next, cross-domain general-purpose data is used to extract cross-domain general-purpose feature samples for information verification training. Cross-domain general-purpose data typically includes data from meteorology, economics, and other fields, such as temperature, humidity, and energy price fluctuations. This data may have indirect impacts on the scheduling and operation of power systems. By incorporating this data into model training, the model can learn how to handle cross-domain factors related to power systems, thereby improving its ability to solve complex problems. For example, by analyzing mathematical formulas and physical principles in cross-domain general-purpose data, the model can better verify the correctness of relevant technical descriptions in power-related documents. During information verification training, the model learns how to compare information from different sources, check the logical relationships and consistency between information, and thus construct an information verification unit that can efficiently verify the accuracy of information. For example, after the feature extraction unit extracts key information from "methods for diagnosing power equipment faults," the information verification unit can use relevant knowledge from cross-domain general-purpose data to verify whether this information is consistent with known fault diagnosis theories and practices, and whether it is consistent across different documents, thereby improving the quality and reliability of information extraction.

[0045] Through the above training steps, not only is the feature extraction capability of professional data in the power field enhanced, but the accuracy and robustness of the information verification unit are also improved through cross-domain data verification training.

[0046] Furthermore, in the model training module 20, the hierarchical memory network architecture also includes a self-attention mechanism, which is used to perform weighted aggregation of information at different levels in the long text segmentation layer and the document extraction layer.

[0047] Optionally, in the model training module P20, the further design of the hierarchical memory network architecture includes a self-attention mechanism, primarily used for information processing and weighted aggregation in the long text segmentation layer and document extraction layer. Self-attention is an important technique in deep learning; its function is to automatically assign a weight to each element by calculating the correlation between elements in the input sequence, thus enabling the model to more flexibly focus on important parts of the input data. In the task of electricity knowledge retrieval, understanding long texts and complex documents requires more efficient information aggregation and context modeling capabilities, and the self-attention mechanism is designed precisely for this purpose.

[0048] In the long-text segmentation layer, the self-attention mechanism weights and aggregates the multi-level encoding results of long power technology documents, enabling the model to more flexibly focus on the importance of different parts of the document. For example, in a long paper on power system stability analysis, some chapters may contain key theoretical derivations and experimental results, while other parts may be relatively less important. The self-attention mechanism can automatically learn the importance weights of these different parts, thus more accurately preserving key information and avoiding the loss of important details when segmenting long documents. This weighted aggregation method not only improves the model's understanding of long text structures but also enhances its adaptability and flexibility in handling complex power documents.

[0049] In the document extraction layer, the self-attention mechanism plays a more prominent role. After the feature extraction unit extracts key information from the text unit, the information verification unit needs to verify and correct this information. The self-attention mechanism here performs weighted aggregation of the extracted information, enabling the model to focus more on information highly relevant to both power industry expertise and cross-domain general knowledge. For example, when verifying information extracted from "power equipment fault diagnosis methods," the self-attention mechanism helps the model identify which information is highly consistent with known fault diagnosis theories and which information may require further verification or correction. In this way, the self-attention mechanism not only improves the accuracy of information verification but also enhances the overall performance of the model when handling power knowledge retrieval tasks.

[0050] By introducing a self-attention mechanism, the hierarchical memory network architecture can more intelligently process and aggregate information at different levels in the long text segmentation layer and document extraction layer. This mechanism enables the model to automatically learn and adapt to the importance of different parts, thereby more accurately understanding and extracting key information when processing long text data in the power field, and further improving the performance and accuracy of the large-scale power knowledge retrieval model.

[0051] The retrieval instruction recognition module 30 is used to receive user retrieval requests from the knowledge retrieval user terminal, perform request recognition and analysis, and generate a breakdown of user requests.

[0052] Furthermore, the retrieval instruction recognition module 30 is also used to perform the following steps:

[0053] P31: Perform natural language processing on the user's input search request to identify the key terms and question types and obtain the user's search requirements; P32: Decompose the user's search requirements into multiple sub-requests, each sub-request corresponding to a specific power knowledge search task, and output the decomposed user requirements.

[0054] Specifically, the retrieval instruction recognition module 30 of this application is responsible for receiving retrieval requests from users in the power knowledge retrieval system, performing request recognition and analysis, and transforming the complex retrieval requests put forward by users into specific tasks that can be processed by the system, thereby achieving accurate power knowledge retrieval.

[0055] First, the retrieval instruction recognition module 30 analyzes the user's input retrieval request using Natural Language Processing (NLP) technology. In this process, the model utilizes NLP techniques such as lexical analysis, syntactic analysis, and semantic understanding to identify and obtain key terms and question types in the retrieval request. Key terms refer to professional vocabulary in the power industry, such as "load forecasting," "grid optimization," and "power fault diagnosis." These terms help the model understand the key areas of the user's needs. Question type identification involves analyzing whether the user's retrieval needs are for solving technical problems, obtaining equipment information, understanding industry policies, or other types of queries. Through NLP technology, the module can identify these terms and their related context, thereby obtaining the user's retrieval needs.

[0056] Subsequently, the identified user search requests are broken down into multiple sub-requests, each corresponding to a specific power knowledge retrieval task. This decomposition process breaks down a relatively complex search request into smaller, more specific parts, making each sub-request more focused and clear, facilitating processing by the subsequent knowledge retrieval module. For example, if the user's search request is "fault diagnosis methods and their applications in smart grids," the module first identifies the key terms "smart grid," "fault diagnosis methods," and "applications," and determines the question type as seeking specific technical methods and practical application cases. Then, this complex request is broken down into two sub-requests: one is to search for fault diagnosis methods in smart grids, and the other is to find practical application cases of these methods. Through this decomposition method, the system can more accurately search for each sub-request, avoiding information redundancy and low accuracy caused by an overly broad search scope, thereby improving the relevance and practicality of the search results.

[0057] Finally, the broken-down user search requests are further passed to the power knowledge retrieval module, where knowledge retrieval and information extraction are performed separately for each sub-request. This meticulous requirement breakdown ensures that the system can accurately capture the details of different questions when faced with complex user queries, thereby providing more precise and comprehensive power knowledge retrieval results.

[0058] The power knowledge retrieval module 40 is embedded with the power knowledge retrieval model. It is used to receive the user's broken-down requirements, input them into the power knowledge retrieval model to perform power knowledge retrieval, and output the knowledge retrieval results.

[0059] Furthermore, after receiving the user's request for information breakdown, the power knowledge retrieval module 40 is also used to perform the following steps:

[0060] P41: The document classification layer of the power knowledge retrieval model performs document type matching based on the decomposed user needs to generate a retrieval document set, wherein the retrieval document set has long and short text classification labels; P42: Based on the retrieval document set, the demand long text set is extracted and input into the long text segmentation layer for multi-level encoding processing to generate a pre-encoded long text data set; P43: Based on the retrieval document set, the demand short text set is extracted and transmitted together with the pre-encoded long text data set to the document extraction layer for power knowledge extraction to obtain the knowledge retrieval results.

[0061] It should be understood that the power knowledge retrieval module 40 of this application is the core module for implementing the power knowledge retrieval function in this system. This module has a carefully trained power knowledge retrieval model embedded in it, which can receive user requirements decomposed by the retrieval instruction recognition module 30, perform power knowledge retrieval accordingly, and finally output accurate knowledge retrieval results.

[0062] Once the power knowledge retrieval module 40 receives the decomposed user request, it is first processed by the document classification layer of the power knowledge retrieval model. By analyzing the specific task of the user's retrieval, the document classification layer identifies the document types related to the user's request and classifies them according to their content, format, and length. The classified documents are divided into two categories: long documents and short documents. Long documents typically contain complex technical details or policy documents, while short documents may contain brief data such as equipment descriptions or fault records. Through document type matching, a set of retrieval documents containing long and short document classification identifiers is generated. This set provides a clear document reference basis for subsequent retrieval tasks.

[0063] Next, after generating the retrieval document set, the power knowledge retrieval module 40 further processes the documents in the set. For long texts, it extracts the required long text set and inputs it into the long text segmentation layer of the power knowledge retrieval model. The long text segmentation layer utilizes its long text processing capabilities acquired during the training phase to perform multi-level encoding processing on the long texts, including segmenting the long texts into chapters, paragraphs, and even sentences. It also uses techniques such as self-attention mechanisms to weighted aggregate information at different levels, thereby generating a pre-encoded long text dataset. This pre-encoded long text dataset not only retains the key information of the long texts but also presents them in a structure that is easier for subsequent processing, preparing for subsequent knowledge extraction.

[0064] Simultaneously, the short text portions of the retrieved document set also need to be processed. Based on the retrieved document set, a set of short texts related to the user's needs is extracted and transmitted along with the previously generated pre-coded long text data set to the document extraction layer. The document extraction layer, as a key part of the large-scale power knowledge retrieval model responsible for final knowledge extraction, comprehensively utilizes the capabilities of the feature extraction unit and the information verification unit to extract power knowledge from both long and short texts. In this process, the feature extraction unit is responsible for accurately extracting power knowledge relevant to the user's needs from the text, while the information verification unit is responsible for ensuring the accuracy and consistency of this knowledge. Ultimately, knowledge retrieval results highly matching the user's needs are obtained and output to the user.

[0065] Through these three steps, the power knowledge retrieval module can not only efficiently match relevant documents according to the broken-down user needs, but also accurately extract and integrate power knowledge from long and short documents, thereby providing users with accurate and high-value knowledge retrieval results.

[0066] Furthermore, when performing power knowledge extraction, the power knowledge retrieval module 40 is also used to perform the following steps:

[0067] P43-1: The document extraction layer extracts power knowledge from the short document set and the pre-encoded long document set through the feature extraction unit, obtains initial retrieval information, and transmits it synchronously to the information verification unit; P43-2: The information verification unit verifies the accuracy and consistency of the initial retrieval information, and outputs the knowledge retrieval result after the verification is passed.

[0068] Specifically, when the power knowledge retrieval module 40 performs power knowledge extraction, further detailed steps can be taken to ensure that the knowledge extracted from the short demand document set and the pre-coded long document set is accurate and consistent.

[0069] First, the document extraction layer extracts power knowledge from the short document set and the pre-coded long document set through the feature extraction unit. It identifies and extracts key information from these documents, including rapid extraction of key information from the short document set and in-depth mining of multi-level encoded information from the pre-coded long document set. For example, for equipment parameter descriptions in the short document set, the feature extraction unit can quickly extract key parameter values; while for technical principle explanations in the pre-coded long document set, it can extract core technical concepts and steps. Through this process, the document extraction layer obtains initial retrieval information and immediately transmits it synchronously to the information verification unit, providing timely input for subsequent verification work.

[0070] Next, the information verification unit verifies the accuracy and consistency of the initial retrieved information, ensuring that the information extracted from the documents is technically correct and conforms to relevant standards and regulations in the power industry. The verification process not only checks the accuracy of the information but also ensures that the extracted information is consistent with existing power knowledge bases to avoid errors. Verification methods may involve comparing the extracted information with industry standards, existing literature, and historical data. For example, verifying whether the extracted technical parameters conform to known industry standards, whether the technical concepts are consistent with existing power theories, and whether there are contradictions in the descriptions of the same topic in different documents. Only after the initial retrieved information passes these rigorous verifications will the information verification unit output it as the final knowledge retrieval result.

[0071] Through these two steps, the power knowledge retrieval module 40 can not only efficiently extract knowledge related to user needs from the power knowledge base when performing power knowledge extraction, but also ensure the accuracy and consistency of this knowledge through strict information verification, providing users with efficient, accurate and reliable power knowledge retrieval services.

[0072] Furthermore, the system also includes a domain expert feedback module 50, which is communicatively connected to the knowledge retrieval user terminal, periodically receives expert feedback, and performs incremental error correction training on the power knowledge retrieval big model.

[0073] Optionally, this application can further expand its functionality by adding a domain expert feedback module 50. This module is closely connected to the knowledge retrieval user terminal and can periodically receive feedback from domain experts. Based on this feedback, the power knowledge retrieval big model can be incrementally corrected and trained to ensure that the power knowledge retrieval system can gradually optimize and correct its retrieval capabilities as it is used, thereby improving its accuracy and reliability in long-term use.

[0074] In actual operation, after the power knowledge retrieval module 40 outputs knowledge retrieval results according to user needs, domain experts can evaluate the retrieval results through the knowledge retrieval user terminal and provide corresponding feedback. This feedback may cover multiple aspects such as the accuracy, completeness, and relevance of the retrieval results. For example, experts may point out that certain key power technology details are missing from the retrieval results, or that some information is misinterpreted.

[0075] The domain expert feedback module 50 periodically collects this feedback and transforms it into training data. After preprocessing, this data is used for incremental error correction training of the large-scale power knowledge retrieval model. During incremental error correction training, the model focuses on analyzing the errors or deficiencies pointed out by experts, adjusting model parameters, and optimizing the model structure to avoid similar problems from recurring in subsequent retrieval tasks. For example, if expert feedback indicates a deviation in the description of "power system stability analysis methods" in a search result, the model will specifically strengthen its understanding and expression of this knowledge point during incremental error correction training.

[0076] Through this process, the model can enhance its understanding of knowledge in the power sector through continuous iteration and improve its responsiveness to user search needs. The introduction of this feedback mechanism not only improves the model's intelligence but also enables the system to adapt to changing power industry environments and user demands, thereby continuously optimizing the effectiveness of power knowledge retrieval and ensuring the model's adaptability and accuracy in different scenarios.

[0077] In summary, the embodiments of this application have at least the following technical effects:

[0078] This application constructs a balanced training dataset by collecting professional data from the power sector and general data from other fields, thereby improving the model's understanding of power and cross-domain knowledge. It employs a hierarchical memory network architecture to achieve multi-level encoding and information extraction of long texts, solving the problem of information loss in long text processing. Through the collaborative work of document classification, long text segmentation, and document extraction layers, it accurately retrieves power knowledge, improving the accuracy and relevance of search results. It utilizes a self-attention mechanism to optimize long text understanding and information extraction. A search instruction recognition module performs natural language processing and decomposition of user needs, improving search accuracy and efficiency. Finally, through periodic feedback and incremental error correction training from a domain expert feedback module, the model is continuously optimized to ensure the accuracy and practicality of search results.

[0079] The technology has achieved the goal of improving the efficiency and accuracy of power knowledge retrieval through a hierarchical memory network architecture based on a large language model and balanced internal and external training.

[0080] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, the above description focuses on specific embodiments of this specification. Additionally, the processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results. In some implementations, multitasking and parallel processing are possible or may be advantageous.

[0081] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

[0082] This specification and accompanying drawings are merely illustrative examples of this application and are intended to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from its scope. Therefore, if such modifications and variations fall within the scope of this application and its equivalents, this application intends to include such modifications and variations.

Claims

1. A power knowledge retrieval system based on a large language model, characterized in that, The system includes: The training data acquisition module is used to collect specialized training datasets in the power sector through big data. These specialized training datasets include power sector-specific data and cross-domain general data. The model training module is used to perform internal and external balance training on a preset hierarchical memory network architecture based on the professional data in the power field and cross-domain general data, so as to build a large power knowledge retrieval model. The search instruction recognition module is used to receive user search requests from the knowledge retrieval user terminal, perform request recognition and analysis, and generate a breakdown of user requests. The power knowledge retrieval module, which embeds the power knowledge retrieval model, is used to receive the user's broken-down requirements, input them into the power knowledge retrieval model, perform power knowledge retrieval, and output the knowledge retrieval results. The model training module is also used for: Design a hierarchical memory network architecture, which includes a document classification layer, a long text segmentation layer, and a document extraction layer. The document extraction layer adopts a dual extraction verification mechanism, including a feature extraction unit and an information verification unit. Based on the professional data in the power field and the general data across fields, the document classification layer, long text segmentation layer and document extraction layer of the hierarchical memory network architecture are trained with internal and external balance to generate the large power knowledge retrieval model. The model training module, during internal and external balance training, is also used for: The power sector-specific data and cross-domain general data are subjected to dataset balancing processing, and training weights for each type of data are defined. Based on the training weights, classification training is performed using specialized data from the power sector and general cross-domain data to generate a document classification layer. Extract long-form power technology documents and perform multi-level coding training to generate long-text segmentation layers; Feature extraction training is performed using specialized data from the power sector, and parallel information verification training is performed using cross-domain general data to generate a document extraction layer. After receiving the user's request for information, the power knowledge retrieval module is further used for: The document classification layer of the power knowledge retrieval model performs document type matching based on the decomposed user needs to generate a retrieval document set, wherein the retrieval document set has long and short text classification labels; Based on the retrieved document set, the required long text set is extracted, input into the long text segmentation layer for multi-level encoding processing, and a pre-encoded long text data set is generated. Based on the retrieved document set, a set of short articles is extracted and transmitted together with the pre-encoded long article set to the document extraction layer for power knowledge extraction, thereby obtaining the knowledge retrieval results.

2. The power knowledge retrieval system based on a large language model as described in claim 1, characterized in that, When training the document extraction layer, the model training module is also used for: Using professional data from the power sector, we extracted hierarchical coding samples of long texts and coding samples of short texts, and trained them to extract features from long texts and short texts respectively, thus constructing a feature extraction unit. Cross-domain common feature samples are extracted using cross-domain common data, and information verification training is performed to construct information verification units.

3. The power knowledge retrieval system based on a large language model as described in claim 1, characterized in that, The search instruction recognition module is also used for: Natural language processing is performed on the user's search request to identify key terms and question types and obtain the user's search needs; The user's search request is broken down into multiple sub-requests, each sub-request corresponding to a specific power knowledge search task, and the broken down user request is output.

4. The power knowledge retrieval system based on a large language model as described in claim 1, characterized in that, In the model training module, the hierarchical memory network architecture also includes a self-attention mechanism, which is used to perform weighted aggregation of information at different levels in the long text segmentation layer and the document extraction layer.

5. The power knowledge retrieval system based on a large language model as described in claim 1, characterized in that, The power knowledge retrieval module, when performing power knowledge extraction, is also used for: The document extraction layer extracts power knowledge from the short document set and the pre-encoded long document set through the feature extraction unit, obtains initial retrieval information, and transmits it synchronously to the information verification unit. The information verification unit verifies the accuracy and consistency of the initial search information. If the verification is successful, the knowledge search result is output.

6. The power knowledge retrieval system based on a large language model as described in claim 1, characterized in that, The system also includes a domain expert feedback module, which is connected to the knowledge retrieval user terminal and periodically receives expert feedback to perform incremental error correction training on the power knowledge retrieval big model.

Images