Electric power knowledge retrieval system based on large language model

Abstract

The invention discloses an electric power knowledge retrieval system based on a large language model, and relates to the technical field of data processing, and the system comprises a training data collection module which is used for collecting professional data and cross-domain general data in the electric power field; the model training module is used for performing internal and external balance training on the hierarchical memory network architecture based on the training data and constructing a large electric power knowledge retrieval model; the retrieval instruction identification module is used for receiving a retrieval demand of a user side and analyzing and generating a disassembly user demand; and the electric power knowledge retrieval module is used for receiving disassembly user requirements and inputting the electric power knowledge retrieval large model to perform electric power knowledge retrieval. The technical problems that an existing power knowledge retrieval system is low in efficiency and poor in accuracy when processing complex documents and cross-domain knowledge are solved, and the technical effect that the power knowledge retrieval efficiency and accuracy are improved through a hierarchical memory network architecture based on a large language model and internal and external balance training is achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and particularly relates to a power knowledge retrieval system based on a large language model. BACKGROUND

[0002] With the rapid development of the power industry, the accumulation and complexity of power knowledge are increasing, which puts forward higher requirements for power knowledge retrieval systems. Existing power knowledge retrieval systems usually rely on keyword-based matching or rule-driven methods. These methods often have low retrieval efficiency and poor accuracy when facing complex technical documents, real-time data and cross-domain knowledge in the power field, especially when dealing with long technical documents, which easily loses key information. In addition, the power field is increasingly crossing with other fields, and the existing system has obvious shortcomings in integrating cross-domain knowledge, which limits the accuracy and practicality of the retrieval results. SUMMARY

[0003] The present application provides a power knowledge retrieval system based on a large language model, which solves the technical problems of low efficiency and poor accuracy of existing power knowledge retrieval systems when dealing with complex documents and cross-domain knowledge.

[0004] The present application provides a power knowledge retrieval system based on a large language model, which includes: a training data collection module for collecting a special training data set in the power field through big data, the special training data set including power field professional data and cross-domain general data; a model training module for conducting internal and external balance training on a preset hierarchical memory network architecture based on the power field professional data and cross-domain general data, and constructing a power knowledge retrieval large model; a retrieval instruction recognition module for receiving user retrieval requirements of a knowledge retrieval user end and conducting requirement recognition analysis to generate disassembled user requirements; a power knowledge retrieval module embedded with the power knowledge retrieval large model, for receiving the disassembled user requirements and inputting the power knowledge retrieval large model for power knowledge retrieval, and outputting knowledge retrieval results.

[0005] One or more technical solutions provided in the present application have at least the following technical effects or advantages:

[0006] The application provides a power knowledge retrieval system based on a large language model, relates to the technical field of data processing, and collects professional data in the power field and general data across fields through a training data collection module, supports internal and external balance training of a model training module, constructs a power knowledge retrieval large model, analyzes user retrieval requirements through a retrieval instruction recognition module, receives user requirements through a power knowledge retrieval module, inputs the retrieval large model to perform knowledge retrieval, and outputs accurate retrieval results, thereby solving the technical problems of low efficiency and poor accuracy of existing power knowledge retrieval systems in processing complex documents and cross-field knowledge, and achieving the technical effects of improving power knowledge retrieval efficiency and accuracy through a hierarchical memory network architecture based on a large language model and internal and external balance training. BRIEF DESCRIPTION OF DRAWINGS

[0007] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0008] Figure 1 A structure schematic diagram of a power knowledge retrieval system based on a large language model is provided for the embodiments of the present application.

[0009] Figure 2 A structure schematic diagram of a hierarchical memory network architecture in a power knowledge retrieval system based on a large language model is provided for the embodiments of the present application.

[0010] Explanation of reference signs: training data collection module 10, model training module 20, retrieval instruction recognition module 30, power knowledge retrieval module 40, and field expert feedback module 50. DETAILED DESCRIPTION

[0011] The application provides a power knowledge retrieval system based on a large language model, which is used to solve the technical problems of low efficiency and poor accuracy of existing power knowledge retrieval systems in processing complex documents and cross-field knowledge.

[0012] The technical solutions in the embodiments of the present application will be described clearly and completely in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0013] It should be noted that the terms "first", "second", etc. in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or server including a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but can include other steps or modules not clearly listed or inherent to these processes, methods, products or devices.

[0014] Embodiment one, as shown in the present application provides a power knowledge retrieval system based on large language model, which comprises: Figure 1

[0015] A training data acquisition module 10 is used to collect power field specific training data set through big data, and the specific training data set includes power field professional data and cross-field general data.

[0016] Specifically, the core function of the training data acquisition module 10 of the present application is to collect power field specific training data set through big data technology, and the data set includes power field professional data and cross-field general data.

[0017] First, in order to ensure the breadth and comprehensiveness of the data, the training data acquisition module 10 uses big data technology (such as distributed data acquisition and processing framework) to automatically collect data from various power-related resources. These resources include but are not limited to power equipment operation data, maintenance records, technical reports, standard documents, academic papers and various sensor data, etc. Power field professional data refers to various data directly related to power system, such as power load, power generation efficiency, power distribution network state, equipment failure record and other information. These data usually come from various aspects of power production, transmission, distribution and consumption, and are the basis for power system management and optimization.

[0018] In order to enhance the generality and adaptability of the model, the training data acquisition module 10 will also collect cross-field general data. Cross-field data refers to data in fields indirectly related to power systems but having reference value in certain contexts, such as weather data (weather conditions affecting power load), economic indicators (such as the relationship between energy price fluctuations and demand changes), social behavior data (such as changes in consumption patterns), etc. By combining these external data with power professional data, the model can better handle complex multi-factor problems, thereby improving its accuracy and generalization ability in practical applications.

[0019] ​After data collection, the module pre-processes and integrates the data, removing redundant and irrelevant information through cleaning, standardization, and denoising techniques to ensure data quality. During this step, feature engineering is also performed, including time series analysis of sensor data, fault detection, and prediction techniques to extract high-information features. With these techniques, the collected power professional data not only reflects the operation status of the power system but also uncovers potential patterns and trends, providing a solid data foundation for subsequent model training.

[0020] Finally, the multi-source data is fused and modeled, with different sources of data (such as power data, meteorological data, social and economic data, etc.) being fused according to certain rules to generate a comprehensive data set suitable for large language model training. In this process, the balance and representativeness of the data set are crucial, so the module will select and collect data based on the demand characteristics and future development trends of the power industry to ensure that the collected data covers all important areas of the power system and avoids data bias towards a specific direction or field, providing a rich and high-value data source for the power knowledge retrieval system.

[0021] The model training module 20 is configured to perform the following steps:

[0022] Further, as shown in Figure 2

[0023] P21: Design a hierarchical memory network architecture, which includes a document classification layer, a long text cutting layer, and a document extraction layer, wherein the document extraction layer adopts a double extraction verification mechanism, including a feature extraction unit and an information verification unit; P22: Based on the power field professional data and cross-field general data, the document classification layer, the long text cutting layer, and the document extraction layer of the hierarchical memory network architecture are trained internally and externally balanced to generate the power knowledge retrieval large model.

[0024] It should be understood that the model training module 20 of the present application is responsible for constructing a power knowledge retrieval large model by internally and externally balancing the power field professional data and cross-field general data. In this module, a hierarchical memory network architecture suitable for power knowledge retrieval requirements needs to be designed first. This architecture includes a document classification layer, a long text cutting layer, and a document extraction layer. In the design process, the main task of the document classification layer is to preliminarily classify the input text data, dividing the power field documents into different categories such as power load, equipment failure, and operation log. The classified data facilitates subsequent processing, ensuring that the model can efficiently analyze different categories of data in subsequent steps.​

[0025] The long text cutting layer's role is to cut and split the input long text, avoiding information loss due to excessive input. This layer will disassemble the document according to chapters or paragraphs, so that the model can analyze each smaller part in depth, which helps to improve the accuracy and efficiency of long text processing. The cut data can provide the model with more detailed context understanding, especially in the technical literature of the power industry, long reports and standard documents are common, and fine processing can effectively reduce information omission.

[0026] The document extraction layer's task is to extract key information from the cut document, including the running state of power equipment, load prediction, fault detection, etc. This layer uses a double extraction verification mechanism to improve the accuracy of extraction. First, the feature extraction unit identifies key features from the text, such as device operating parameters, fault types, or technical details. Then, the information verification unit verifies these extracted features to ensure the accuracy and consistency of the information. This verification process can compare the extracted information with existing power industry knowledge bases to confirm whether the information is reasonable, thereby avoiding the impact of incorrect data on model training.

[0027] After completing the design of the hierarchical memory network architecture, the model training module will then proceed with internal and external balance training. In this stage, professional data in the power field and cross-domain general data are introduced into the training process. Professional data in the power field includes power load data, power equipment operation logs, technical reports, etc., which enable the model to deeply learn the working principles of power systems, device operation, and fault diagnosis, etc. core technology. Cross-domain general data, including meteorological data, economic data, and social behavior data, are also incorporated into the training process, with the aim of enabling the model to learn from external factors, such as the impact of weather changes on power load, the impact of energy price fluctuations on power demand, etc. In this way, the model not only focuses on the core knowledge of the power field, but also effectively absorbs knowledge from other fields, enhancing its ability to handle complex situations and cross-disciplinary problems.

[0028] In addition, the internal-external balance training balances the training proportion of power professional data and cross-domain data by dynamically adjusting the weight of the loss function. Specifically, the model adjusts the training process according to the contribution of different data sources during training, ensuring that it learns deeply in the power professional field while not neglecting knowledge in other fields, making the training process more adaptable and generalizable. Internal training is mainly aimed at power field professional data, and each level of the model is trained using labeled power field data through supervised learning. For example, in the document classification layer, the model learns how to classify documents into the correct topic category based on the title and abstract; in the long text cutting layer, the model learns how to reasonably divide long documents into chapters, paragraphs, and sentences; in the document extraction layer, the model learns how to accurately extract key information from text units and ensures the accuracy and consistency of the extracted information through the information verification unit.

[0029] External training uses cross-domain general data, and through unsupervised or semi-supervised learning, it enhances the generalization ability and flexibility of the model. For example, in the document classification layer, the model learns how to handle cross-domain documents related to the power field; in the long text cutting layer, the model learns how to handle structural differences in documents from different fields; in the document extraction layer, the model learns how to extract general information from cross-domain documents. Through internal-external balance training, the model can not only fully utilize its domain expertise when dealing with power field problems, but also flexibly handle general problems across domains.

[0030] Finally, through this series of training, the model training module successfully generates a power knowledge retrieval large model, which not only accurately understands the professional terminology and technical details of the power field, but also effectively processes knowledge from other fields, becoming the core driving force of the power knowledge retrieval system.

[0031] Further, the model training module 20, when performing internal-external balance training, is further configured to perform the following steps:

[0032] P22-1: Perform dataset balancing processing on the power field professional data and cross-domain general data, and define the training weight of each type of data; P22-2: Based on the training weight, use power field professional data and cross-domain general data for classification training to generate a document classification layer; P22-3: Extract long power technical documents for multi-level encoding training to generate a long text cutting layer; P22-4: Use power field professional data for feature extraction training, and combine cross-domain general data for parallel information verification training to generate a document extraction layer.

[0033] Optionally, when performing internal-external balance training, the model training module 20 further optimizes the training process to ensure that power field professional data and cross-domain general data can be reasonably processed and integrated in the model.

[0034] Firstly, the dataset balancing is performed on the professional data in the power field and the general data across fields to ensure a reasonable weight distribution of the two types of data during the training process. Specifically, according to expert experience or industry data experience, the training weight of each type of data is defined to ensure that the professional data in the power field can fully train the power knowledge of the model, while the general data across fields helps to improve the generalization ability of the model and avoid overfitting to a specific field.

[0035] On this basis, the defined training weights are used to classify and train the professional data in the power field and the general data across fields. The generation process of the document classification layer relies on the guidance of these weights. Through classification of the input data, the model can automatically identify different categories in the power documents, such as power equipment, load scheduling, fault detection, etc. In this process, the model learns how to classify power field documents into different topic categories based on information such as document titles, abstracts, and keywords. For example, documents about "power system stability analysis" are classified into one category, while documents about "smart grid technology application" are classified into another category. This classification helps the subsequent layers to more accurately process and extract information, improving the overall performance of the model. At the same time, by combining the training of general data across fields, the model can better understand the commonalities and differences of documents in different fields, further enhancing its generalization ability.

[0036] Next, multi-level encoding training is performed on long power technical documents to generate a long text cutting layer. Long power technical documents are usually long and complex, containing a large amount of technical details, formula derivation, and experimental data. Through multi-level encoding training, the model can decompose long documents into smaller text units such as chapters, paragraphs, and sentences, and encode and aggregate each level of text unit. This process not only helps the model better understand and extract key information from long documents, but also avoids the problem of information loss or understanding bias when processing long text. For example, when processing a long paper on power system stability analysis, the model can first encode each chapter of the paper to extract the core content of each chapter; then further encode the paragraphs in each chapter to extract the key information of the paragraphs; finally, encode the sentences to capture the detailed information at the sentence level. Through this multi-level encoding and aggregation process, the model can better grasp the overall structure and key information of long documents, thereby optimizing the context understanding and information extraction ability of long documents.

[0037] Finally, when generating the document extraction layer, professional data in the power field is used for feature extraction training, and cross-domain general data is used for parallel information verification training. The document extraction layer is a key part of the hierarchical memory network architecture, which adopts a dual extraction and verification mechanism, including a feature extraction unit and an information verification unit. The feature extraction unit is responsible for extracting key information from text units, such as technical terms, parameters, formulas, etc. For example, when extracting "power equipment fault diagnosis method", the feature extraction unit will identify the names, principles and steps of various fault diagnosis technologies. The information verification unit verifies and corrects the extracted information to ensure the accuracy and consistency of the extracted information. In this process, the model not only uses professional data in the power field for feature extraction training to ensure that it can accurately extract key information in the power field, but also uses cross-domain general data for parallel information verification training to verify and correct the extracted information by comparing information from multiple sources and checking the logical relationships between information. For example, the model can check whether the extracted fault diagnosis technology is consistent with known fault diagnosis theories and practices, and whether it is consistent in different literature, etc., to improve the quality of information extraction.

[0038] Through these steps, the model training module 20 can effectively train the hierarchical memory network architecture for internal and external balance, generating a power knowledge retrieval large model with efficient long text understanding and information extraction capabilities. When dealing with complex knowledge retrieval tasks in the power field, the model can accurately understand professional terms and concepts in the power field and flexibly handle general problems across domains.

[0039] Further, when performing training of the document extraction layer, the model training module 20 is further configured to perform the following steps:

[0040] P22-41: Use professional data in the power field to extract long text hierarchical coding samples and short text coding samples, respectively perform long text feature and short text feature extraction training, and construct a feature extraction unit; P22-42: Use cross-domain general data to extract cross-domain general feature samples, perform information verification training, and construct an information verification unit.

[0041] In a possible embodiment of the present application, when performing training of the document extraction layer, the model training module 20 further refines the training process of feature extraction and information verification to ensure that key information can be efficiently and accurately extracted from long and short documents, and the extracted information is effectively verified.

[0042] First, long-text hierarchical encoding samples and short-text encoding samples are extracted using professional data in the field of electric power. Long-text hierarchical encoding samples are usually derived from technical reports, industry standards, and equipment manuals in the field of electric power, which often contain complex contextual information and are lengthy. The model needs to be trained for hierarchical encoding of these long documents to ensure that it can understand each level (such as chapters, paragraphs, etc.) in the document and extract key information at each level. Short-text encoding samples are mainly derived from electric power equipment failure reports, load scheduling records, and other relatively short texts, which usually focus more on the description of specific problems and have relatively less information but are also of great value.

[0043] Through separate long-text feature extraction training and short-text feature extraction training, the model can be optimized for different types of documents. Long-text feature extraction training focuses on contextual understanding and hierarchical information extraction, while short-text feature extraction training focuses on quickly and accurately extracting key data from short documents. During training, the model uses professional data in the field of electric power to learn how to extract important features such as electric power equipment status and system operating parameters from long and short documents, and constructs a feature extraction unit that can handle various information types involved in electric power documents, ensuring that key information can be quickly and accurately extracted in subsequent information extraction.

[0044] Next, cross-domain general feature samples are extracted using cross-domain general data for information verification training. Cross-domain general data usually includes data from fields such as meteorology and economics, such as temperature, humidity, and energy price fluctuations, which may have indirect effects on the scheduling and operation of the electric power system. By incorporating these data into model training, the model can learn how to handle cross-domain factors related to the electric power system, thereby improving its ability to solve complex problems. For example, by analyzing mathematical formulas and physical principles in cross-domain general data, the model can better verify the correctness of relevant technical descriptions in electric power field documents. During information verification training, the model learns how to compare information from different sources, check the logical relationships and consistency between information, and thus constructs an information verification unit that can efficiently verify the accuracy of information. For example, after the feature extraction unit extracts the key information in the "electric power equipment fault diagnosis method", the information verification unit can use relevant knowledge from cross-domain general data to verify whether these information is consistent with known fault diagnosis theories and practices, and whether it is consistent in different literature, etc., thereby improving the quality and reliability of information extraction.

[0045] Through the above training steps, not only the feature extraction capability of professional data in the field of electric power is enhanced, but also the accuracy and robustness of the information verification unit are improved through the verification training of cross-domain data.

[0046] Further, in the model training module 20, the hierarchical memory network architecture further includes a self-attention mechanism for weighting and aggregating information at different levels in the long text cutting layer and the document extraction layer.

[0047] Optionally, in the model training module P20, the hierarchical memory network architecture further includes a self-attention mechanism, mainly for information processing and weighted aggregation in the long text cutting layer and the document extraction layer. The self-attention mechanism is an important technology in deep learning, which automatically assigns a weight to each element by calculating the correlation between elements in the input sequence, so that the model can pay more attention to important parts of the input data. In the task of power knowledge retrieval, the understanding of long text and complex documents requires more efficient information aggregation and context modeling capabilities, and the self-attention mechanism is designed for this purpose.

[0048] In the long text cutting layer, the self-attention mechanism performs weighted aggregation on the multi-level encoding results of long power technical documents, so that the model can pay more attention to the importance of different parts of the document. For example, for a long paper on power system stability analysis, some chapters may contain key theoretical derivation and experimental results, while other parts may be relatively less important. The self-attention mechanism can automatically learn the importance weights of these different parts, so that when cutting long documents, it can more accurately retain key information and avoid losing important details. This weighted aggregation method not only improves the model's understanding of long text structure, but also enhances its adaptability and flexibility in handling complex power documents.

[0049] In the document extraction layer, the role of the self-attention mechanism is more prominent. After the feature extraction unit extracts key information from the text unit, the information verification unit needs to verify and correct these information. The role of the self-attention mechanism here is to perform weighted aggregation on the extracted information, so that the model can pay more attention to the information that is highly relevant to both power domain professional knowledge and cross-domain general knowledge. For example, in verifying the information extracted in "power equipment fault diagnosis method", the self-attention mechanism can help the model identify which information is highly consistent with known fault diagnosis theory, and which information may need further verification or correction. In this way, the self-attention mechanism not only improves the accuracy of information verification, but also enhances the overall performance of the model in handling power knowledge retrieval tasks.

[0050] By introducing the self-attention mechanism, the hierarchical memory network architecture can more intelligently process and aggregate information at different levels in the long text segmentation layer and the document extraction layer. This mechanism enables the model to automatically learn and adapt to the importance of different parts, allowing it to more accurately understand and extract key information when processing long text data in the power field, thereby further improving the performance and accuracy of the power knowledge retrieval large model.

[0051] The retrieval instruction recognition module 30 is used to receive the user retrieval demand of the knowledge retrieval user terminal and perform demand recognition analysis to generate a disassembled user demand.

[0052] Further, the retrieval instruction recognition module 30 is further used to perform the following steps:

[0053] P31: performing natural language processing on the user input retrieval request, identifying and obtaining key terms and problem types, and obtaining the user retrieval demand; P32: disassembling the user retrieval demand into multiple sub-requests, each sub-request corresponding to a specific power knowledge retrieval task, and outputting the disassembled user demand.

[0054] Specifically, the retrieval instruction recognition module 30 of the present application is responsible for receiving retrieval demands from the user terminal in the power knowledge retrieval system and performing demand recognition analysis to convert complex retrieval requests proposed by users into specific tasks that can be processed by the system, thereby achieving accurate power knowledge retrieval.

[0055] First, the retrieval instruction recognition module 30 analyzes the user input retrieval request through natural language processing (NLP) technology. In this process, the model uses natural language processing techniques such as morphological analysis, syntactic analysis, and semantic understanding to identify and obtain key terms and problem types in the retrieval request. Key terms refer to professional vocabulary in the power field, such as "load forecasting", "power grid optimization", "power fault diagnosis", etc. These terms can help the model understand the key direction of the user's demand. The identification of problem types involves analyzing whether the user's retrieval demand is to solve technical problems, obtain equipment information, understand industry policies, or other types of queries. Through natural language processing technology, the module can identify these terms and their related context to obtain the user's retrieval demand.

[0056] Subsequently, the identified user search demand is disassembled into multiple sub-requests, each corresponding to a specific power knowledge search task. This disassembly process breaks down a complex search request into smaller, specific parts, making each sub-request more focused and explicit, facilitating subsequent processing by the knowledge search module. For example, if the user's input search request is "fault diagnosis methods in smart grids and their applications", the module first identifies the key terms "smart grid", "fault diagnosis method", and "application", and determines the problem type as seeking specific technical methods and practical application cases. Subsequently, this complex request is disassembled into two sub-requests: one is to search for fault diagnosis methods in smart grids, and the other is to find application cases of these methods. Through this disassembly method, the system can more accurately search for each sub-request, avoiding information redundancy and low accuracy due to a too wide search range, thereby improving the relevance and practicality of the search results.

[0057] Finally, the disassembled user search demand is further passed to the power knowledge search module, and knowledge search and information extraction are performed according to each sub-request. The entire process ensures that the system can accurately capture the details of different problems when faced with complex user problems, thereby providing more accurate and comprehensive power knowledge search results.

[0058] The power knowledge search module 40 has the power knowledge search large model embedded therein, for receiving the disassembled user demand, inputting the power knowledge search large model for power knowledge search, and outputting the knowledge search result.

[0059] Further, the power knowledge search module 40, after receiving the disassembled user demand, is further configured to perform the following steps:

[0060] P41: A document type matching is performed by the document classification layer of the power knowledge search large model based on the disassembled user demand, to generate a search document set, wherein the search document set is provided with long and short document classification labels; P42: Based on the search document set, a long document set is extracted and input to the long document cutting layer for multi-level encoding processing to generate a pre-encoded long document material set; P43: Based on the search document set, a short document set is extracted and transmitted to the document extraction layer together with the pre-encoded long document material set for power knowledge extraction to obtain the knowledge search result.

[0061] It should be understood that the power knowledge search module 40 of the present application is the core module for realizing the power knowledge search function in the system. The module has a carefully trained power knowledge search large model embedded therein, which can receive the user demand disassembled by the search instruction recognition module 30 and perform power knowledge search accordingly, and finally output accurate knowledge search results.

[0062] When the power knowledge retrieval module 40 receives the disassembled user demand, it is first handled by the document classification layer of the power knowledge retrieval large model. By analyzing the specific task of user retrieval, the document classification layer identifies the document type related to the user demand, and classifies according to the content, format and length of the document. The classified documents are divided into two categories: long text and short text. Long text usually contains more complex technical details or policy documents, and short text may be short data such as equipment instructions and fault records. Through the matching of document types, a retrieval document set containing long and short text classification identifiers is generated, which provides a clear document reference basis for subsequent retrieval tasks.

[0063] Next, after generating the retrieval document set, the power knowledge retrieval module 40 further processes the documents in the set. For the long text part, the demand long text set is extracted and input into the long text cutting layer of the power knowledge retrieval large model. The long text cutting layer uses its long text processing ability mastered in the training stage to perform multi-level coding processing on the long text, including hierarchical processing of chapters, paragraphs, and sentences, and weighting aggregation of information at different levels through self-attention mechanism and other technologies, thereby generating a pre-encoding long text data set. The pre-encoding long text data set not only retains the key information of the long text, but also presents it in a structure that is easier to process subsequently, making it ready for subsequent knowledge extraction.

[0064] At the same time, the short text part in the retrieval document set also needs to be processed. Based on the retrieval document set, the demand short text set is extracted and transmitted to the document extraction layer together with the pre-encoding long text data set generated earlier. The document extraction layer, as the key part of the power knowledge retrieval large model responsible for final knowledge extraction, comprehensively uses the capabilities of the feature extraction unit and the information verification unit to extract power knowledge from long and short texts. In this process, the feature extraction unit is responsible for accurately extracting power knowledge related to user demand from text, while the information verification unit is responsible for ensuring the accuracy and consistency of these knowledge. Finally, the knowledge retrieval result highly matched with the user demand is obtained and output to the user.

[0065] Through these three steps, the power knowledge retrieval module not only can efficiently match relevant documents according to the disassembled user demand, but also can accurately extract and integrate power knowledge from long and short documents, thereby providing accurate and high-value knowledge retrieval results for users.

[0066] Further, the power knowledge retrieval module 40, when performing power knowledge extraction, is further configured to perform the following steps:

[0067] P43-1: The document extraction layer extracts power knowledge from the demand short text set and the pre-encoded long text set through the feature extraction unit, obtains initial retrieval information, and synchronously transmits it to the information verification unit; P43-2: The information verification unit verifies the accuracy and consistency of the initial retrieval information, and outputs the knowledge retrieval result after verification.

[0068] Specifically, when the power knowledge retrieval module 40 performs power knowledge extraction, the extracted knowledge from the demand short text set and the pre-encoded long text set can be further ensured to be accurate and consistent through refined steps.

[0069] First, the document extraction layer extracts power knowledge from the demand short text set and the pre-encoded long text set through the feature extraction unit, identifies and extracts key information from these documents, including rapid extraction of key information in the short text set and in-depth mining of multi-level coded information in the pre-encoded long text set. For example, for the description of equipment parameters in the short text set, the feature extraction unit can quickly extract the key parameter values; while for the description of technical principles in the pre-encoded long text set, it can extract the core technical concepts and steps. Through this process, the document extraction layer obtains initial retrieval information and immediately synchronously transmits it to the information verification unit to provide timely input for subsequent verification work.

[0070] Next, the information verification unit takes over the verification of the accuracy and consistency of the initial retrieval information, ensuring that the information extracted from the documents is technically correct and consistent with the relevant standards and rules in the power field. The verification process not only needs to check the accuracy of the information, but also needs to ensure that the extracted information is consistent with the existing power knowledge base, avoiding the appearance of false information. The verification method can be to compare the extracted information with industry standards, existing literature and historical data, for example, to verify whether the extracted technical parameters meet the known industry standards, whether the technical concepts are consistent with the existing power theory, and whether the descriptions of the same theme in different documents are contradictory, etc. Only when the initial retrieval information passes these strict verifications, the information verification unit will output it as the final knowledge retrieval result.

[0071] Through these two steps, when the power knowledge retrieval module 40 performs power knowledge extraction, it not only can efficiently extract knowledge related to user demand from the power knowledge base, but also can ensure the accuracy and consistency of these knowledge through strict information verification, providing efficient, accurate and reliable power knowledge retrieval service for users.

[0072] Further, the system also comprises a domain expert feedback module 50, which is in communication connection with the knowledge retrieval user end, periodically receives expert feedback opinions, and performs incremental error correction training on the power knowledge retrieval large model.

[0073] Optionally, the application can further expand the function, and add a domain expert feedback module 50, which is closely connected with the knowledge retrieval user end, can periodically receive feedback opinions of domain experts, and based on these feedbacks, perform incremental error correction training on the power knowledge retrieval large model, to ensure that the power knowledge retrieval system can gradually optimize and correct its retrieval ability, and improve its accuracy and reliability in long-term use.

[0074] In actual operation, when the power knowledge retrieval module 40 outputs the knowledge retrieval result according to the user demand, the domain expert can evaluate the retrieval result through the knowledge retrieval user end, and give corresponding feedback opinions. These feedback opinions can cover the accuracy, completeness, relevance and other aspects of the retrieval result. For example, the expert may point out that some key power technical details are missing in the retrieval result, or some information is misinterpreted.

[0075] The domain expert feedback module 50 regularly collects these feedback opinions and converts them into training data. After preprocessing, these data are used to perform incremental error correction training on the power knowledge retrieval large model. In the incremental error correction training process, the model focuses on analyzing the errors or deficiencies pointed out by the expert, adjusts the model parameters, optimizes the model structure, so as to avoid the reoccurrence of similar problems in subsequent retrieval tasks. For example, if the expert feedbacks that the description of "power system stability analysis method" in a retrieval result is biased, the model will specifically strengthen the understanding and expression ability of this knowledge point in the incremental error correction training.

[0076] Through this process, the model can enhance the understanding of the power domain knowledge and improve the response ability to the user retrieval demand in continuous iteration. The introduction of this feedback mechanism not only can improve the intelligent degree of the model, but also can make the system adapt to the changing environment of the power industry and the user demand, so as to continuously optimize the effect of power knowledge retrieval, and ensure the adaptability and accuracy of the model in different scenarios.

[0077] In summary, the embodiments of the application have at least the following technical effects:

[0078] The application collects professional data in the power field and cross-field general data, constructs an internally and externally balanced training data set, improves the understanding of the model for power and cross-field knowledge, adopts a hierarchical memory network architecture, realizes multi-level coding and information extraction of long text, solves the information loss problem in long text processing, cooperates through the document classification layer, long text cutting layer and document extraction layer, accurately retrieves power knowledge, improves the accuracy and relevance of the search results, uses the self-attention mechanism, optimizes long text understanding and information extraction, uses the retrieval instruction recognition module to process and disassemble the user demand in natural language, improves the search accuracy and efficiency, uses the periodic feedback and incremental error correction training of the field expert feedback module to continuously optimize the model, and ensures the accuracy and practicality of the search results.

[0079] The technical effect of improving the efficiency and accuracy of power knowledge retrieval is achieved through the hierarchical memory network architecture based on the large language model and the internal and external balanced training.

[0080] It should be noted that the above sequence of the embodiments of the application is only for description, and does not represent the advantages and disadvantages of the embodiments. And the above describes the specific embodiments of the present application. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multi-task processing and parallel processing are also possible or can be advantageous.

[0081] The above is only the preferred embodiment of the application, and does not limit the application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the application shall be included in the protection scope of the application.

[0082] The present application and the drawings are only exemplary descriptions of the application, and are considered to cover any and all modifications, changes, combinations or equivalents within the scope of the application. Obviously, those skilled in the art can make various modifications and changes to the application without departing from the scope of the application. Thus, if these modifications and changes of the application belong to the scope of the application and its equivalent technology, the application intends to include these modifications and changes.

Claims

1. A power knowledge retrieval system based on a large language model, characterized by, The system comprises: a training data collection module for collecting power field-specific training data sets through big data, wherein the power field-specific training data sets comprise power field professional data and cross-field general data; a model training module for performing internal-external balance training on a preset hierarchical memory network architecture based on the power field professional data and the cross-field general data, and constructing a power knowledge retrieval large model; a retrieval instruction recognition module for receiving user retrieval requirements of a knowledge retrieval user end, performing requirement recognition analysis, and generating decomposed user requirements; a power knowledge retrieval module, which is embedded with the power knowledge retrieval large model, for receiving the decomposed user requirements, inputting the power knowledge retrieval large model for power knowledge retrieval, and outputting knowledge retrieval results.

2. The power knowledge retrieval system based on a large language model of claim 1, wherein, The model training module is further configured to: design a hierarchical memory network architecture, which comprises a document classification layer, a long document cutting layer, and a document extraction layer, wherein the document extraction layer adopts a double extraction verification mechanism, including a feature extraction unit and an information verification unit; perform internal-external balance training on the document classification layer, the long document cutting layer, and the document extraction layer of the hierarchical memory network architecture based on the power field professional data and the cross-field general data, and generate the power knowledge retrieval large model. 3.The power knowledge retrieval system based on a large language model of claim 2, wherein, When performing the internal-external balance training, the model training module is further configured to: perform data set balance processing on the power field professional data and the cross-field general data, and define training weights of each type of data; perform classification training on the power field professional data and the cross-field general data based on the training weights, and generate the document classification layer; perform multi-level coding training on long power technical documents, and generate the long document cutting layer; perform feature extraction training on the power field professional data, and perform parallel information verification training in combination with the cross-field general data, and generate the document extraction layer.

4. The power knowledge retrieval system based on a large language model of claim 3, wherein, When performing the training of the document extraction layer, the model training module is further configured to: use the power field professional data to extract long document hierarchical coding samples and short document coding samples, perform long document feature extraction training and short document feature extraction training respectively, and construct the feature extraction unit; use the cross-field general data to extract cross-field general feature samples, perform information verification training, and construct the information verification unit.

5. The power knowledge retrieval system based on a large language model of claim 3, wherein, The retrieval instruction recognition module is further configured to: perform natural language processing on a retrieval request input by a user, identify and acquire key terms and question types, and obtain user retrieval requirements; decompose the user retrieval requirements into multiple sub-requests, each of which corresponds to a specific power knowledge retrieval task, and output the decomposed user requirements.

6. The power knowledge retrieval system based on a large language model of claim 2, wherein, In the model training module, the hierarchical memory network architecture further comprises a self-attention mechanism for weighting and aggregating information at different levels in the long document cutting layer and the document extraction layer.

7. The power knowledge retrieval system based on a large language model of claim 4, wherein, After receiving the decomposed user requirements, the power knowledge retrieval module is further configured to: perform document type matching based on the decomposed user requirements by the document classification layer of the power knowledge retrieval large model, and generate a retrieval document set, wherein the retrieval document set is provided with long and short document classification identifiers. Based on the search document set, the demand long text set is extracted and input to the long text cutting layer for multi-level coding processing to generate a pre-encoded long text data set; Based on the search document set, the demand short text set is extracted and transmitted to the document extraction layer together with the pre-encoded long text data set for power knowledge extraction to obtain the knowledge search result.

8. The power knowledge retrieval system based on a large language model of claim 7, wherein, When performing power knowledge extraction, the power knowledge search module is further configured to: The document extraction layer extracts power knowledge from the demand short text set and the pre-encoded long text data set through a feature extraction unit to obtain initial search information and synchronously transmit the initial search information to the information verification unit; The information verification unit verifies the accuracy and consistency of the initial search information, and outputs the knowledge search result after verification. 9.The power knowledge retrieval system based on a large language model of claim 1, wherein, The system further comprises a domain expert feedback module in communication connection with the knowledge search user end, periodically receives expert feedback opinions, and performs incremental error correction training on the power knowledge search large model.

Images