Power standard knowledge question answering method and system and related equipment

Abstract

The invention discloses an electric power standard knowledge question answering method and system and related equipment. The method comprises the steps of obtaining a question asked by a user; a semantic routing module determines to distribute the problem to a standard business scene model or a power industry large model according to the correlation between the problem and the power industry standard business scene; performing intention query and routing of various standard questions based on the questions, and distributing the questions to corresponding standard service scene models for processing; different standard service scene modules perform processing according to different types of problems, perform standard problem query of corresponding routing scenes according to the problems to obtain context reference, and send the context reference to the power industry large model; and the power industry large model judges and generates an answer according to the question. Different types of questions can be automatically answered according to user questions, standard and power grid business fusion is promoted, and the guiding effect of technical standards on power grid grassroots is enhanced.

Description

Technical Field

[0001] This application relates to the technical field of power knowledge issues, and particularly to a method for answering power standard knowledge questions and related devices. Background Art

[0002] A standard is a document that is developed through standardization activities, negotiated and agreed upon in accordance with specified procedures, and provides rules, guidelines, or characteristics for various activities or their results for common and repeated use. However, the low level of intelligence in the current application of standards limits the working efficiency of power grid management and operation. There are problems such as the mismatch between standard clause knowledge and business processes, low search efficiency of personnel, and difficulties in integrating operation scenarios during the processes of standard compilation, implementation, search, testing, and verification for front-line business personnel of the company. The business scenarios involve a large number of systems and different roles, and it is difficult to efficiently associate fine-grained standard knowledge with system status data and role process information, resulting in difficulties in integrating standard knowledge into business scenarios, which are mainly reflected in the following aspects: First, there is an obvious mismatch between standard clause knowledge and the actual business processes of power grid management and operation. With the increasing complexity of power grid systems and the growing demand for intelligence, traditional standard-making methods often struggle to quickly adapt to the rapidly changing business environment, leading to standard clauses lagging behind actual operation requirements, thereby affecting the execution efficiency and effectiveness of standards.

[0003] Second, business personnel face low efficiency when conducting standard retrieval. Due to the large number and variety of standard documents involved in power grid management, coupled with the lack of efficient and convenient retrieval tools and methods, front-line personnel spend a large amount of time and effort in searching for specific standard clauses, which undoubtedly reduces work efficiency and increases the risk of operation errors.

[0004] Furthermore, there are numerous difficulties in integrating standard knowledge with operation scenarios. Power grid management and operation involve collaborative operations of multiple system platforms, and each system has its own unique data format and operation process. At the same time, the standard knowledge required by personnel in different roles during the operation process also varies. However, the existing standard system often fails to achieve efficient association and accurate push of fine-grained standard knowledge, resulting in an inaccurate correspondence between standard knowledge and actual operation scenarios. Summary of the Invention

[0005] In view of problems such as low efficiency in finding technical conditions of standard clauses, poor integration of standard knowledge with on-site intelligence, and model training of semantic routing, this application studies a power standard knowledge question understanding module and a standard knowledge question-answering system. This system consists of a standard question semantic routing module and several standard question-answering modules, which can automatically answer different types of questions according to user questions, promote the integration of standards and power grid operations, and strengthen the guiding role of technical standards for the grass-roots level of the power grid.

[0006] To solve the above technical problems, the technical solution adopted in this application is as follows: In a first aspect, this application provides a power standard knowledge Q&A system, including: An input end, which is used to obtain the questions asked by users; A semantic routing module, which is used to determine whether to distribute to a standard business scenario model or a power industry large model according to the relevance between the question and the power industry standard business scenario; and perform intent queries and routing of multiple standard questions based on the question, and distribute the question to the corresponding standard business scenario model for processing; A standard business scenario model, where different standard business scenario modules process according to different types of questions, and are used to query standard questions in the corresponding routing scenario according to the question, obtain a context reference, and send the context reference to the power industry large model; A power industry large model, which is used to judge according to the question. If the question has no context reference, it directly generates an answer; if there is a context reference, it generates an answer according to the context reference; the power industry large model is a power industry base large language model pre-trained with power industry knowledge.

[0007] As a further improvement of the present invention, the semantic routing module is specifically used for: If the question is not relevant to the standard business scenario, the semantic routing module will not distribute the question, and the question will be directly sent to the power industry large model; If the question is relevant to the standard business scenario, the semantic routing module will distribute the question to the corresponding standard business scenario module; Different standard business scenario modules process according to different types of questions, and after processing, obtain a context reference and send it to the power industry large model.

[0008] As a further improvement of the present invention, the different standard business scenario modules process according to different types of questions, including: If the question needs to be compared with standard difference clauses, query using the standard knowledge graph according to the question, and after querying, return the entity-level knowledge of the standard fine-grainedness, and return the entity knowledge as the context reference of the question to the power industry large model; If the question needs standard knowledge Q&A, retrieve using the retrieval vector library according to the question, and after retrieval, sort to obtain the answers to several questions with high similarity as the standard clause knowledge, and return the standard clause knowledge as the context reference of the question to the power industry large model; If the question needs standard knowledge classification, generate an instruction project using the instruction template according to the question, and then splice the classification keywords through the instruction project as the context reference of the question and return it to the power industry large model.

[0009] As a further improvement of the present invention, the semantic routing module is trained by a large language model and is used to classify user questions; the training method of the large language model includes: Constructing dataset data through annotation, and the form of the dataset data is question-answer pairs; Expanding the magnitude of the dataset data; Training the large language model with the expanded dataset data, and making the large language model adapt to the standard question classification task based on the curriculum learning method.

[0010] As a further improvement of the present invention, the training method further includes a hard example mining method, including: The large language model performs supervised first-stage training according to the full-scale dataset, and the training makes the loss function gradient decrease; After the supervised first-stage training, stop the training, and perform hard example mining according to the following strategy: After the end of the first-stage training, in the form of interface calls, keep other parameters frozen and only adjust the generation text control parameter temperature of the large language model, so that the sampling parameter range of temperature needs to cover 0 to 1 and should not exceed 1, and repeat k comparison experiments; Calculate the accuracy rates of M categories under the results of k comparison experiments; for a certain category i, if the classification accuracy rates of the results of k comparison experiments are all higher than the first threshold, then delete the samples of this category from the dataset, and then screen out hard samples; Update the screened hard samples into a new dataset and perform supervised second-stage training; During the second-stage training process, if the loss function is lower than the second threshold, the training ends.

[0011] As a further improvement of the present invention, in the training where the loss function gradient decreases, the loss function is:

[0012] In the formula: is the training loss value of the full parameters of the model in the k-th stage, is the full-scale parameter, is the dataset in the k-th stage, and (x, y) is the input and target text sequence pair.

[0013] In a second aspect, the present application provides a method for answering questions about power standard knowledge, including: Obtaining the question asked by the user; The semantic routing module determines whether to distribute the question to the standard business scenario model or the large model of the power industry based on the relevance between the question and the standard business scenarios of the power industry; and performs intent queries and routing for multiple standard questions based on the question, and distributes the question to the corresponding standard business scenario model for processing; Different standard business scenario modules process different types of questions, perform standard question queries for the corresponding routing scenarios according to the question, obtain context references, and send the context references to the large model of the power industry; The large model of the power industry makes a judgment based on the question. If the question has no context reference, it directly generates an answer; if there is a context reference, it generates an answer based on the context reference; the large model of the power industry is a large language model of the power industry base pre-trained with power industry knowledge.

[0014] Optionally, the semantic routing module determines whether to distribute the question to the standard business scenario model or the large model of the power industry based on the relevance between the question and the standard business scenarios of the power industry, including: If the question is not relevant to the standard business scenario, the semantic routing module will not distribute the question, and the question will be directly sent to the large model of the power industry; If the question is relevant to the standard business scenario, the semantic routing module will distribute the question to the corresponding standard business scenario module; Different standard business scenario modules process different types of questions, and after processing, obtain context references and send them to the large model of the power industry.

[0015] Optionally, the processing of different standard business scenario modules according to different types of questions includes: If the question needs to be compared with the standard difference clauses, query using the standard knowledge graph according to the question, and after querying, return the standard fine-grained entity-level knowledge, and return the entity knowledge as the context reference of the question to the large model of the power industry; If the question requires standard knowledge Q&A, retrieve using the retrieval vector library according to the question, and after retrieval, obtain the answers to several questions with high similarity in ranking as the standard clause knowledge, and return the standard clause knowledge as the context reference of the question to the large model of the power industry; If the question requires standard knowledge classification, generate an instruction project using the instruction template according to the question, and then splice the classified keywords through the instruction project as the context reference of the question and return it to the large model of the power industry.

[0016] Optionally, the semantic routing module is trained by a large language model for classifying user questions; the training method of the large language model includes: Construct a dataset by annotation, and the form of the dataset is question-answer pairs; Expand the scale of the dataset data; Train the large language model with the expanded dataset data, and adapt the large language model to the standard question classification task based on the curriculum learning method.

[0017] Optionally, the training method further includes a hard example mining method, including: The large language model performs supervised first-stage training based on the full dataset, and the training makes the loss function gradient decrease; After the supervised first-stage training, stop the training, and perform hard example mining according to the following strategy: After the end of the first-stage training, in the form of interface call, keep other parameters frozen and only adjust the generation text control parameter temperature of the large language model. On the premise that the sampling parameter range of temperature needs to cover 0 to 1 and should not exceed 1, repeat k comparison experiments; Calculate the accuracy of M categories under the results of k comparison experiments; for a certain category i, if the classification accuracy of the results of k comparison experiments is higher than the first threshold, delete the samples of this category from the dataset, and then screen out hard samples; Update the screened hard samples to a new dataset and perform supervised second-stage training; During the second-stage training, if the loss function is lower than the second threshold, the training ends.

[0018] Optionally, in the training where the loss function gradient decreases, the loss function is:

[0019] In the formula: is the training loss value of the full parameters of the model in the k-th stage, are the full parameters, is the dataset in the k-th stage, and (x, y) is the input and target text sequence pair.

[0020] In a third aspect, the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the power standard knowledge question answering method is implemented.

[0021] In a fourth aspect, the present application provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the power standard knowledge question answering method is implemented.

[0022] Fifth aspect, the present application provides a computer program product, the computer program product includes computer instructions, and the computer instructions direct a computer to execute the power standard knowledge Q&A method.

[0023] The beneficial effects of the present application compared with the prior art are as follows: Through the semantic routing module, the Q&A system of the present application can quickly parse the input content of the user, automatically identify the user's intention, call the corresponding standard scenario model, obtain context references. The power industry large model makes a judgment based on the question. If the question has no context reference, it directly generates an answer; if there is a context reference, it generates an answer based on the context reference and returns the user result, improving the efficiency of information retrieval and solving various needs of grass-roots grid employees for standard use in one stop.

[0024] Furthermore, by retraining the semantic routing module, the usage scenarios supported by the Q&A system can be dynamically adjusted, improving the flexibility and adaptability of the Q&A system.

[0025] Furthermore, a difficult example mining method based on generation uncertainty is also proposed. This mining method is designed for large language models, improving the model's ability to process complex training data and enhancing the robustness and reliability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following introduces the accompanying drawings of the relevant technical solutions in the embodiments of the present application or the prior art. It should be understood that the accompanying drawings below only facilitate a clear description of some embodiments of the technical solutions in the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative efforts.

[0027] Figure 1 is a schematic diagram of the standard Q&A system of the present application; Figure 2 is the training process of the semantic routing module of the present application; Figure 3 is the dynamic self-adjusting negative example training method of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0028] Embodiments of the present application will be described in detail below. Examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals denote the same or similar elements or elements with the same or similar functions throughout. The embodiments described below by referring to the accompanying drawings are exemplary and are only used to explain the present application and should not be construed as a limitation to the present application. For the step numbers in the following embodiments, they are only set for the convenience of elaboration and explanation, and no limitation is imposed on the order between the steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0029] In the description of the present application, unless otherwise clearly defined, words such as "set", "installed", "connected", etc. should be understood in a broad sense. Those skilled in the art can reasonably determine the specific meanings of the above words in the present application in combination with the specific content of the technical solution.

[0030] In order to elaborate the technical solution of the present application in detail, the content of the present application will be described in detail below with reference to the accompanying drawings: Figure 1 For the work process of a power standard knowledge Q&A system, the present application consists of the following parts, specifically including: An input end, which is used to obtain the questions asked by users; A power industry large model: which is used to make a judgment according to the question. If the question has no context reference, it directly generates an answer; if there is a context reference, it generates an answer according to the context reference; a power industry base large language model pre-trained with power industry knowledge. The power industry large model combines data and calculations in multiple aspects such as power network theory, power market analysis, and climate impact. By collecting, processing, and analyzing a large amount of power data, it realizes the comprehensive monitoring and accurate prediction of the power system. These models usually have high scalability and flexibility and can adapt to the power system requirements of different scales and different scenarios.

[0031] A semantic routing module: which is used to determine whether to distribute to a standard business scenario model or a power industry large model according to the relevance between the question and the power industry standard business scenario; and perform intention query and routing of multiple standard questions based on the question, and distribute the question to the corresponding standard business scenario model for processing; realize the intention query and routing of multiple standard questions and distribute the question to the corresponding process for processing.

[0032] Several standard business scenario models: Different standard business scenario modules process according to different types of questions, which are used to query standard questions in the corresponding routing scenario according to the question, obtain a context reference, and send the context reference to the power industry large model.

[0033] The standard business scenario model includes but is not limited to: A standard knowledge graph for comparing with standard difference terms; A standard vector library for standard knowledge Q&A; Instruction template generation for standard knowledge classification.

[0034] Specifically, Figure 1 In [the document], the models corresponding to 3 standard business scenarios are taken as examples, which can be increased or decreased according to actual needs. The specific process is as follows: Step 1, the user asks a question; the user asks a standard-related question at the input end of the Q&A system, and the question will be distributed to both the power industry large model and the semantic routing module simultaneously.

[0035] Step 2, the semantic routing module distributes according to the user's question to different processes for processing.

[0036] 1) If the user's question is not related to the standard business scenario, at this time the semantic routing module will not distribute, and the power industry large model will directly answer and directly jump to 4.

[0037] 2) If the user's question meets the usage scope of a certain scenario model, the semantic routing module will distribute the standard question to the processing process corresponding to the standard business scenario.

[0038] Step 3, taking three different standard business scenarios as examples, according to different types of questions, enter the following processing processes: 1) Standard difference term comparison: For this type of question, the standard knowledge graph needs to be used, and after querying, the standard fine-grained entity-level knowledge is returned. The entity knowledge will be returned to the power industry large model as the context reference of the question for the large language model to output an answer.

[0039] 2) Standard knowledge Q&A: This type of question mainly uses the retrieval vector library. Through vector retrieval, the answers to several questions that are the most similar in sorting are obtained. The retrieved standard clause knowledge will be returned to the power industry large model as the context reference of the question for the large language model to output an answer.

[0040] 3) Standard knowledge classification: This type of question mainly uses instruction engineering. Through instruction engineering, the classified keywords are spliced and returned to the power industry large model as the context reference of the question for the large language model to output an answer.

[0041] Step 4, if there is no context reference, the power industry large model will directly generate an answer; if there is, it will generate an answer according to the context reference returned in Step 3.

[0042] This semantic routing module consists of a large language model with 7B parameters (the number of parameters can be replaced with a larger number of parameters of a large language model), and this 7B large language model is trained to classify user questions.

[0043] Large language models with 7B parameters refer to large models (Large Language Models, LLMs) with a parameter scale of 7 billion in the field of artificial intelligence. They are used to handle complex natural language understanding and generation tasks. These parameters include weights and biases in the neural network, which are adjusted through the backpropagation algorithm during model training to capture the correlations and patterns in the input data (such as text, images), and have the ability of language understanding and generation.

[0044] For the specific training methods and processes, see Figure 2 as shown, there are a total of 3 steps. The specific steps are as follows: S1, Construct a dataset: Construct dataset data through expert annotation. The form reference is question-answer pairs, and the structure is shown in Table 1.

[0045] Table 1

[0046] S2, Data augmentation: Based on the large language model, perform a magnitude expansion on the positive example data constructed in step 1. The advantage of this step is to enhance the diversity and balance of the data and avoid the drift towards a certain category distribution during training.

[0047] S3, Fine-tuning of the large language model: The fine-tuning process of the large language model uses the dataset in step 2 for further training to make the large language model adapt to the standard question classification task. Specifically, for the input shown as "instruction" + "input" in Table 1, the large language model continuously predicts the next character to generate the "output" process.

[0048] Regarding the problem of inaccurate training results caused by insufficient difficult examples during the training process, and the problem that the standard semantic routing model has a long training time and is prone to falling into local optima, the semantic routing module of this application dynamically proposes a difficult example mining method based on generation uncertainty, which realizes the reduction of training time and the improvement of model robustness by mining difficult examples during the training process.

[0049] Specifically, a difficult example mining method based on generation uncertainty, the training method is shown in Figure 3 as shown.

[0050] Figure 3 It is a dynamic self-adjusting negative example training method, and the specific method is as follows: Step 31: The model performs supervised first-stage training (abbreviated as sft) based on the full dataset in Step 2. The methods include but are not limited to LoRA and QLoRA. Among them, LoRA, short for Low-Rank Adaptation, is a parameter-efficient fine-tuning technique for large language models (LLMs). This technique was developed by Semtech and aims to reduce the computational resources and storage space required for fine-tuning large language models.

[0051] QLoRA, short for Quantized Low-Rank Adaptation, is a technique that combines model quantization and low-rank adaptation. It introduces quantization technology on the basis of LoRA to further reduce memory and computational overhead.

[0052] The training direction is along the direction of the gradient descent of the loss function. The loss function is shown in Equation 1.

[0053] is the training loss value of the full parameters of the model in the k-th stage, are the full parameters. is the dataset in the k-th stage. (x, y) is the input and target text sequence pair.

[0054]

[0055] Step 32: Stop training after 100 (this number can be adjusted according to the total amount of training data) training steps (abbreviated as epochs in English), and perform hard example mining according to the following strategy: The following content involves the control parameter of the text generated by the large language model, temperature (hereinafter referred to as temperature): Temperature is a parameter used to control the creativity level of the text generated by artificial intelligence, and its range is between 0 and 2.

[0056] After the first-stage training is completed, in the form of interface calls, other parameters are frozen and unchanged, and only the control parameter temperature of the text generated by the large language model is adjusted. On the premise that the sampling parameter range of temperature needs to cover 0 to 1 and should not exceed 1, repeat k comparison experiments. The setting of temperature should not exceed 1. In the case of exceeding 1, the text generated by the large language model may be too active and irrelevant to the question.

[0057] Calculate the accuracy of M categories under the results of k comparison experiments. For a certain category i, if the classification accuracy of the results of k comparison experiments is higher than a certain first threshold α (α can be set according to the accuracy requirement, and in this system, it is set to 90%), it proves that the current model weights have good classification effect on category i, then the samples of this category are deleted from the dataset. Through this method, difficult samples are screened out for the second-stage pre-training.

[0058] By mining difficult samples, it can help the model better learn how to distinguish various category samples, reduce the training time at the same time, and avoid the training results from deviating in a certain direction.

[0059] The following takes Figure 1 the system shown as an example to explain the specific solution of this application: Step 321: There are 3 routing scenarios in System 1, plus 1 branch that directly allows the large language model to answer, a total of 4 categories: "Differential Clause Comparison", "Standard Knowledge Q&A", "Standard Knowledge Classification", "Large Language Model Direct Answer". After the first-stage training is completed, 5 repeated experiments of generating uncertainty are carried out. Suppose the several temperatures are: 0.01, 0.2, 0.5, 0.8, 1, then calculate the accuracy of 4 categories under the results of 5 comparison experiments.

[0060] Step 322: For category 4, that is, "Directly generate an answer by the large language model", the performance of the 4 comparison experiments is about 92%, then this category is deleted from the dataset, and then the second-stage pre-training is carried out.

[0061] Step 323: Update the mined difficult examples to a new dataset and conduct the second-stage training. This task is a classification task, and the training function refers to Formula 1.

[0062] Step 324: When the loss function is lower than the second threshold Յ, the training ends.

[0063] The semantic routing module of this application accurately understands the user's question through semantic parsing and provides a context reference index of relevant standard content to the large language model. Combining with the powerful generation ability of the large language model, the semantic routing module can significantly improve the flexibility, adaptability and standard query efficiency of the question-answering system. Based on the method of mining difficult samples based on generation uncertainty, difficult samples are mined by adjusting the temperature parameter of the large language model. By training the difficult samples separately, the generalization ability of the model is improved, the model convergence is accelerated, and the robustness of the model is improved.

[0064] The second object of this application is to provide a method for answering questions about power standard knowledge. Based on the above-mentioned power standard knowledge question-answering system, the method includes: S1, obtain the question asked by the user; S2. The semantic routing module determines whether to distribute the question to the standard business scenario model or the large power industry model based on the relevance between the question and the standard business scenarios in the power industry; and performs intent queries and routing for multiple standard questions based on the question, and distributes the question to the corresponding standard business scenario model for processing; Optionally, the semantic routing module determines whether to distribute the question to the standard business scenario model or the large power industry model based on the relevance between the question and the standard business scenarios in the power industry, including: If the question is not relevant to the standard business scenario, the semantic routing module will not distribute the question, and the question will be directly sent to the large power industry model; If the question is relevant to the standard business scenario, the semantic routing module will distribute the question to the corresponding standard business scenario module; Different standard business scenario modules process different types of questions, and after processing, obtain context references and send them to the large power industry model.

[0065] S3. Different standard business scenario modules process different types of questions, perform standard question queries for the corresponding routing scenarios based on the questions, obtain context references, and send the context references to the large power industry model; Optionally, the different standard business scenario modules process different types of questions, including: If the question needs to be compared with the standard difference clauses, query using the standard knowledge graph according to the question, and after querying, return the entity-level knowledge at the standard fine-grained level, and use the entity knowledge as the context reference of the question and return it to the large power industry model; If the question requires standard knowledge Q&A, retrieve using the retrieval vector library according to the question, and after retrieval, obtain the answers to several questions with high similarity in ranking as the standard clause knowledge, and use the standard clause knowledge as the context reference of the question and return it to the large power industry model; If the question requires standard knowledge classification, generate an instruction project using the instruction template according to the question, and then splice the classification keywords through the instruction project as the context reference of the question and return it to the large power industry model.

[0066] Optionally, the semantic routing module is trained by a large language model for classifying user questions; the training method of the large language model includes: Construct dataset data through annotation, and the form of the dataset data is question-answer pairs; Expand the scale of the dataset data; Train the large language model with the expanded dataset data, and make the large language model adapt to the standard question classification task based on the curriculum learning method.

[0067] Optionally, the training method further includes a hard example mining method, including: The large language model performs supervised first-stage training based on the full dataset, and the training makes the loss function gradient descend. After the supervised first-stage training, stop the training and perform hard example mining according to the following strategy: After the end of the first-stage training, in the way of interface call, with other parameters frozen and unchanged, only adjust the generation text control parameter temperature of the large language model, so that the sampling parameter range of temperature needs to cover 0 to 1 and should not exceed 1, and repeat k comparison experiments. Calculate the accuracy rates of M categories under the results of k comparison experiments; for a certain category i, if the classification accuracy rates of the results of k comparison experiments are all higher than the first threshold, then delete the samples of this category from the dataset, and then screen out the hard samples. Update the screened hard samples into a new dataset and perform supervised second-stage training. During the second-stage training, if the loss function is lower than the second threshold, the training ends.

[0068] Optionally, in the training where the loss function gradient descends, the loss function is:

[0069] In the formula: is the training loss value of all parameters of the model in the k-th stage, is all parameters, is the dataset in the k-th stage, and (x, y) is the input and target text sequence pair.

[0070] S4. The large model in the power industry judges according to the problem. If the problem has no context reference, directly generate an answer; if there is a context reference, generate an answer according to the context reference; the large model in the power industry is a large language model for the power industry base pre-trained with power industry knowledge.

[0071] The third object of the embodiment of the present application is to provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the above-mentioned power standard knowledge question-answering method is implemented. It also includes a communication interface and a bus.

[0072] The fourth object of the embodiment of the present application is to provide a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned power standard knowledge question-answering method is implemented.

[0073] The fifth object of the embodiments of the present application is to provide a computer program product, which includes computer instructions that direct a computer to execute the above-mentioned power standard knowledge Q&A method.

[0074] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured article including an instruction device, and the instruction device realizes the functions specified in one process Figure 1 one process or multiple processes and / or blocks Figure 1 or multiple blocks.

[0075] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Therefore, the instructions executed on the computer or other programmable device provide steps for realizing the functions specified in one process Figure 1 one process or multiple processes and / or blocks Figure 1 or multiple blocks.

[0076] The present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memory, readable storage media, optical memory, etc.) containing computer-usable program code.

[0077] The present application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate a device for realizing the functions specified in one process Figure 1 one process or multiple processes and / or blocks Figure 1 or multiple blocks.

[0078] Obviously, the described embodiments are only partial embodiments of the present application, rather than all embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative efforts shall fall within the protection scope of the present application.

[0079] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not intended to limit them. Although the present application has been described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that: modifications or equivalent replacements can still be made to the specific implementation manners of the present application, and any modifications or equivalent replacements that do not depart from the spirit and scope of the present application shall be covered by the protection scope of the claims of the present application.

Claims

1. A power standard knowledge Q&A system, characterized in that Including: An input end for obtaining the question asked by the user; A semantic routing module for determining to distribute to a standard business scenario model or a large model of the power industry according to the relevance between the question and the standard business scenarios of the power industry; and performing intent query and routing of multiple standard questions based on the question, and distributing the question to the corresponding standard business scenario model for processing; Standard business scenario models, where different standard business scenario modules process according to different types of questions, and are used to query standard questions for the corresponding routing scenarios based on the question, obtain context references, and send the context references to the large model of the power industry; The large model of the power industry is used to make a judgment based on the question. If there is no context reference for the question, an answer is directly generated; If there is a context reference, an answer is generated according to the context reference; The large model of the power industry is a large language model of the power industry base pre-trained with power industry knowledge.

2. The power standard knowledge Q&A system according to claim 1, characterized in that, The semantic routing module is specifically used for: If the question is not relevant to the standard business scenario, the semantic routing module will not distribute the question, and the question is directly sent to the large model of the power industry; If the question is relevant to the standard business scenario, the semantic routing module distributes the question to the corresponding standard business scenario module; Different standard business scenario modules process according to different types of questions, and after processing, obtain context references and send them to the large model of the power industry.

3. The power standard knowledge Q&A system according to claim 1, characterized in that, In the standard business scenario model, different standard business scenario modules process according to different types of questions, including: If the question needs to be compared with standard difference clauses, query using the standard knowledge graph according to the question, and after querying, return the entity-level knowledge of the standard fine-grained level, and use the entity knowledge as the context reference of the question and return it to the large model of the power industry; If the question needs standard knowledge Q&A, retrieve using the retrieval vector library according to the question, and after retrieval, obtain the answers to several questions with high similarity in sorting as the standard clause knowledge, and use the standard clause knowledge as the context reference of the question and return it to the large model of the power industry; If the question needs standard knowledge classification, generate an instruction project using the instruction template according to the question, and then splice the classification keywords through the instruction project as the context reference of the question and return it to the large model of the power industry.

4. A power standard knowledge Q&A system according to claim 1, characterized in that, The semantic routing module is used to classify user questions after being trained by a large language model; The training method of the large language model includes: Constructing dataset data through annotation, and the form of the dataset data is question-answer pairs; Expanding the magnitude of the dataset data; Training the large language model with the expanded dataset data, and making the large language model adapt to the standard question classification task based on the curriculum learning method.

5. The power standard knowledge Q&A system according to claim 4, characterized in that, The training method also includes a hard example mining method, including: The large language model performs supervised first-stage training according to the full-scale dataset, and the training makes the loss function gradient decrease; After the supervised first-stage training, stop training, and perform hard example mining according to the following strategy: After the end of the first-stage training, in the form of interface calls, with other parameters frozen and unchanged, only the generation text control parameter temperature of the large language model is adjusted. On the premise that the sampling parameter range of temperature needs to cover 0 to 1 and should not exceed 1, repeat k comparison experiments; Calculate the accuracy rates of M categories under the results of k comparison experiments; for a certain category i, if the classification accuracy rates of the results of k comparison experiments are all higher than the first threshold, then delete the samples of this category from the dataset, and then screen out difficult samples; Update the screened difficult samples as a new dataset and conduct supervised second-stage training; During the second-stage training, if the loss function is lower than the second threshold, the training ends.

6. The power standard knowledge Q&A system according to claim 4, characterized in that, In the training where the loss function decreases in gradient, the loss function is: Where: is the training loss value of the full model parameters in the k-th stage, are the full parameters, is the dataset in the k-th stage, and (x, y) is the input and target text sequence pair.

7. A method for answering questions about power standard knowledge, characterized in that, Including: Obtain the question asked by the user; The semantic routing module determines to distribute to the standard business scenario model or the power industry large model according to the relevance between the question and the standard business scenarios of the power industry; and conducts intention queries and routing of multiple standard questions based on the question, and distributes the question to the corresponding standard business scenario model for processing; Different standard business scenario modules process according to different types of questions, query standard questions for the corresponding routing scenarios according to the question, obtain context references, and send the context references to the power industry large model; The power industry large model makes a judgment according to the question. If there is no context reference for the question, it directly generates an answer; If there is a context reference, it generates an answer according to the context reference; The power industry large model is a power industry base large language model pre-trained with power industry knowledge.

8. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the power standard knowledge question and answer method described in claim 7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, it implements the power standard knowledge question and answer method described in claim 7.

10. A computer program product, the computer program product comprising computer instructions, characterized in that, The computer instructions instruct the computer to execute the power standard knowledge question and answer method described in claim 7.

Images