Interaction method and device of large language model based on electric power knowledge and electronic equipment

Abstract

The invention provides an interaction method and device of a large language model based on electric power knowledge and electronic equipment. The method comprises the steps that power field keywords are obtained, retrieval is conducted according to the power field keywords, and explanation information of the power field keywords is obtained; a large language model is obtained, instruction fine adjustment data are generated through the large language model according to the interpretation information, the large language model is obtained through training of multiple sets of language data in the power field, and the instruction fine adjustment data are data for adjusting parameters of the large language model; and evaluating the instruction fine-tuning data to obtain an evaluation score, determining high-quality instruction fine-tuning data according to the evaluation score, training the large language model through the high-quality instruction fine-tuning data to obtain a power field large language model, and performing interaction through the power field large language model. Through the method, the problem that the user interaction effect is poor due to the fact that an existing large language model cannot understand terminologies in the vertical field of the power industry is solved.

Description

Technical Field

[0001] The present application relates to the technical field of natural language processing in the electric power field, and in particular to an interactive method, device, computer-readable storage medium and electronic device based on a large language model of electric power knowledge. Background Art

[0002] In recent years, large language models have made breakthrough progress in the field of natural language processing, and their performance in tasks such as text generation, question answering, and dialogue has continued to surpass human levels. Through large-scale unsupervised pre-training, these models have learned rich semantic and grammatical knowledge and can be flexibly applied to various natural language understanding and generation tasks. With the continuous improvement of the performance of large language models, they have gradually been applied to various industry verticals, such as electricity, finance, medical care, and law, bringing new opportunities and challenges to these fields. However, compared with the massive data in general fields, there is a major problem of scarcity of high-quality annotated data in industry verticals. This makes it common for large language models to have very serious hallucination problems when they are applied to downstream tasks in industry verticals. This is usually because the large language models trained on general scenario corpora do not understand the professional terms in the industry verticals. For example, when asking questions about the power scenario, "What is the defect of SF6 gas pressure being lower than the standard value and no prohibition sign being hung?", because the large model does not understand the power professional term "SF6 gas pressure" in the question, it may lead to incorrect or even dangerous answers and suggestions.

[0003] In order to solve the problem of scarcity of labeled data for large language models in industry verticals, existing methods can be divided into two categories. One is to hire annotators to perform data labeling in the form of crowdsourcing, and the other is to use the powerful generation capabilities of closed-source large language models (such as ChatGPT, Wenxinyiyan, etc.) to generate instruction fine-tuning data. Based on manually labeled industry vertical data, strict manual review can avoid possible labeling errors and ensure the lower limit of data quality; however, manual labeling often has problems such as huge costs and long time consumption. By designing prompt words to guide closed-source large language models to use their powerful generation capabilities to effectively generate instruction fine-tuning data, this method alleviates the problems of labeling cost and time consumption of manual labeling, and is more efficient and lower cost than manual labeling; however, large language models trained on general domain data often have hallucinations when generating industry vertical domain instruction data because they do not understand professional terms, thereby generating obviously low-quality vertical domain fine-tuning data.

[0004] The existing semi-automatic method of using large language models for data synthesis has been widely used in various tasks and scenarios such as improving instruction following, mathematical ability, reasoning ability and code generation, showing great potential. Therefore, how to apply the large model data synthesis method to vertical industries such as electricity and automatically construct high-quality, diversified, and large-scale industry vertical instruction fine-tuning data is an important issue for the application of large language models in vertical industries such as electricity. Summary of the invention

[0005] The main purpose of this application is to provide an interactive method, device, computer-readable storage medium and electronic device based on a large language model of power knowledge, so as to at least solve the problem in the prior art that the large language model cannot understand the professional terms in the vertical field of the power industry, resulting in poor user interaction effect.

[0006] In order to achieve the above-mentioned purpose, according to one aspect of the present application, an interactive method of a large language model based on electric power knowledge is provided, including: obtaining electric power field keywords, searching according to the electric power field keywords, and obtaining explanation information of the electric power field keywords, wherein the electric power field keywords are professional terms in the electric power field, and the explanation information is information explaining the professional terms; obtaining a large language model, generating instruction fine-tuning data according to the explanation information through the large language model, wherein the large language model is obtained by training with multiple groups of language data in the electric power field, and the instruction fine-tuning data is data for adjusting the parameters of the large language model; evaluating the instruction fine-tuning data to obtain an evaluation score, determining high-quality instruction fine-tuning data according to the evaluation score, and training the large language model through the high-quality instruction fine-tuning data to obtain a large language model in the electric power field, and interacting through the large language model in the electric power field.

[0007] Optionally, obtaining electric power field keywords, searching based on the electric power field keywords, and obtaining explanatory information of the electric power field keywords, including: obtaining an electric power knowledge graph, and obtaining the electric power field keywords from the electric power knowledge graph; calling a search engine to search based on the electric power field keywords to obtain web page text, wherein the web page text contains explanatory text information of the electric power field keywords; summarizing the explanatory text information in the web page text to obtain the explanatory information of the electric power field keywords.

[0008] Optionally, generating instruction fine-tuning data according to the interpretation information through the large language model includes: obtaining a prompt word template, wherein the prompt word template represents a template for generating the instruction fine-tuning data and at least includes an output format of the instruction fine-tuning data; generating the instruction fine-tuning data according to the prompt word template based on the interpretation information through the large language model.

[0009] Optionally, the instruction fine-tuning data is evaluated to obtain an evaluation score, including: obtaining a pre-trained model, and evaluating the accuracy of the instruction fine-tuning data through the pre-trained model to obtain an accuracy score, wherein the pre-trained model is trained by the power field keywords and the corresponding interpretation information; obtaining input instructions and output data corresponding to the instruction fine-tuning data, and calculating the correlation between the input instructions and the output data to perform a correlation evaluation to obtain a correlation score; determining the integrity of the instruction fine-tuning data to perform an integrity evaluation to obtain an integrity score; determining the difficulty of the instruction fine-tuning data to perform a difficulty evaluation to obtain a difficulty score.

[0010] Optionally, determining high-quality instruction fine-tuning data based on the evaluation score includes: obtaining an accuracy score threshold corresponding to the accuracy evaluation, a relevance score threshold corresponding to the relevance evaluation, an integrity score threshold corresponding to the integrity evaluation, and a difficulty threshold corresponding to the difficulty evaluation; and determining that the instruction fine-tuning data whose accuracy score is greater than the accuracy score threshold, the relevance score is greater than the relevance score threshold, the integrity score is greater than the integrity score threshold, and the difficulty score is greater than the difficulty threshold is the high-quality instruction fine-tuning data.

[0011] Optionally, the large language model is trained using the high-quality instruction fine-tuning data to obtain a large language model in the power field, including: calculating the output vector of the large language model using a low-rank adaptive method h=W0·x+ΔW·x, wherein h represents the output vector, x represents the input vector, W0 represents the weight value matrix, and ΔW represents the parameters adjusted in the large language model.

[0012] Optionally, the large language model is trained by fine-tuning the high-quality instruction data to obtain a large language model for the power field, further comprising: Calculate the loss function of the large language model in the power field, where M represents the total number of words in the large language model in the power field, y o,c represents the binary function of the observation o and the cth word, p o,c represents the predicted probability that the observation o belongs to class c, L CE Represents the loss value of the loss function.

[0013] According to another aspect of the present application, an interactive device based on a large language model of electric power knowledge is provided, comprising: a retrieval unit, used to obtain electric power field keywords, search according to the electric power field keywords, and obtain explanation information of the electric power field keywords, wherein the electric power field keywords are professional terms in the electric power field, and the explanation information is information explaining the professional terms; a generation unit, used to obtain a large language model, generate instruction fine-tuning data according to the explanation information through the large language model, wherein the large language model is obtained by training with multiple groups of language data in the electric power field, and the instruction fine-tuning data is data for adjusting parameters of the large language model; a training unit, used to evaluate the instruction fine-tuning data to obtain an evaluation score, determine high-quality instruction fine-tuning data according to the evaluation score, and train the large language model through the high-quality instruction fine-tuning data to obtain a large language model in the electric power field, and interact through the large language model in the electric power field.

[0014] According to another aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes a stored program, wherein when the program is run, the device where the computer-readable storage medium is located is controlled to execute any one of the interactive methods based on the large language model of power knowledge.

[0015] According to another aspect of the present application, an electronic device is provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include an interactive method for executing any one of the large language models based on power knowledge.

[0016] Apply the technical solution of the present application, obtain the electric power field keywords, search according to the electric power field keywords, obtain the explanation information of the electric power field keywords, the electric power field keywords are professional terms in the electric power field, generate instruction fine-tuning data according to the explanation information through the large language model, and adjust the parameters of the large language model; evaluate the instruction fine-tuning data to obtain the evaluation score, determine the high-quality instruction fine-tuning data according to the evaluation score, and train the large language model through the high-quality instruction fine-tuning data to obtain the large language model in the electric power field, and interact through the large language model in the electric power field. Compared with the prior art, in which the large language model cannot understand the professional terms of the electric power industry, the present application generates instruction fine-tuning data through the electric power field keywords, supplements and adjusts the professional terms in the vertical field of the electric power industry in the large language model, so that the large language model interacts on the basis of fully understanding the professional terms in the electric power vertical industry. Therefore, it can solve the problem that the existing large language model cannot understand the professional terms in the vertical field of the electric power industry, resulting in poor user interaction effect, and achieve the technical effect of answering questions in the electric power field accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings constituting part of the present application are used to provide a further understanding of the present application. The exemplary embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0018] Figure 1 A hardware structure block diagram of a mobile terminal for executing an interactive method of a large language model based on electric power knowledge provided by an embodiment of the present application is shown;

[0019] Figure 2 A schematic diagram of a flow chart of an interactive method based on a large language model of electric power knowledge provided in an embodiment of the present application is shown;

[0020] Figure 3 A flowchart of a specific interaction method of a large language model based on power knowledge provided in an embodiment of the present application is shown;

[0021] Figure 4 A structural block diagram of an interactive device based on a large language model of electric power knowledge provided in an embodiment of the present application is shown.

[0022] The above drawings include the following reference numerals:

[0023] 102, processor; 104, memory; 106, transmission device; 108, input and output devices. DETAILED DESCRIPTION

[0024] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0025] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0026] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0027] As introduced in the background technology, the large language model in the prior art cannot understand the professional terminology in the vertical field of the electric power industry, resulting in poor user interaction effect. In order to solve the problem that the large language model cannot understand the professional terminology in the vertical field of the electric power industry, resulting in poor user interaction effect, the embodiments of the present application provide an interaction method, device, computer-readable storage medium and electronic device based on a large language model based on electric power knowledge.

[0028] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0029] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 FIG. 1 is a hardware structure block diagram of a mobile terminal of an interactive method based on a large language model of electric power knowledge according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is for illustration only and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0030] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the interactive method of the large language model based on power knowledge in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, the above method is implemented. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. The transmission device 106 is used to receive or send data via a network. The specific example of the above-mentioned network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0031] In this embodiment, an interactive method based on a large language model of power knowledge running on a mobile terminal, a computer terminal or a similar computing device is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0032] Figure 2 FIG. 1 is a flow chart of an interactive method based on a large language model of electric power knowledge according to an embodiment of the present application. Figure 2As shown, the method comprises the following steps:

[0033] Step S201, obtaining electric power field keywords, searching according to the electric power field keywords, and obtaining explanation information of the electric power field keywords, wherein the electric power field keywords are professional terms in the electric power field, and the explanation information is information explaining the professional terms;

[0034] Specifically, to obtain keywords in the power field, the main words of power professional terms can be extracted through the power field knowledge graph, and then the most relevant web pages can be retrieved from the Internet through online retrieval. Finally, the retrieved text information is summarized and abstracted to generate explanatory information of the keywords in the power field.

[0035] Step S202, obtaining a large language model, and generating instruction fine-tuning data according to the interpretation information through the large language model, wherein the large language model is obtained by training multiple groups of language data in the power field, and the instruction fine-tuning data is data for adjusting parameters of the large language model;

[0036] Specifically, multiple pre-designed prompt words are used to guide a closed-source large language model (e.g., Chatgpt, etc.) to generate a variety of instruction fine-tuning data in different task forms based on the interpretation information of the above-mentioned power terms, and adjust the large language model.

[0037] Step S203, evaluate the instruction fine-tuning data to obtain an evaluation score, determine high-quality instruction fine-tuning data according to the evaluation score, and train the large language model through the high-quality instruction fine-tuning data to obtain a large language model in the power field, and interact through the large language model in the power field.

[0038] Specifically, the generated instruction fine-tuning data is evaluated and scored using a scoring model, and an appropriate threshold is set to filter out low-scoring data of poor quality, thereby obtaining the final high-quality instruction fine-tuning data; the generated power field instruction fine-tuning data is used to train a large language model to obtain a power field large language model.

[0039] Through this embodiment, the electric power field keywords are obtained, and the electric power field keywords are searched according to the electric power field keywords to obtain the explanation information of the electric power field keywords, the electric power field keywords are professional terms in the electric power field, and the instruction fine-tuning data is generated according to the explanation information by the large language model, and the parameters of the large language model are adjusted; the instruction fine-tuning data is evaluated to obtain the evaluation score, and the high-quality instruction fine-tuning data is determined according to the evaluation score, and the large language model is trained by the high-quality instruction fine-tuning data to obtain the large language model in the electric power field, and the large language model in the electric power field is interacted. Compared with the prior art, in which the large language model cannot understand the professional terms of the electric power industry, the present application generates instruction fine-tuning data through the electric power field keywords, supplements and adjusts the professional terms in the vertical field of the electric power industry in the large language model, so that the large language model interacts on the basis of fully understanding the professional terms in the electric power vertical industry. Therefore, it can solve the problem that the existing large language model cannot understand the professional terms in the vertical field of the electric power industry, resulting in poor user interaction effect, and achieve the technical effect of answering questions in the electric power field accurately.

[0040] In the specific implementation process, the above step S201 obtains the electric power field keywords, searches according to the electric power field keywords, and obtains the explanation information of the electric power field keywords, which can be achieved by the following steps: Step S2011: obtain the electric power knowledge graph, and obtain the electric power field keywords from the electric power knowledge graph; Step S2012: call the search engine to search according to the electric power field keywords to obtain the web page text, wherein the web page text contains the explanation text information of the electric power field keywords; Step S2013: summarize the explanation text information in the web page text to obtain the explanation information of the electric power field keywords. The method first retrieves the electric power field keywords and the corresponding explanation information required for training the large language model through the above steps, so that the data foundation can be established for training the large language model.

[0041] Specifically, the subject words of power professional terminology (i.e., keywords in the power field) are extracted from the power knowledge graph; for the subject words of power professional terminology, the search engine API is called to retrieve the web page text most relevant to the power professional terminology from the Internet; prompt words are designed to guide the large language model to use its powerful generation ability to summarize the retrieved text knowledge, and finally a detailed introduction of each keyword in the power field is obtained.

[0042] In the specific implementation process, the above step S202 generates instruction fine-tuning data according to the interpretation information through the large language model, which can be implemented by the following steps: Step S2021: Acquire a prompt word template, wherein the prompt word template represents a template for generating the instruction fine-tuning data and at least includes the output format of the instruction fine-tuning data; Step S2022: Generate the instruction fine-tuning data according to the prompt word template according to the interpretation information through the large language model. Through the above steps, the method generates instruction fine-tuning data according to the prompt word template, so that the predetermined output format and standardized instruction fine-tuning data can be obtained.

[0043] In the specific implementation process, the prompt word template is composed of the task description e task , Electricity term context information reference , Instruction fine-tuning data example demo and output format requirements restrict These four parts will be filled into the prompt word template E to form the prompt word p of the instruction fine-tuning data. The overall symbolic representation of the prompt word p generated by the instruction fine-tuning data is shown in the following formula: p(T,D)←E(e task ,e reference ,e demo ,e restrict), where T and D represent the task type of instruction fine-tuning data expected to be generated and the manually constructed high-quality instruction fine-tuning dataset, respectively, which will affect the design of the prompt word p for instruction fine-tuning data generation. The design of the prompt word template for instruction fine-tuning data generation adopts advanced structural design, with clear content distinction between different parts, easy to read and modify, and can better make the big model understand, thereby guiding it to use its powerful generation ability to generate high-quality instruction fine-tuning data; the task description part clearly points out the specific task description expected to be completed by the closed-source big language model, for example: in traditional crowdsourcing annotation scenarios, a task description is usually provided to the recruited employees, which specifies the necessary context, such as the purpose of the task, data interpretation, and other background knowledge, so that they can better understand their work. Similarly, such task specifications are crucial to setting the right context for data generation driven by the big language model, which can also include role-playing, format clarification, knowledge enhancement, etc. Studies have shown that a simple opening statement, such as "assuming you are a {xxx}", can significantly improve the output performance by setting an appropriate data generation scenario and allowing the big language model to better assume the role. The context information of electric power terminology is composed of two different sources: the electric power knowledge graph subgraph and the retrieved electric power professional terminology information. The reliability of reference knowledge is increased by mutual verification between electric power knowledge from different sources. If there is a knowledge conflict between knowledge from different sources, this part of the conflicting knowledge is called unreliable knowledge, so it is discarded and only the non-conflicting knowledge is used as a reference to generate instruction fine-tuning data. The instruction fine-tuning data examples are high-quality data carefully written by humans, with a total of 20 pieces. Each time the closed-source large language model is called to generate instruction fine-tuning data, three of them will be randomly extracted from the 20 manually written high-quality data and embedded in the instruction fine-tuning data example part of the prompt word template as an example reference for the closed-source large language model to generate instruction fine-tuning data. Providing output samples to the large language model in this way can not only deepen the large model's understanding of the task, but also increase the diversity of generated data and avoid the problem of generated data tending to be consistent. By filling in the contextual information of power terms and examples of instruction fine-tuning data in the prompt words, the content and form prompt words can be obtained, thereby guiding the large language model to generate more diverse instruction fine-tuning data. At the same time, by setting appropriate model generation parameters (such as temperature, etc.), the diversity of the generated instruction fine-tuning data can be further ensured; the output format requirement part refers to the requirement in the prompt words that the closed-source large language model will output in the specified format. By restricting the output format, the large model generates json format that is easy to parse. The generated instruction fine-tuning data can be post-processed very conveniently later, and the instruction and output parts can be split to obtain complete instruction fine-tuning data in the form of 'instruction-output' pairs.Finally, the designed instruction fine-tuning data prompt is used to guide the closed-source large language model to generate instruction fine-tuning data that meets the requirements.

[0044] In some optional implementations, the above step S203 evaluates the instruction fine-tuning data to obtain an evaluation score, which can be achieved by the following steps: step S2031: obtaining a pre-trained model, and evaluating the accuracy of the instruction fine-tuning data through the pre-trained model to obtain an accuracy score, wherein the pre-trained model is obtained by training the power field keywords and the corresponding interpretation information; step S2032: obtaining the input instructions and output data corresponding to the instruction fine-tuning data, and calculating the correlation between the input instructions and the output data to perform a correlation evaluation to obtain a correlation score; step S2033: determining the integrity of the instruction fine-tuning data to perform an integrity evaluation to obtain an integrity score; step S2034: determining the difficulty of the instruction fine-tuning data to perform a difficulty evaluation to obtain a difficulty score. The method screens high-quality data from the above four aspects through the above steps, so that the large language model after training can be more accurate.

[0045] Specifically, after the above instruction fine-tuning data generation step, overflow data D may be excessively generated. gen . However, these datasets usually contain a considerable number of noisy, worthless or even toxic samples, which is mainly due to two reasons. First, there is an upper limit to the instruction-following ability of closed-source large language models. Overly complex instruction fine-tuning data prompts will make it impossible for closed-source large language models to complete the desired tasks. Second, invalid prompts containing ambiguous descriptions may deceive the model into generating irrelevant or redundant samples. Therefore, directly utilizing these low-quality data without proper processing may have significant negative effects.

[0046] In order to solve this problem, a heuristic scoring standard is designed to score the samples of the generated instruction fine-tuning data, so as to distinguish data samples of different quality. Specifically, each generated data sample is scored from four different dimensions, and the score of each dimension is from 1 to 5, with 1 being the lowest and 5 being the highest.

[0047] The first quality rating dimension: accuracy S accuracy . Use a pre-trained model (such as ChatGPT, Wenxinyiyan, etc.) to directly evaluate the output of the sample to determine whether it accurately meets the instruction requirements or expected answers. This scoring dimension can directly reflect the accuracy of the sample and obtain the accuracy score.

[0048] The second quality scoring dimension: relevance S relevance. Evaluate the correlation between the sample's input instructions and output data. A high-quality sample should have a high degree of instruction relevance, that is, the output should be closely related to the input instructions and closely connected to the input information to obtain a relevance score.

[0049] The third quality rating dimension: completeness S completeness Check the completeness of the sample, including whether the instructions are clear, the input is sufficient, the output is complete, etc. A complete sample should contain all necessary elements to support effective training of the model and obtain a completeness score.

[0050] The fourth quality rating dimension: Difficulty S difficulty The difficulty of the task given by the instruction is scored. High-difficulty instructions can usually inspire the model to learn more complex skills and knowledge, and obtain a difficulty score.

[0051] In some optional implementations, the above step S203 determines high-quality instruction fine-tuning data according to the evaluation score, which can be achieved by the following steps: step S2035: obtaining the accuracy score threshold corresponding to the accuracy evaluation, the relevance score threshold corresponding to the relevance evaluation, the integrity score threshold corresponding to the integrity evaluation, and the difficulty threshold corresponding to the difficulty evaluation; step S2036: determining that the instruction fine-tuning data whose accuracy score is greater than the accuracy score threshold, the relevance score is greater than the relevance score threshold, the integrity score is greater than the integrity score threshold, and the difficulty score is greater than the difficulty threshold is the high-quality instruction fine-tuning data. The method determines high-quality instruction fine-tuning data through the above steps, so that the generated data samples can be screened, and high-quality data samples can be selected for model training to obtain a more accurate large language model.

[0052] In the specific implementation process, after scoring the generated instruction fine-tuning data, it is necessary to select an appropriate score threshold to filter the generated data samples. Finally, only the data with a score of more than three points in each scoring dimension will be retained, and the other data will be filtered out to ensure the quality of the final generated data. curated It can be expressed as the following formula D curated ←Simple(D gen ,S accuracy ,S relevance ,S completeness ,S difficulty ).

[0053] In some optional embodiments, the above-mentioned step S203 trains the large language model through the high-quality instruction fine-tuning data to obtain a large language model in the power field, which can be achieved through the following steps: calculate the output vector of the large language model through the low-rank adaptive method h=W0·x+ΔW·x, where h represents the output vector, x represents the input vector, W0 represents the weight value matrix, and ΔW represents the parameters to be adjusted in the large language model.

[0054] In the specific implementation process, low-rank adaptive (LoRA) is used in the fine-tuning stage of the model to improve training stability and reduce computational costs. d×k Decomposed into two matrices A∈R r ×k and B∈R d×r , which simplifies parameter insertion during training. For a given input x, LoRA calculates the output of each layer as h = W0·x + ΔW·x, where ΔW = AB T This approach assumes r <<min{d,k}, greatly reducing the number of parameters that must be fine-tuned.

[0055] In some optional implementations, the above step S203 trains the large language model through the high-quality instruction fine-tuning data to obtain the large language model in the power field, and can also be implemented by the following steps: Calculate the loss function of the large language model in the power field, where M represents the total number of words in the large language model in the power field, y o,c represents the binary function of the observation o and the cth word, p o,c represents the predicted probability that the observation o belongs to class c, L CE The method determines the loss function through the above steps, so that a more accurate large language model can be trained.

[0056] Specifically, in the supervised fine-tuning (SFT) stage, a dataset for the power field is carefully curated through the previous steps. Following the established supervised fine-tuning method, the model is trained to produce professional and accurate responses consistent with user expectations based on the comprehensive collection of command-output pairs. We combine LoRA with autoregressive cross entropy loss to fine-tune our model, and the specific loss function is shown in the above formula.

[0057] In order to enable those skilled in the art to more clearly understand the technical solution of the present application, the implementation process of the interactive method of the large language model based on power knowledge of the present application will be described in detail below in combination with specific embodiments.

[0058] This embodiment relates to a specific interactive method based on a large language model of power knowledge, such as Figure 3 As shown, the following steps are included:

[0059] Step S1: Retrieve the introduction of professional power terminology based on the power knowledge graph network;

[0060] Step S2: Customize prompt words to guide the large model to generate instruction fine-tuning data with diverse task forms;

[0061] Step S3: Score the generated instruction fine-tuning data using a quality scoring model;

[0062] Step S4: Design a scoring threshold to filter out low-quality instruction fine-tuning data;

[0063] Step S5: Train a professional large model in the power field.

[0064] The embodiment of the present application also provides an interactive device based on a large language model of electric power knowledge. It should be noted that the interactive device based on a large language model of electric power knowledge in the embodiment of the present application can be used to execute the interactive method for a large language model based on electric power knowledge provided in the embodiment of the present application. The device is used to implement the above-mentioned embodiments and preferred implementation modes, and those that have been described will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0065] The following introduces the interactive device based on the large language model of power knowledge provided in the embodiment of the present application.

[0066] Figure 4 Schematic diagram of an interactive device based on a large language model of power knowledge according to an embodiment of the present application. Figure 4 As shown, the device comprises:

[0067] The search unit 10 is used to obtain electric power field keywords, perform search based on the electric power field keywords, and obtain explanation information of the electric power field keywords, wherein the electric power field keywords are professional terms in the electric power field, and the explanation information is information explaining the professional terms;

[0068] Specifically, to obtain keywords in the power field, the main words of power professional terms can be extracted through the power field knowledge graph, and then the most relevant web pages can be retrieved from the Internet through online retrieval. Finally, the retrieved text information is summarized and abstracted to generate explanatory information of the keywords in the power field.

[0069] A generating unit 20, configured to obtain a large language model, and generate instruction fine-tuning data according to the interpretation information through the large language model, wherein the large language model is obtained by training multiple groups of language data in the electric power field, and the instruction fine-tuning data is data for adjusting parameters of the large language model;

[0070] Specifically, multiple pre-designed prompt words are used to guide a closed-source large language model (e.g., Chatgpt, etc.) to generate a variety of instruction fine-tuning data in different task forms based on the interpretation information of the above-mentioned power terms, and adjust the large language model.

[0071] The training unit 30 is used to evaluate the instruction fine-tuning data to obtain an evaluation score, determine high-quality instruction fine-tuning data according to the evaluation score, and train the large language model through the high-quality instruction fine-tuning data to obtain a large language model in the power field, and interact through the large language model in the power field.

[0072] Specifically, the generated instruction fine-tuning data is evaluated and scored using a scoring model, and an appropriate threshold is set to filter out low-scoring data of poor quality, thereby obtaining the final high-quality instruction fine-tuning data; the generated power field instruction fine-tuning data is used to train a large language model to obtain a power field large language model.

[0073] Through this embodiment, the electric power field keywords are obtained, and the electric power field keywords are searched according to the electric power field keywords to obtain the explanation information of the electric power field keywords, the electric power field keywords are professional terms in the electric power field, and the instruction fine-tuning data is generated according to the explanation information by the large language model, and the parameters of the large language model are adjusted; the instruction fine-tuning data is evaluated to obtain the evaluation score, and the high-quality instruction fine-tuning data is determined according to the evaluation score, and the large language model is trained by the high-quality instruction fine-tuning data to obtain the large language model in the electric power field, and the large language model in the electric power field is interacted. Compared with the prior art, in which the large language model cannot understand the professional terms of the electric power industry, the present application generates instruction fine-tuning data through the electric power field keywords, supplements and adjusts the professional terms in the vertical field of the electric power industry in the large language model, so that the large language model interacts on the basis of fully understanding the professional terms in the electric power vertical industry. Therefore, it can solve the problem that the existing large language model cannot understand the professional terms in the vertical field of the electric power industry, resulting in poor user interaction effect, and achieve the technical effect of answering questions in the electric power field accurately.

[0074] In the specific implementation process, the above-mentioned retrieval unit includes a first acquisition module, a retrieval module and a summary module. The first acquisition module is used to obtain the electric power knowledge graph and obtain the electric power field keywords from the electric power knowledge graph; the retrieval module is used to call the search engine to search according to the electric power field keywords to obtain the web page text, wherein the web page text contains the explanatory text information of the electric power field keywords; the summary module is used to summarize the explanatory text information in the web page text to obtain the explanatory information of the electric power field keywords. The device first retrieves the electric power field keywords and the corresponding explanatory information required for training the large language model through the above steps, so that the data foundation can be established for training the large language model.

[0075] Specifically, the subject words of power professional terminology (i.e., keywords in the power field) are extracted from the power knowledge graph; for the subject words of power professional terminology, the search engine API is called to retrieve the web page text most relevant to the power professional terminology from the Internet; prompt words are designed to guide the large language model to use its powerful generation ability to summarize the retrieved text knowledge, and finally a detailed introduction of each keyword in the power field is obtained.

[0076] In a specific implementation process, the generation unit includes a second acquisition module and a generation module, wherein the second acquisition module is used to acquire a prompt word template, wherein the prompt word template represents a template for generating the instruction fine-tuning data and at least includes the output format of the instruction fine-tuning data; the generation module is used to generate the instruction fine-tuning data according to the prompt word template based on the interpretation information through the large language model. The device generates instruction fine-tuning data according to the prompt word template through the above steps, so that the predetermined output format and standardized instruction fine-tuning data can be obtained.

[0077] In the specific implementation process, the prompt word template is composed of the task description e task , Electricity term context information reference , Instruction fine-tuning data example demo and output format requirements restrict These four parts will be filled into the prompt word template E to form the prompt word p of the instruction fine-tuning data. The overall symbolic representation of the prompt word p generated by the instruction fine-tuning data is shown in the following formula: p(T,D)←E(e task ,e reference ,e demo ,e restrict), where T and D represent the task type of instruction fine-tuning data expected to be generated and the manually constructed high-quality instruction fine-tuning dataset, respectively, which will affect the design of the prompt word p for instruction fine-tuning data generation. The design of the prompt word template for instruction fine-tuning data generation adopts advanced structural design, with clear content distinction between different parts, easy to read and modify, and can better make the big model understand, thereby guiding it to use its powerful generation ability to generate high-quality instruction fine-tuning data; the task description part clearly points out the specific task description expected to be completed by the closed-source big language model, for example: in traditional crowdsourcing annotation scenarios, a task description is usually provided to the recruited employees, which specifies the necessary context, such as the purpose of the task, data interpretation, and other background knowledge, so that they can better understand their work. Similarly, such task specifications are crucial to setting the right context for data generation driven by the big language model, which can also include role-playing, format clarification, knowledge enhancement, etc. Studies have shown that a simple opening statement, such as "assuming you are a {xxx}", can significantly improve the output performance by setting an appropriate data generation scenario and allowing the big language model to better assume the role. The context information of electric power terminology is composed of two different sources: the electric power knowledge graph subgraph and the retrieved electric power professional terminology information. The reliability of reference knowledge is increased by mutual verification between electric power knowledge from different sources. If there is a knowledge conflict between knowledge from different sources, this part of the conflicting knowledge is called unreliable knowledge, so it is discarded and only the non-conflicting knowledge is used as a reference to generate instruction fine-tuning data. The instruction fine-tuning data examples are high-quality data carefully written by humans, with a total of 20 pieces. Each time the closed-source large language model is called to generate instruction fine-tuning data, three of them will be randomly extracted from the 20 manually written high-quality data and embedded in the instruction fine-tuning data example part of the prompt word template as an example reference for the closed-source large language model to generate instruction fine-tuning data. Providing output samples to the large language model in this way can not only deepen the large model's understanding of the task, but also increase the diversity of generated data and avoid the problem of generated data tending to be consistent. By filling in the contextual information of power terms and examples of instruction fine-tuning data in the prompt words, the content and form prompt words can be obtained, thereby guiding the large language model to generate more diverse instruction fine-tuning data. At the same time, by setting appropriate model generation parameters (such as temperature, etc.), the diversity of the generated instruction fine-tuning data can be further ensured; the output format requirement part refers to the requirement in the prompt words that the closed-source large language model will output in the specified format. By restricting the output format, the large model generates json format that is easy to parse. The generated instruction fine-tuning data can be post-processed very conveniently later, and the instruction and output parts can be split to obtain complete instruction fine-tuning data in the form of 'instruction-output' pairs.Finally, the designed instruction fine-tuning data prompt is used to guide the closed-source large language model to generate instruction fine-tuning data that meets the requirements.

[0078] In some optional embodiments, the training unit includes a first evaluation module, a second evaluation module, a third evaluation module and a fourth evaluation module, wherein the first evaluation module is used to obtain a pre-trained model, and the accuracy of the instruction fine-tuning data is evaluated by the pre-trained model to obtain an accuracy score, wherein the pre-trained model is obtained by training the power field keywords and the corresponding interpretation information; the second evaluation module is used to obtain the input instructions and output data corresponding to the instruction fine-tuning data, and calculate the correlation between the input instructions and the output data to perform a correlation evaluation to obtain a correlation score; the third evaluation module is used to determine the integrity of the instruction fine-tuning data to perform an integrity evaluation to obtain an integrity score; the fourth evaluation module is used to determine the difficulty of the instruction fine-tuning data to perform a difficulty evaluation to obtain a difficulty score. The device screens high-quality data from the above four aspects through the above steps, so that the large language model after training can be more accurate.

[0079] Specifically, after the above instruction fine-tuning data generation step, overflow data D may be excessively generated. gen . However, these datasets usually contain a considerable number of noisy, worthless or even toxic samples, which is mainly due to two reasons. First, there is an upper limit to the instruction-following ability of closed-source large language models. Overly complex instruction fine-tuning data prompts will make it impossible for closed-source large language models to complete the desired tasks. Second, invalid prompts containing ambiguous descriptions may deceive the model into generating irrelevant or redundant samples. Therefore, directly utilizing these low-quality data without proper processing may have significant negative effects.

[0080] In order to solve this problem, a heuristic scoring standard is designed to score the samples of the generated instruction fine-tuning data, so as to distinguish data samples of different quality. Specifically, each generated data sample is scored from four different dimensions, and the score of each dimension is from 1 to 5, with 1 being the lowest and 5 being the highest.

[0081] The first quality rating dimension: accuracy S accuracy . Use a pre-trained model (such as ChatGPT, Wenxinyiyan, etc.) to directly evaluate the output of the sample to determine whether it accurately meets the instruction requirements or expected answers. This scoring dimension can directly reflect the accuracy of the sample and obtain the accuracy score.

[0082] The second quality scoring dimension: relevance S relevance. Evaluate the correlation between the sample's input instructions and output data. A high-quality sample should have a high degree of instruction relevance, that is, the output should be closely related to the input instructions and closely connected to the input information to obtain a relevance score.

[0083] The third quality rating dimension: completeness S completeness Check the completeness of the sample, including whether the instructions are clear, the input is sufficient, the output is complete, etc. A complete sample should contain all necessary elements to support effective training of the model and obtain a completeness score.

[0084] The fourth quality rating dimension: Difficulty S difficulty The difficulty of the task given by the instruction is scored. High-difficulty instructions can usually inspire the model to learn more complex skills and knowledge, and obtain a difficulty score.

[0085] In some optional embodiments, the training unit further includes a third acquisition module and a determination module, the third acquisition module is used to acquire the accuracy score threshold corresponding to the accuracy evaluation, the relevance score threshold corresponding to the relevance evaluation, the integrity score threshold corresponding to the integrity evaluation, and the difficulty threshold corresponding to the difficulty evaluation; the determination module is used to determine that the instruction fine-tuning data having the accuracy score greater than the accuracy score threshold, the relevance score greater than the relevance score threshold, the integrity score greater than the integrity score threshold, and the difficulty score greater than the difficulty threshold is the high-quality instruction fine-tuning data. The device determines the high-quality instruction fine-tuning data through the above steps, so that the generated data samples can be screened, and high-quality data samples can be selected for model training to obtain a more accurate large language model.

[0086] In the specific implementation process, after scoring the generated instruction fine-tuning data, it is necessary to select an appropriate score threshold to filter the generated data samples. Finally, only the data with a score of more than three points in each scoring dimension will be retained, and the other data will be filtered out to ensure the quality of the final generated data. curated It can be expressed as the following formula D curated ←Simple(D gen ,S accuracy ,S relevance ,S completeness ,S difficulty ).

[0087] In some optional embodiments, the training unit further includes a first computing module for computing the output vector of the large language model through a low-rank adaptive device h=W0·x+ΔW·x, wherein h represents the output vector, x represents the input vector, W0 represents the weight value matrix, and ΔW represents the parameters adjusted in the large language model.

[0088] In the specific implementation process, low-rank adaptive (LoRA) is used in the fine-tuning stage of the model to improve training stability and reduce computational costs. d×k Decomposed into two matrices A∈R r ×k and B∈R d×r , which simplifies parameter insertion during training. For a given input x, LoRA calculates the output of each layer as h = W0·x + ΔW·x, where ΔW = AB T This setup assumes r <<min{d,k}, greatly reducing the number of parameters that must be fine-tuned.

[0089] In some optional implementations, the training unit further includes a second calculation module for calculating Calculate the loss function of the large language model in the power field, where M represents the total number of words in the large language model in the power field, y o,c represents the binary function of the observation o and the cth word, p o,c represents the predicted probability that the observation o belongs to class c, L CE The loss value of the loss function is represented. The device determines the loss function through the above steps, so that a more accurate large language model can be trained.

[0090] Specifically, in the supervised fine-tuning (SFT) stage, a dataset for the power field is carefully curated through the previous steps. Following the established supervised fine-tuning device, the model is trained to produce professional and accurate responses consistent with user expectations based on the comprehensive collection of command-output pairs. We combine LoRA with autoregressive cross entropy loss to fine-tune our model, and the specific loss function is shown in the above formula.

[0091] The interactive device based on the large language model of power knowledge includes a processor and a memory, and the retrieval unit, the generation unit and the training unit are all stored in the memory as program units, and the processor executes the program units stored in the memory to realize the corresponding functions. The above modules are all located in the same processor; or, the above modules are located in different processors in any combination.

[0092] The processor contains a kernel, which calls the corresponding program unit from the memory. One or more kernels can be set, and the kernel parameters can be adjusted to solve the problem that the existing large language model cannot understand the professional terms in the vertical field of the power industry, resulting in poor user interaction.

[0093] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0094] An embodiment of the present invention provides a computer-readable storage medium, which includes a stored program, wherein when the program is run, the device where the computer-readable storage medium is located is controlled to execute the interactive method of the large language model based on power knowledge.

[0095] Specifically, the interactive method based on the large language model of power knowledge includes:

[0096] Step S201, obtaining electric power field keywords, searching according to the electric power field keywords, and obtaining explanation information of the electric power field keywords, wherein the electric power field keywords are professional terms in the electric power field, and the explanation information is information explaining the professional terms;

[0097] Specifically, to obtain keywords in the power field, the main words of power professional terms can be extracted through the power field knowledge graph, and then the most relevant web pages can be retrieved from the Internet through online retrieval. Finally, the retrieved text information is summarized and abstracted to generate explanatory information of the keywords in the power field.

[0098] Step S202, obtaining a large language model, and generating instruction fine-tuning data according to the interpretation information through the large language model, wherein the large language model is obtained by training multiple groups of language data in the power field, and the instruction fine-tuning data is data for adjusting parameters of the large language model;

[0099] Specifically, multiple pre-designed prompt words are used to guide a closed-source large language model (e.g., Chatgpt, etc.) to generate a variety of instruction fine-tuning data in different task forms based on the interpretation information of the above-mentioned power terms, and adjust the large language model.

[0100] Step S203, evaluate the instruction fine-tuning data to obtain an evaluation score, determine high-quality instruction fine-tuning data according to the evaluation score, and train the large language model through the high-quality instruction fine-tuning data to obtain a large language model in the power field, and interact through the large language model in the power field.

[0101] Specifically, the generated instruction fine-tuning data is evaluated and scored using a scoring model, and an appropriate threshold is set to filter out low-scoring data of poor quality, thereby obtaining the final high-quality instruction fine-tuning data; the generated power field instruction fine-tuning data is used to train a large language model to obtain a power field large language model.

[0102] An embodiment of the present invention provides an electronic device, including a processor, a memory, and a program stored in the memory and executable on the processor, wherein when the processor executes the program, at least the following steps are implemented:

[0103] Step S201, obtaining electric power field keywords, searching according to the electric power field keywords, and obtaining explanation information of the electric power field keywords, wherein the electric power field keywords are professional terms in the electric power field, and the explanation information is information explaining the professional terms;

[0104] Specifically, to obtain keywords in the power field, the main words of power professional terms can be extracted through the power field knowledge graph, and then the most relevant web pages can be retrieved from the Internet through online retrieval. Finally, the retrieved text information is summarized and abstracted to generate explanatory information of the keywords in the power field.

[0105] Step S202, obtaining a large language model, and generating instruction fine-tuning data according to the interpretation information through the large language model, wherein the large language model is obtained by training multiple groups of language data in the power field, and the instruction fine-tuning data is data for adjusting parameters of the large language model;

[0106] Specifically, multiple pre-designed prompt words are used to guide a closed-source large language model (e.g., Chatgpt, etc.) to generate a variety of instruction fine-tuning data in different task forms based on the interpretation information of the above-mentioned power terms, and adjust the large language model.

[0107] Step S203, evaluate the instruction fine-tuning data to obtain an evaluation score, determine high-quality instruction fine-tuning data according to the evaluation score, and train the large language model through the high-quality instruction fine-tuning data to obtain a large language model in the power field, and interact through the large language model in the power field.

[0108] Specifically, the generated instruction fine-tuning data is evaluated and scored using a scoring model, and an appropriate threshold is set to filter out low-scoring data of poor quality, thereby obtaining the final high-quality instruction fine-tuning data; the generated power field instruction fine-tuning data is used to train a large language model to obtain a power field large language model.

[0109] The devices in this article can be servers, PCs, PADs, mobile phones, etc.

[0110] The present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the method described in each embodiment of the present application:

[0111] Step S201, obtaining electric power field keywords, searching according to the electric power field keywords, and obtaining explanation information of the electric power field keywords, wherein the electric power field keywords are professional terms in the electric power field, and the explanation information is information explaining the professional terms;

[0112] Specifically, to obtain keywords in the power field, the main words of power professional terms can be extracted through the power field knowledge graph, and then the most relevant web pages can be retrieved from the Internet through online retrieval. Finally, the retrieved text information is summarized and abstracted to generate explanatory information of the keywords in the power field.

[0113] Step S202, obtaining a large language model, and generating instruction fine-tuning data according to the interpretation information through the large language model, wherein the large language model is obtained by training multiple groups of language data in the power field, and the instruction fine-tuning data is data for adjusting parameters of the large language model;

[0114] Specifically, multiple pre-designed prompt words are used to guide a closed-source large language model (e.g., Chatgpt, etc.) to generate a variety of instruction fine-tuning data in different task forms based on the interpretation information of the above-mentioned power terms, and adjust the large language model.

[0115] Step S203, evaluate the instruction fine-tuning data to obtain an evaluation score, determine high-quality instruction fine-tuning data according to the evaluation score, and train the large language model through the high-quality instruction fine-tuning data to obtain a large language model in the power field, and interact through the large language model in the power field.

[0116] Specifically, the generated instruction fine-tuning data is evaluated and scored using a scoring model, and an appropriate threshold is set to filter out low-scoring data of poor quality, thereby obtaining the final high-quality instruction fine-tuning data; the generated power field instruction fine-tuning data is used to train a large language model to obtain a power field large language model.

[0117] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0118] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0119] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0120] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0121] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0122] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0123] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0124] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0125] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0126] From the above description, it can be seen that the above embodiments of the present application achieve the following technical effects:

[0127] 1) In the interactive method of the large language model based on electric power knowledge of the present application, obtain electric power field keywords, search according to the electric power field keywords, obtain the explanation information of the electric power field keywords, the electric power field keywords are professional terms in the electric power field, generate instruction fine-tuning data according to the explanation information through the large language model, and adjust the parameters of the large language model; evaluate the instruction fine-tuning data to obtain the evaluation score, determine high-quality instruction fine-tuning data according to the evaluation score, and train the large language model through the high-quality instruction fine-tuning data to obtain the large language model in the electric power field, and interact through the large language model in the electric power field. Compared with the prior art, in which the large language model cannot understand the professional terms of the electric power industry, the present application generates instruction fine-tuning data through the electric power field keywords, supplements and adjusts the professional terms in the vertical field of the electric power industry in the large language model, so that the large language model interacts on the basis of fully understanding the professional terms in the electric power vertical industry. Therefore, it can solve the problem that the existing large language model cannot understand the professional terms in the vertical field of the electric power industry, resulting in poor user interaction effect, and achieve the technical effect of answering questions in the electric power field accurately.

[0128] 2) In the interactive device of the large language model based on electric power knowledge of the present application, obtain electric power field keywords, search according to the electric power field keywords, obtain the explanation information of the electric power field keywords, the electric power field keywords are professional terms in the electric power field, generate instruction fine-tuning data according to the explanation information through the large language model, and adjust the parameters of the large language model; evaluate the instruction fine-tuning data to obtain the evaluation score, determine high-quality instruction fine-tuning data according to the evaluation score, and train the large language model through the high-quality instruction fine-tuning data to obtain the large language model in the electric power field, and interact through the large language model in the electric power field. Compared with the prior art, in which the large language model cannot understand the professional terms of the electric power industry, the present application generates instruction fine-tuning data through the electric power field keywords, supplements and adjusts the professional terms in the vertical field of the electric power industry in the large language model, so that the large language model interacts on the basis of fully understanding the professional terms in the electric power vertical industry. Therefore, it can solve the problem that the existing large language model cannot understand the professional terms in the vertical field of the electric power industry, resulting in poor user interaction effect, and achieve the technical effect of answering questions in the electric power field accurately.

[0129] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. An interactive method based on a large language model of power knowledge, characterized in that: include: Acquire electric power field keywords, perform a search based on the electric power field keywords, and obtain explanation information of the electric power field keywords, wherein the electric power field keywords are professional terms in the electric power field, and the explanation information is information explaining the professional terms; Acquire a large language model, and generate instruction fine-tuning data according to the interpretation information through the large language model, wherein the large language model is obtained by training multiple groups of language data in the power field, and the instruction fine-tuning data is data for adjusting parameters of the large language model; The instruction fine-tuning data is evaluated to obtain an evaluation score, high-quality instruction fine-tuning data is determined according to the evaluation score, and the large language model is trained through the high-quality instruction fine-tuning data to obtain a large language model in the power field, and interaction is performed through the large language model in the power field.

2. The interactive method based on the large language model of power knowledge according to claim 1 is characterized in that: Obtaining electric power field keywords, searching according to the electric power field keywords, and obtaining explanation information of the electric power field keywords, including: Obtaining an electric power knowledge graph, and obtaining the electric power field keywords from the electric power knowledge graph; Invoke a search engine to search according to the power field keyword to obtain a web page text, wherein the web page text contains explanatory text information of the power field keyword; The explanatory text information in the webpage text is summarized to obtain the explanatory information of the electric power field keyword.

3. The interactive method based on the large language model of electric power knowledge according to claim 1 is characterized in that: Generating instruction fine-tuning data according to the interpretation information through the large language model includes: Acquire a prompt word template, wherein the prompt word template represents a template for generating the instruction fine-tuning data and at least includes an output format of the instruction fine-tuning data; The instruction fine-tuning data is generated by the large language model according to the explanation information and the prompt word template.

4. The interactive method based on a large language model of electric power knowledge according to claim 1 is characterized in that: The instruction fine-tuning data is evaluated to obtain an evaluation score, including: Acquire a pre-trained model, and evaluate the accuracy of the instruction fine-tuning data through the pre-trained model to obtain an accuracy score, wherein the pre-trained model is trained by the power field keywords and the corresponding explanation information; Obtaining input instructions and output data corresponding to the instruction fine-tuning data, and calculating the correlation between the input instructions and the output data to perform correlation evaluation, and obtaining a correlation score; Determining the integrity of the instruction fine-tuning data to perform integrity evaluation and obtain an integrity score; The difficulty of the instruction fine-tuning data is determined to perform a difficulty evaluation and obtain a difficulty score.

5. The interactive method based on the large language model of electric power knowledge according to claim 4 is characterized in that: Determining high-quality instruction fine-tuning data according to the evaluation score includes: Obtaining an accuracy score threshold corresponding to the accuracy evaluation, a relevance score threshold corresponding to the relevance evaluation, an integrity score threshold corresponding to the integrity evaluation, and a difficulty threshold corresponding to the difficulty evaluation; The instruction fine-tuning data whose accuracy score is greater than the accuracy score threshold, the relevance score is greater than the relevance score threshold, the completeness score is greater than the completeness score threshold, and the difficulty score is greater than the difficulty threshold is determined to be the high-quality instruction fine-tuning data.

6. The interactive method based on the large language model of electric power knowledge according to claim 1 is characterized in that: The large language model is trained by using the high-quality instruction fine-tuning data to obtain a large language model in the electric power field, including: The output vector of the large language model is calculated by a low-rank adaptive method h=W0·x+ΔW·x, wherein h represents the output vector, x represents the input vector, W0 represents the weight value matrix, and ΔW represents the parameters adjusted in the large language model.

7. The interactive method based on a large language model of electric power knowledge according to claim 1, characterized in that: The large language model is trained by fine-tuning the high-quality instruction data to obtain a large language model in the electric power field, further comprising: By formula Calculate the loss function of the large language model in the power field, where M represents the total number of words in the large language model in the power field, y o,c represents the binary function of the observation o and the cth word, p o,c represents the predicted probability that the observation o belongs to class c, L CE Represents the loss value of the loss function.

8. An interactive device based on a large language model of electric power knowledge, characterized in that: include: A search unit, used to obtain electric power field keywords, perform search based on the electric power field keywords, and obtain explanation information of the electric power field keywords, wherein the electric power field keywords are professional terms in the electric power field, and the explanation information is information explaining the professional terms; a generating unit, configured to obtain a large language model, and generate instruction fine-tuning data according to the interpretation information through the large language model, wherein the large language model is obtained by training multiple groups of language data in the electric power field, and the instruction fine-tuning data is data for adjusting parameters of the large language model; A training unit is used to evaluate the instruction fine-tuning data to obtain an evaluation score, determine high-quality instruction fine-tuning data according to the evaluation score, and train the large language model through the high-quality instruction fine-tuning data to obtain a large language model in the power field, and interact through the large language model in the power field.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the interactive method based on the large language model of power knowledge as described in any one of claims 1 to 7.

10. An electronic device, characterized in that: include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include an interactive method for executing the large language model based on power knowledge as described in any one of claims 1 to 7.

Images