Power industry knowledge question-answering method and system based on large language model

Abstract

The invention provides a power industry knowledge question-answering method and system based on a large language model, and relates to the technical field of artificial intelligence, and the method comprises the steps: obtaining power industry data; constructing a knowledge graph with node weights; acquiring a training set taking the power industry question as a feature and taking the power industry answer as a label; constructing a generative question and answer model with knowledge graph embedded information; establishing a generation loss function; taking the generation loss function value smaller than a preset generation loss function value as a target, training the generative question-answer model to determine an optimal hyper-parameter of the generative question-answer model; obtaining a to-be-processed power industry problem; and inputting a to-be-processed power industry question into the trained generative question and answer model, and outputting an expected power industry answer corresponding to the to-be-processed power industry question. And in combination with the knowledge graph with the node weight and the generative question and answer model, the interference of training set noise on model output answers is reduced, and the consistency and accuracy of the generated answers are improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a knowledge question-answering method and system for the electric power industry based on a large language model. Background Art

[0002] Power industry knowledge refers to professional knowledge related to the operation, maintenance, management and optimization of power systems, covering areas such as power equipment (such as transformers, generators), power dispatching, distribution networks, transmission lines, fault diagnosis, maintenance procedures, safety regulations and energy management. This knowledge is usually highly professional and complex.

[0003] The power industry involves a large number of complex equipment, operating procedures and fault diagnosis. Traditional information retrieval methods are difficult to provide professional answers quickly and accurately. The large language model can understand natural language questions and provide accurate and context-related answers based on the expertise of the power industry, improving the efficiency of problem solving and reducing human errors. It also provides intelligent support for power engineers and operators, helping to improve overall operational efficiency and safety.

[0004] However, the existing knowledge question-and-answer system in the power industry only uses the semantic analysis capability of the large language model to infer answers when introducing the large language model. This is random and the generated answers are easily affected by the noise of the training data, resulting in deviations in the accuracy of the question-and-answer system. In addition, due to the limited context window of the large model, when the conversation length exceeds this window, the model cannot remember the previous conversation content. As a result, in long conversations or multi-round question-and-answer sessions, the model may ignore previous discussions, resulting in a broken context for the generated answers and an inability to maintain long-term memory and consistency. Summary of the invention

[0005] In order to solve the technical problem that the prior art of electric power industry knowledge question and answer only uses the semantic analysis ability of the large language model to infer the answer during the process of introducing the large language model, which is random, and the generated answer is easily disturbed by the noise of the training data, resulting in deviation in the accuracy of the question and answer, and because the context window of the large model is limited, when the conversation length exceeds this window, the model cannot remember the previous conversation content, which makes the model ignore the previous discussion in long conversations or multiple rounds of question and answer, resulting in a broken context of the generated answer and an inability to maintain long-term memory and consistency, the present invention provides a method and system for electric power industry knowledge question and answer based on a large language model.

[0006] The technical solution provided by the embodiment of the present invention is as follows:

[0007] First aspect

[0008] An embodiment of the present invention provides a method for answering questions about knowledge in the electric power industry based on a large language model, comprising:

[0009] S1: Obtain power industry data;

[0010] S2: Based on the power industry data, a knowledge graph with node weights is constructed in combination with the PageRank algorithm with an adaptive damping coefficient, where the knowledge graph includes nodes representing various power entities and edges representing the relationships between various power entities;

[0011] S3: Obtain a training set with power industry questions as features and power industry answers as labels;

[0012] S4: A generative question answering model with knowledge graph embedding information is constructed in combination with the knowledge graph, wherein the generative question answering model includes an encoder, a scaled dot product attention layer, and an autoregressive decoder connected in sequence, wherein the knowledge graph is connected to the scaled dot product attention layer through a graph embedding layer;

[0013] S5: Combined with soft cosine similarity, a generation loss function is established to evaluate the generation loss of the answers generated by the generative question-answering model in the power industry;

[0014] S6: Combined with the particle swarm optimization algorithm with multi-state dynamic inertia weights, with the goal of generating a loss function value less than a preset generation loss function value, the generative question answering model is trained using the training set to determine the optimal hyperparameters of the generative question answering model;

[0015] S7: Obtain pending power industry issues;

[0016] S8: Input the power industry question to be processed into the trained generative question-answering model, and output the expected power industry answer corresponding to the power industry question to be processed.

[0017] Second aspect

[0018] An embodiment of the present invention provides a power industry knowledge question-answering system based on a large language model, comprising:

[0019] processor;

[0020] A memory having computer-readable instructions stored thereon, which, when executed by a processor, implements the electric power industry knowledge question-answering method based on a large language model as in the first aspect.

[0021] The third aspect

[0022] An embodiment of the present invention provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method for answering questions about knowledge in the electric power industry based on a large language model as described in the first aspect is implemented.

[0023] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0024] In the present invention, by connecting the knowledge graph with node weights to the scaled dot product attention layer of the generative question-answering model, and then using the autoregressive decoder to output the answer to the question, the interference caused by the training process is compensated, the weight relationship between different knowledge nodes is captured more accurately, and the focus on key knowledge points is improved. On the basis of the autoregressive decoder, the model refers to the weight information of the knowledge graph in each step of the generation process to ensure that the generated answers are more logical, accurate and contextually consistent, and the consistency, logical rationality and accuracy of the answers can still be guaranteed based on the knowledge graph even in limited context windows and multi-round conversations. Combined with the node weights determined by the PageRank algorithm with an adaptive damping coefficient, the importance of the nodes in the graph can be dynamically adjusted to ensure that the correlation and influence between nodes are more accurately reflected in different situations, thereby providing more accurate weight distribution in complex networks, and further improving the reasoning ability of the model and the reliability of the answers. In the training process of the generative question-answering model, a generative loss function based on soft cosine similarity is used, which can capture the semantic similarity between texts more finely, especially for long texts and inconsistent word orders. The particle swarm optimization algorithm based on the multi-state dynamic inertia weight calculated based on particle swarm diversity and particle swarm fitness is combined for hyperparameter optimization. It can adaptively adjust according to the search state at different optimization stages, balance global search and local development, and improve the efficiency of hyperparameter tuning and the semantic consistency and accuracy of model generation results. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 A schematic diagram of a flow chart of a method for answering questions about knowledge in the electric power industry based on a large language model provided by an embodiment of the present invention;

[0027] Figure 2 A schematic diagram of the structure of a generative question-answering model provided by an embodiment of the present invention;

[0028] Figure 3 A schematic diagram of the structure of a power industry knowledge question and answer system based on a large language model provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0029] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0030] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0031] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0032] Reference Manual Attached Figure 1 , showing a flow chart of a method for answering questions in the electric power industry knowledge based on a large language model provided by an embodiment of the present invention.

[0033] The embodiment of the present invention provides a method for answering questions about the knowledge of the electric power industry based on a large language model. The method can be implemented by a device for answering questions about the knowledge of the electric power industry based on a large language model. The device for answering questions about the knowledge of the electric power industry based on a large language model can be a terminal or a server. The processing flow of the method for answering questions about the knowledge of the electric power industry based on a large language model can include the following steps:

[0034] S1: Obtain power industry data.

[0035] Among them, power industry data includes equipment information, operation data, maintenance records, operation procedures, fault diagnosis and repair steps of the power system, etc. These data cover the entire process from power generation to distribution, involving the working status of power equipment, historical operation records, real-time monitoring data and related technical standards and specifications. They are the basis for building knowledge graphs and training generative question-answering models, and can help the models accurately understand and answer professional questions in the power industry.

[0036] S2: Based on the power industry data, a knowledge graph with node weights is constructed in combination with the PageRank algorithm with an adaptive damping coefficient.

[0037] Among them, the knowledge graph includes nodes representing various power entities and edges representing the relationships between various power entities.

[0038] Among them, the adaptive damping coefficient is a parameter in the PageRank algorithm, which is used to control the probability of random jumps. PageRank is a graph-based sorting algorithm that was first used for web page sorting. It iteratively calculates the influence of each node in the graph, taking into account the node's out-degree and connection relationship with other nodes. The higher the PageRank value, the greater the importance of the node in the network. The knowledge graph is a structured data representation that uses nodes and edges to represent entities (such as devices) and their relationships. It can help the model understand the knowledge hierarchy and association relationships in complex fields and provide clear structural information. The node weight indicates the importance of each node in the graph. The larger the weight, the higher the influence or more important role of the node in the knowledge graph.

[0039] By constructing a knowledge graph with node weights through the PageRank algorithm with adaptive damping coefficient adjustment, it can more accurately reflect the connections and influences between entities in the power industry, enable the generative question-answering model to effectively focus on important knowledge nodes, and improve the accuracy of model reasoning and the logic of answer generation.

[0040] In a possible implementation, S2 specifically includes:

[0041] S201: Extract basic data from power industry data, wherein the basic data includes power entities, power entity attributes, power entity relationships, and power entity relationship attributes.

[0042] Specifically, entities and entity attributes are extracted through the entity extraction model, and relations and relationship attributes are extracted through the relationship extraction model.

[0043] S202: Connect each node according to the basic data to build a primary knowledge graph:

[0044] G=(V,E)

[0045] v i =(e i ,A i ),v i ∈V

[0046]

[0047] Among them, G represents the primary knowledge graph composed of the node set V and the edge set E, v i The power entity is e i The power entity attribute is A i The i-th node, e ij Indicates that the power entity relationship between the i-th node and the j-th node is r ij , the power entity relationship attribute between the i-th node and the j-th node is edge.

[0048] S203: Initialize the PageRank value of each node, where the PageRank value is the reciprocal value of the total number of nodes:

[0049]

[0050] Among them, PR (v i ) represents the i-th node v i PageRank value, N represents the total number of nodes.

[0051] S204: Update the PageRank value of each node according to the number of adjacent nodes and out-degree of each node:

[0052]

[0053] Among them, PR'(v i ) represents node v i The updated PageRank value of , d represents the adaptive damping coefficient representing the probability of randomly jumping to other nodes in each calculation, O(v i ) represents node v i The incoming neighbor set of node v i The adjacent node set of PR(v j ) represents node v j The PageRank value of N represents the total number of nodes. and They represent the average out-degree and average in-degree of nodes in the primary knowledge graph, H and H max They represent the actual information entropy of the primary knowledge graph and the maximum information entropy under the condition of completely uniform node access distribution, log represents the logarithmic function, P(v i ) represent the nodes v i The probability of being visited before updating is the PageRank value.

[0054] Among them, the average out-degree of a node indicates how many edges each node has pointing to other nodes on average in the entire primary knowledge graph, reflecting the overall connectivity between nodes. The average in-degree of a node indicates the average in-degree of all nodes in the primary knowledge graph, indicating how many edges each node has pointing to it on average. Specifically, It represents the normalized value of information entropy, ranging from 0 to 1. When H = H maxWhen H is small, it means that the access probability of the node is completely uniformly distributed. At this time, the randomness is maximum, and the damping coefficient should be closer to 1. When H is small, it means that the randomness of the system is small and the access between nodes is uneven. At this time, the damping coefficient should be small to make better use of random jumps. By combining information entropy, the overall randomness and structural tightness of the system are balanced. Information entropy reflects the randomness and uncertainty of the system. Combining the connectivity of the nodes (out-degree and in-degree) and the access probability distribution, the damping coefficient can be selected more accurately.

[0055] It should be noted that in the calculation process of the damping coefficient, when the out-degree and in-degree are larger, the damping coefficient should tend to 1, indicating that the relationship between the nodes in the graph is stronger. When the out-degree and in-degree are smaller, the damping coefficient should be reduced to better utilize random jumps to avoid falling into an infinite loop. Among them, O(v i ) is the node v i The set of all incoming neighbors points to node v i The setting of the damping coefficient avoids the problem of the calculation falling into an infinite loop. Optionally, the specific value of d can be 0.85. By combining the average out-degree, average in-degree and information entropy of the nodes to calculate the PageRank value, it is possible to more accurately balance the structural association and random jumps between nodes, taking into account the density of node connections in the graph and reflecting the randomness and uncertainty of the system through information entropy. This method effectively avoids local structural traps, ensures the stability and wide applicability of the algorithm, and at the same time improves the adaptability to complex network structures, making the generated PageRank value more accurate and reasonable.

[0056] S205: Add the updated PageRank value as the weight of the corresponding node to the primary knowledge graph to obtain a knowledge graph with node weights.

[0057] It should be noted that the knowledge graph constructed by combining the PageRank algorithm with an adaptive damping coefficient can more accurately reflect the connection relationship and importance between power entities. This method comprehensively considers the out-degree, in-degree of the node and the randomness of the system, so that the knowledge graph can better capture the structural features in the graph and effectively avoid local structural traps, improve the stability and adaptability of the algorithm, thereby generating a more accurate and reliable knowledge graph and improving the reliability of power Q&A.

[0058] S3: Obtain a training set with power industry questions as features and power industry answers as labels.

[0059] Among them, features are the input data of the model, representing the task context that we want the model to learn. In the power industry knowledge question-answering model, features refer to questions related to the power industry raised by users. Labels are the output data of the model, representing the target that the model should learn to generate. Labels are the standard answers or solutions corresponding to the features, and are the correct outputs that the model should generate through learning. The training set is used to supervise the learning of the model. By learning the relationship between questions and corresponding answers, the model gradually learns how to generate accurate answers based on questions.

[0060] Reference Manual Attached Figure 2 , showing a structural diagram of a generative question-answering model provided by an embodiment of the present invention.

[0061] S4: Combine knowledge graphs to build a generative question-answering model with knowledge graph embedding information.

[0062] Among them, the generative question-answering model includes an encoder, a scaled dot product attention layer and an autoregressive decoder connected in sequence, wherein the knowledge graph is connected to the scaled dot product attention layer through a graph embedding layer.

[0063] Among them, the encoder is responsible for processing the input power industry questions and converting natural language into numerical representation (vector). The scaled dot product attention layer is used to capture the association between the various parts of the input features, focusing on important keywords or information in the question. The autoregressive decoder is responsible for gradually generating answers based on the vector representation generated by the encoder. During the autoregressive process, each time the model generates a word, it will take the previously generated word as input and continue to generate the next word until the complete answer is output. The graph embedding layer embeds the nodes and relationships in the knowledge graph into the vector space. By connecting with the attention layer, the information in the knowledge graph is injected into the model, helping the model to better understand the knowledge structure and entity relationships of the power industry.

[0064] It should be noted that by embedding the knowledge graph into the encoder and scaled dot product attention layer of the generative question-answering model, the model can use the structured knowledge of the power industry to better understand the semantics and background information of the question during the generation process, and improve the ability to accurately capture and reason about key concepts. Combined with the weight information of the knowledge graph, the autoregressive decoder can generate more professional, logically clear and contextually consistent answers, especially in complex power industry question-answering scenarios, greatly improving the accuracy and relevance of the answers.

[0065] Specifically, when processing dialogues, traditional language models cannot utilize the entire long document due to the limitation of input length, which may lead to the loss of important knowledge and affect the quality of the generated dialogue. In the power industry, since power analysis and reasoning is complex and requires cascade reasoning, if the knowledge of the previous stage cannot be accurately recorded and learned, the generated power industry answers will lack depth and cannot be effectively analyzed and given accurate answers. The introduction of knowledge graphs converts the key information in the document into triples and constructs a graph structure. These knowledge graphs can effectively summarize long documents, ensure that important information is included in the input, and enable the model to reduce the learning length limit of the model itself in the answer, and improve the accuracy and coherence of answer generation. By converting documents into knowledge graphs, the model overcomes the capacity limitation problem of pre-trained language models when processing long documents, that is, the key information of long documents is condensed through graph structures to avoid the loss of important knowledge caused by truncating documents.

[0066] In actual application, the generation process of the generative question-answering model is introduced by combining the search-based knowledge graph to make the output answers more logical, and the accuracy of the output answers in the multi-round dialogue process can be increased based on the connection relationship of the indicator graph. It can also refer to the device attributes, processes and standards in the knowledge graph to generate more professional answers. Generative models may sometimes generate vague and imprecise answers when dealing with complex and technical questions. The introduction of knowledge graphs can constrain the generation process, making the answers generated by the model more accurate and consistent, and avoiding the generation of wrong or irrelevant answers. When answering questions involving multiple devices or multiple steps, knowledge graphs can help the model eliminate ambiguity and generate clear answers through the relationships and processes between devices. In a multi-round dialogue, users may ask multiple consecutive questions with complex contextual relationships between these questions. Knowledge graphs can maintain the background information in the dialogue and the logical relationship between devices, helping the model generate coherent answers, especially for questions across devices and operation processes.

[0067] In a possible implementation, the generative question answering model further includes an input layer and an output layer, wherein the input layer is connected to the encoder, and the output layer is connected to the autoregressive decoder.

[0068] S5: Combined with soft cosine similarity, a generation loss function is established to evaluate the generation loss of the generative question-answering model for generating answers in the power industry.

[0069] Among them, soft cosine similarity is an extension of cosine similarity. It not only considers the angle between vectors, but also the semantic similarity between the elements of the vector (such as words). Traditional cosine similarity only compares the values ​​of two vectors and does not care about the relationship between the elements. Soft cosine similarity introduces a similarity matrix between elements, so that even if two words are not exactly the same, as long as they are semantically related, the model can detect similarities, thereby ensuring that the generated answers meet expectations while reducing the difficulty of training.

[0070] In a possible implementation, the loss function is generated as follows:

[0071]

[0072] Among them, L oss represents the generated loss function, represents the total probability of obtaining the answer generated by the power industry, log represents the logarithmic function, and Y represents the label, i.e., the true answer of the power industry. Indicates the power industry corresponding to the label generates the answer, Indicates that Y and The soft cosine similarity between p represents the pth word in Y, express The qth word in Y, S(p,q) represents p and The cosine similarity between them, S(p,p) and S(q,q) both represent the self-cosine similarity with a value of 1.

[0073] It should be noted that by taking the logarithm of the generation probability, the low-probability generation results can be amplified and penalized, encouraging the model to generate more expected high-quality answers. This item allows the loss function to consider both the accuracy of generation and the flexibility provided by soft cosine similarity. The addition of soft cosine similarity can avoid excessive losses caused by single vocabulary errors. Even if there are minor errors in the generation process, as long as the overall semantics remain close, the penalty of the loss function will not be too high. This makes the optimization process of the model more stable and reduces possible volatility in the generation process. In addition, soft cosine similarity can guide the model to pay more attention to the overall semantic matching rather than just sticking to the form of vocabulary, thereby improving the quality of generated content. It not only considers the accuracy of the generated answer and the real answer (through logarithmic probability), but also takes into account the semantic similarity. Through soft cosine similarity, even if the vocabulary of the generated result is different from that of the real answer, but the semantics are close, the loss will not be too high, thereby improving the flexibility and robustness of the generation model, and then conducting reasonable model training and improving the model training effect.

[0074] S6: Combined with the particle swarm optimization algorithm with multi-state dynamic inertia weights, the generative question answering model is trained using the training set with the goal of generating a loss function value less than the preset generation loss function value to determine the optimal hyperparameters of the generative question answering model.

[0075] Among them, the multi-state dynamic inertia weight is the inertia weight calculated based on the particle swarm diversity and particle swarm fitness.

[0076] Among them, the multi-state dynamic inertia weight is a dynamic parameter in particle swarm optimization, which is used to control the search behavior of the particle swarm. At different stages, the inertia weight is adjusted according to the diversity and fitness of the particle swarm. A larger inertia weight encourages global exploration, while a smaller weight promotes local search, so that the algorithm dynamically balances between global search and local optimization. The particle swarm optimization algorithm is an optimization algorithm based on swarm intelligence, which simulates the collaborative behavior of a particle swarm. Each particle represents a solution. The particle continuously adjusts its position by tracking its own optimal solution and the optimal solution in the swarm, and finally converges to the optimal solution. The generation loss function value represents the error of the model when generating answers, and the loss function is used to measure the gap between the generated answer and the correct answer. The smaller the generation loss value, the better the generation effect of the model. The optimal hyperparameter refers to a set of hyperparameter combinations found by methods such as particle swarm optimization, which makes the generative question answering model perform best on a specific task. Particle diversity measures the degree of difference between particles in a particle swarm and reflects the distribution of the swarm in the search space. The fitness of the particle swarm is to evaluate the quality of the solution corresponding to each particle. The fitness is usually calculated based on the value of the generated loss function. The higher the fitness, the closer the particle's solution is to the global optimal solution.

[0077] By combining the particle swarm optimization algorithm with multi-state dynamic inertia weights, the model can adaptively adjust the search range, accelerate local optimization while ensuring global search, improve the optimization efficiency of the hyperparameters of the generative question-answering model, and ensure that the generation loss converges to the optimal solution faster, making model training more efficient and accurate.

[0078] It should be noted that those skilled in the art can set the size of the generated loss function value according to actual needs, and the present invention is not limited here.

[0079] In one possible implementation, the optimal hyperparameters include an optimal learning rate, an optimal batch size, an optimal number of encoder layers, an optimal number of decoder layers, and an optimal Dropout rate of the generative question answering model.

[0080] In a possible implementation, S6 specifically includes:

[0081] S601: Input the features of the training set, i.e., the power industry problems, into the encoder to obtain the feature hidden state:

[0082] h i =Encoder(x i ,θ enc )

[0083] Among them, h i Represents the i-th word x in the input power industry problem i The hidden state of Encoder represents the Transformer encoder, θ enc Represents a Transformer encoder parameter set.

[0084] S602: Convert the knowledge graph into an embedded representation of a numerical vector through the graph embedding layer:

[0085] z v =GAT(h v ,h N(v) ,A,θ gat )

[0086] Among them, z v represents the embedding representation of the v node in the knowledge graph, GAT represents the graph embedding layer representing the graph attention network, and h N(v) represents the neighbor node feature representation of node v in the graph embedding layer, h v represents the feature representation of v nodes in the graph embedding layer, A represents the adjacency matrix of the knowledge graph, and θ gat Represents the graph embedding layer parameters representing the weight matrices of each layer of the graph embedding layer.

[0087] S603: Combined with the node PageRank value, the attention scores of the feature hidden state and the embedded representation are calculated respectively through the scaled dot product attention layer:

[0088]

[0089] Among them, α i,j represents the attention score between the i-th hidden state and the j-th knowledge graph node, q i represents the corresponding query vector obtained by transforming the hidden state of the i-th word, exp represents the natural exponential function, k j and k k They represent the corresponding key vectors obtained by transforming the embedding representation of the jth node and the embedding representation of the kth node, respectively. k Indicates k k The key vector dimension of , PageRank(j) represents the PageRank value of node j, and T represents transposition.

[0090] S604: Combined with the attention score, the autoregressive decoder is used to sequentially generate answer words based on the feature hidden state and the embedded representation in a cascade manner, wherein each answer word constitutes a generated label, i.e., the power industry generates an answer:

[0091] y t =Decoder(y t-1 ,h,z,α i,j ,θ dec )

[0092]

[0093] Among them, y t and t-1 Denote the answer words generated by the autoregressive decoder at the t-th time step and the t-1-th time step, respectively. Decoder denotes the autoregressive decoder. h denotes the hidden state of the input power industry question. z denotes the embedded representation of the input power industry question. represents the answer generated by the power industry, y i Represents the answer word generated at the i-th time step, i = 1, 2, …, T', T' represents the total number of answer words generated.

[0094] Specifically, the cascade method means that after each answer word, all the answer words are converted into hidden representations. When the next answer word is generated, the converted hidden representation is directly re-input into the decoder to intervene in the generation of the next answer word. The model generates only one word at a time. The first word generated will be used as input to generate the second word. After the second word is generated, it will be used as input to generate the third word. This process continues until a complete sentence is generated or a predetermined end mark is reached (such as a special terminator). Each generated word depends not only on the input context (such as the conversation history or the knowledge graph), but also on the previously generated words. In other words, each generated word affects the generation of subsequent words.

[0095] Specifically, the model first receives an input question or dialogue context and encodes it. The output of this part is the understanding of the question or the encoding of the background context. In the decoding stage, the model generates the first word. This word is generated based on the input context and the knowledge graph. After the first word is generated, this word will be used as the input of the next decoder, and will be passed into the model together with the previous context to generate the next word. Each time a new word is generated, it will be input into the decoder together with the previously generated word, and the next word will continue to be generated until the model generates a complete answer sentence. The advantage of this autoregressive generation is not only that it ensures the coherence and semantic consistency of the context, but also effectively reduces the risk of error propagation through gradual generation and dynamic adjustment. For tasks that require complex reasoning and multi-round dialogues, autoregressive generation is particularly suitable because it can process information step by step and flexibly respond to changes in the task, thereby improving the accuracy and naturalness of the generated results.

[0096] S605: Taking the generation loss function value less than the preset generation loss function value as the optimization goal, determining the multi-state dynamic inertia weight based on the particle swarm diversity and the particle swarm fitness change rate, adjusting the hyperparameters of the generative question-answering model through the particle swarm optimization algorithm of the multi-state dynamic inertia weight, and outputting the optimal hyperparameters.

[0097] In a possible implementation manner, S605 specifically includes:

[0098] S6051: Determine the generated loss function value as the objective function.

[0099] S6052: Initialize hyperparameters of different values ​​as particles to determine particle positions and particle speeds, wherein the particle position is a parameter vector composed of various hyperparameters.

[0100] S6053: Use the multi-state dynamic inertia weights determined based on the Fermat principle to determine the particle position update formula and particle velocity update formula:

[0101] x i (t+1)=x i (t)+v i (t+1)

[0102]

[0103] Among them, x i (t) and x i (t+1) represents the particle position at time t and the particle position at time t+1, respectively, v i (t) and v i(t+1) represents the particle velocity at time t and the particle velocity at time t+1 respectively, w(t) represents the multi-state dynamic inertia weight at time t that controls the inertia of particle movement, c1 and c2 both represent learning factors with a value of 2, l1 and l2 represent the first random number and the second random number with a value range of 0 to 1 respectively, represents the best historical position of particle i, g best represents the global best position, i.e., the best historical position found by all particles. d(t) represents the average distance between particles at time t, which represents the diversity of the particle swarm. min(d) = 10 -6 represents the minimum diversity of the particle group, U represents the total number of particles, represents the center position of the particle swarm at time t, ΔFitness(t) represents the fitness difference between the particle swarm at time t and time t-1 in different iterations, and ΔFitness max Indicates the maximum value of the fitness change of the particle swarm during the historical iteration process.

[0104] Among them, the maximum value of the particle swarm fitness change will continue to decrease with the increase of the number of iterations. Therefore, selecting the maximum value of the particle swarm fitness change in the historical iteration process can ensure the normalization of the particle swarm fitness difference in the subsequent iteration process. The fitness here is the size of the loss function value of the model under the training set, and the particle swarm fitness difference is the difference between the loss function under the current iteration number and the previous iteration number.

[0105] Specifically, the role of Fermat's principle in the calculation process of multi-state dynamic inertia weight can be regarded as the optimized "shortest path" principle. The system adjusts the inertia weight to promote particles to find the optimal solution and avoid searching too many unnecessary paths, which is similar to the process of selecting the shortest path when light propagates. By adjusting the inertia weight, the particle swarm can dynamically switch the search strategy according to its own fitness and diversity at different stages, similar to the process of light finding the shortest path in Fermat's principle. The particle swarm will quickly reach the global optimal solution through the "optimal inertia path".

[0106] S6054: Update the particles using the particle position update formula and the particle velocity update formula respectively, and calculate the update objective function value of the updated particles under the training set.

[0107] S6055: Determine whether the difference between the updated objective function value and the objective function value before updating is less than the preset difference. If so, proceed to step S6056; otherwise, return to step S6054.

[0108] It should be noted that those skilled in the art can set the size of the preset difference according to actual needs, and the present invention is not limited thereto.

[0109] S6056: Output the updated particle position as the optimal particle position, and output the hyperparameter corresponding to the optimal particle position as the optimal hyperparameter, and end the hyperparameter adjustment process.

[0110] In step S6051 to step S6056, the model first generates the loss function value as the objective function, initializes different hyperparameters as particles, and calculates their positions and velocities. Then, through the dynamic inertia weight of Fermat's principle, the position and velocity of the particles are updated, and the search direction is gradually adjusted so that the particle swarm can quickly find the global optimal solution. Next, the model repeatedly calculates the updated objective function value and determines whether it is close to the preset threshold. Finally, the hyperparameter corresponding to the optimal particle position is output as the optimal hyperparameter of the model. By adjusting the dynamic inertia weight, the global search and local optimization are balanced, the efficiency and accuracy of the hyperparameter search are improved, and the model can quickly converge to the optimal state.

[0111] It should be noted that by combining the particle swarm optimization algorithm with multi-state dynamic inertia weights for hyperparameter adjustment, the model can adaptively balance global search and local optimization according to the diversity and fitness changes of the particle swarm. This method can not only speed up the optimization of hyperparameters, but also avoid falling into local optimality, so that the generative question-answering model can obtain a better hyperparameter combination in complex tasks, thereby improving the generation quality, accuracy and training efficiency of the model, while ensuring that the generation loss quickly converges to the optimal solution.

[0112] S7: Obtain pending power industry issues.

[0113] S8: Input the power industry question to be processed into the trained generative question-answering model, and output the expected power industry answer corresponding to the power industry question to be processed.

[0114] In a possible implementation manner, after S8, the method further includes:

[0115] The power industry data is updated at preset intervals, and the knowledge graph and generative question-answering model are updated based on the updated power industry data.

[0116] By regularly updating power industry data and adjusting the knowledge graph and generative question-answering model based on this updated data, we ensure that the model always maintains an accurate grasp of the latest technologies, equipment status, and industry standards, thereby improving the real-time and accuracy of questions and answers.

[0117] In the actual application process, first, the model extracts equipment, operation procedures, and fault information from the power industry data, and then builds a knowledge graph that reflects the importance of entities through the PageRank algorithm. Then, the model uses the training set of questions and answers, combined with the knowledge graph embedding information to establish a generative question-answering model, and generates a loss function through soft cosine similarity optimization. Then, the model hyperparameters are automatically adjusted in combination with the particle swarm optimization algorithm. Finally, the model receives questions from the power industry and generates accurate answers. Combining structured knowledge and language generation capabilities improves the model's reasoning accuracy and question-answer consistency.

[0118] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0119] In the present invention, by connecting the knowledge graph with node weights to the scaled dot product attention layer of the generative question-answering model, and then using the autoregressive decoder to output the answer to the question, the interference caused by the training process is compensated, the weight relationship between different knowledge nodes is captured more accurately, and the focus on key knowledge points is improved. On the basis of the autoregressive decoder, the model refers to the weight information of the knowledge graph in each step of the generation process to ensure that the generated answers are more logical, accurate and contextually consistent, and the consistency, logical rationality and accuracy of the answers can still be guaranteed based on the knowledge graph even in limited context windows and multi-round conversations. Combined with the node weights determined by the PageRank algorithm with an adaptive damping coefficient, the importance of the nodes in the graph can be dynamically adjusted to ensure that the correlation and influence between nodes are more accurately reflected in different situations, thereby providing more accurate weight distribution in complex networks, and further improving the reasoning ability of the model and the reliability of the answers. In the training process of the generative question-answering model, a generative loss function based on soft cosine similarity is used, which can capture the semantic similarity between texts more finely, especially for long texts and inconsistent word orders. The particle swarm optimization algorithm based on the multi-state dynamic inertia weight calculated based on particle swarm diversity and particle swarm fitness is combined for hyperparameter optimization. It can adaptively adjust according to the search state at different optimization stages, balance global search and local development, and improve the efficiency of hyperparameter tuning and the semantic consistency and accuracy of model generation results.

[0120] Reference Manual Attached Figure 3 , showing a structural schematic diagram of a power industry knowledge question and answer system based on a large language model provided by the present invention.

[0121] The present invention further provides a power industry knowledge question answering system 20 based on a large language model, which is applied to the above-mentioned power industry knowledge question answering method based on a large language model, comprising:

[0122] Processor 201.

[0123] The memory 202 stores computer-readable instructions. When the computer-readable instructions are executed by the processor 201, the electric power industry knowledge question-answering method based on a large language model as in the method embodiment is implemented.

[0124] The electric power industry knowledge question and answer system 20 based on a large language model provided by the present invention can execute the above-mentioned electric power industry knowledge question and answer method based on a large language model, and achieve the same or similar technical effects. To avoid repetition, the present invention will not go into details.

[0125] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0126] In the present invention, by connecting the knowledge graph with node weights to the scaled dot product attention layer of the generative question-answering model, and then using the autoregressive decoder to output the answer to the question, the interference caused by the training process is compensated, the weight relationship between different knowledge nodes is captured more accurately, and the focus on key knowledge points is improved. On the basis of the autoregressive decoder, the model refers to the weight information of the knowledge graph in each step of the generation process to ensure that the generated answers are more logical, accurate and contextually consistent, and the consistency, logical rationality and accuracy of the answers can still be guaranteed based on the knowledge graph even in limited context windows and multi-round conversations. Combined with the node weights determined by the PageRank algorithm with an adaptive damping coefficient, the importance of the nodes in the graph can be dynamically adjusted to ensure that the correlation and influence between nodes are more accurately reflected in different situations, thereby providing more accurate weight distribution in complex networks, and further improving the reasoning ability of the model and the reliability of the answers. In the training process of the generative question-answering model, a generative loss function based on soft cosine similarity is used, which can capture the semantic similarity between texts more finely, especially for long texts and inconsistent word orders. The particle swarm optimization algorithm based on the multi-state dynamic inertia weight calculated based on particle swarm diversity and particle swarm fitness is combined for hyperparameter optimization. It can adaptively adjust according to the search state at different optimization stages, balance global search and local development, and improve the efficiency of hyperparameter tuning and the semantic consistency and accuracy of model generation results.

[0127] It should be understood that the processor in the embodiment of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0128] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus random access memory (DR RAM).

[0129] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0130] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0131] In the present invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can be represented by: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0132] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0133] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0134] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0135] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0136] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0137] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0138] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, etc., various media that can store program codes.

[0139] An embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the method for answering questions about knowledge in the electric power industry based on a large language model as described in the method embodiment is implemented.

[0140] A computer-readable storage medium provided by the present invention can implement the steps and effects of the power industry knowledge question and answer method based on a large language model in the above-mentioned method embodiment. To avoid repetition, the present invention will not go into details.

[0141] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0142] In the present invention, by connecting the knowledge graph with node weights to the scaled dot product attention layer of the generative question-answering model, and then using the autoregressive decoder to output the answer to the question, the interference caused by the training process is compensated, the weight relationship between different knowledge nodes is captured more accurately, and the focus on key knowledge points is improved. On the basis of the autoregressive decoder, the model refers to the weight information of the knowledge graph in each step of the generation process to ensure that the generated answers are more logical, accurate and contextually consistent, and the consistency, logical rationality and accuracy of the answers can still be guaranteed based on the knowledge graph even in limited context windows and multi-round conversations. Combined with the node weights determined by the PageRank algorithm with an adaptive damping coefficient, the importance of the nodes in the graph can be dynamically adjusted to ensure that the correlation and influence between nodes are more accurately reflected in different situations, thereby providing more accurate weight distribution in complex networks, and further improving the reasoning ability of the model and the reliability of the answers. In the training process of the generative question-answering model, a generative loss function based on soft cosine similarity is used, which can capture the semantic similarity between texts more finely, especially for long texts and inconsistent word orders. The particle swarm optimization algorithm based on the multi-state dynamic inertia weight calculated based on particle swarm diversity and particle swarm fitness is combined for hyperparameter optimization. It can adaptively adjust according to the search state at different optimization stages, balance global search and local development, and improve the efficiency of hyperparameter tuning and the semantic consistency and accuracy of model generation results.

[0143] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

[0144] There are a few points to note:

[0145] (1) The drawings of the embodiments of the present invention only relate to the structures related to the embodiments of the present invention, and other structures may refer to the general design.

[0146] (2) For the sake of clarity, in the drawings used to describe the embodiments of the present invention, the thickness of the layers or regions is exaggerated or reduced, that is, these drawings are not drawn according to the actual scale. It is understood that when an element such as a layer, film, region or substrate is referred to as being "on" or "under" another element, the element may be "directly" "on" or "under" the other element or there may be intermediate elements.

[0147] (3) In the absence of conflict, the embodiments of the present invention and the features therein may be combined with each other to obtain new embodiments.

[0148] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. The protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A knowledge question-answering method for the electric power industry based on a large language model, characterized in that: include: S1: Obtain power industry data; S2: Based on the power industry data, a knowledge graph with node weights is constructed in combination with a PageRank algorithm with an adaptive damping coefficient, wherein the knowledge graph includes nodes representing various power entities and edges representing relationships between various power entities; S3: Obtain a training set with power industry questions as features and power industry answers as labels; S4: constructing a generative question-answering model with knowledge graph embedding information in combination with the knowledge graph, wherein the generative question-answering model comprises an encoder, a scaled dot product attention layer, and an autoregressive decoder connected in sequence, wherein the knowledge graph is connected to the scaled dot product attention layer through a graph embedding layer; S5: Establishing a generation loss function in combination with soft cosine similarity to evaluate the generation loss of the answers generated by the generative question-answering model in the power industry; S6: In combination with a particle swarm optimization algorithm with a multi-state dynamic inertia weight, with the goal of generating a loss function value less than a preset generation loss function value, the generative question answering model is trained using the training set to determine the optimal hyperparameters of the generative question answering model, wherein the multi-state dynamic inertia weight is an inertia weight calculated based on particle swarm diversity and particle swarm fitness; S7: Obtain pending power industry issues; S8: Input the pending power industry question into the trained generative question-answering model, and output an expected power industry answer corresponding to the pending power industry question.

2. The electric power industry knowledge question answering method based on a large language model according to claim 1 is characterized in that: The S2 specifically includes: S201: extracting basic data from the power industry data, wherein the basic data includes the power entity, power entity attributes, the power entity relationship and the power entity relationship attributes; S202: Connect each node according to the basic data to construct a primary knowledge graph: G=(V,E) v i =(e i ,A i ),v i ∈V Among them, G represents the primary knowledge graph composed of the node set V and the edge set E, v i The power entity is e i The power entity attribute is A i The i-th node, e ij Indicates that the power entity relationship between the i-th node and the j-th node is r ij , the power entity relationship attribute between the i-th node and the j-th node is The edge of S203: Initialize the PageRank value of each node, where the PageRank value is the reciprocal value of the total number of nodes: Among them, PR (v i ) represents the i-th node v i The PageRank value of, N represents the total number of nodes; S204: Update the PageRank value of each node according to the number of adjacent nodes and out-degree of each node: Among them, PR'(ν i ) represents node v i The updated PageRank value of , d represents the adaptive damping coefficient representing the probability of randomly jumping to other nodes in each calculation, O(v i ) represents node v i The incoming neighbor set of node v i The adjacent node set of PR(v j ) represents node v j The PageRank value of N represents the total number of nodes. and They represent the average out-degree and average in-degree of nodes in the primary knowledge graph, H and H max They represent the actual information entropy of the primary knowledge graph and the maximum information entropy under the condition of completely uniform node access distribution, log represents the logarithmic function, P(v i ) represent the nodes v i The probability of being visited before the update is the PageRank value; S205: Add the updated PageRank value as the weight of the corresponding node to the primary knowledge graph to obtain a knowledge graph with node weights.

3. The electric power industry knowledge question-answering method based on a large language model according to claim 1 is characterized in that: The generative question answering model also includes an input layer and an output layer, wherein the input layer is connected to the encoder and the output layer is connected to the autoregressive decoder.

4. The electric power industry knowledge question answering method based on a large language model according to claim 1 is characterized in that: The generation loss function is specifically: Among them, L oss represents the generated loss function, represents the total probability of obtaining the answer generated by the power industry, log represents the logarithmic function, and Y represents the label, i.e., the true answer of the power industry. Indicates the power industry corresponding to the label generates the answer, Indicates that Y and The soft cosine similarity between p represents the pth word in Y, express The qth word in Y, S(p,q) represents p and The cosine similarity between them, S(p,p) and S(q,q) both represent the self-cosine similarity with a value of 1.

5. The electric power industry knowledge question answering method based on a large language model according to claim 1 is characterized in that: The optimal hyperparameters include the optimal learning rate, optimal batch size, optimal number of encoder layers, optimal number of decoder layers and optimal Dropout rate of the generative question answering model.

6. The electric power industry knowledge question answering method based on a large language model according to claim 1 is characterized in that: The S6 specifically includes: S601: Input the features of the training set, i.e., the power industry problem, into the encoder to obtain the feature hidden state: h i =Encoder(x i ,θ enc ) Among them, h i Represents the i-th word x in the input power industry problem i The hidden state of Encoder represents the Transformer encoder, θ enc Represents the Transformer encoder parameter set; S602: Convert the knowledge graph into an embedding representation of a numerical vector through the graph embedding layer: z v GAT(h v ,h N(v) ,A,θ gat ) Among them, z ν represents the embedding representation of the v node in the knowledge graph, GAT represents the graph embedding layer representing the graph attention network, and h N(ν) represents the neighbor node feature representation of node v in the graph embedding layer, h v represents the feature representation of v nodes in the graph embedding layer, A represents the adjacency matrix of the knowledge graph, and θ gat Represents the graph embedding layer parameters representing the weight matrices of each layer of the graph embedding layer; S603: In combination with the node PageRank value, the attention scores of the feature hidden state and the embedded representation are calculated respectively through the scaled dot product attention layer: Among them, α i,j represents the attention score between the i-th hidden state and the j-th knowledge graph node, q i represents the corresponding query vector obtained by transforming the hidden state of the i-th word, exp represents the natural exponential function, k j and k k They represent the corresponding key vectors obtained by transforming the embedding representation of the jth node and the embedding representation of the kth node, respectively. k Indicates k k The key vector dimension is , PageRank(j) represents the PageRank value of node j, and T represents transposition; S604: In combination with the attention score, the autoregressive decoder is used to sequentially generate answer words based on the feature hidden state and the embedded representation in a cascade manner, wherein each answer word constitutes a generated label, i.e., the power industry generates an answer: y t =Decoder(y t-1 ,h,z,a i,j ,i dec ) Among them, y t and t-1 Denote the answer words generated by the autoregressive decoder at the t-th time step and the t-1-th time step, respectively. Decoder denotes the autoregressive decoder. h denotes the hidden state of the input power industry question. z denotes the embedded representation of the input power industry question. represents the answer generated by the power industry, y i represents the answer word generated at the i-th time step, i = 1, 2, ..., T', T' represents the total number of answer words generated; S605: Taking the generation loss function value being less than the preset generation loss function value as the optimization target, determining the multi-state dynamic inertia weight based on the particle swarm diversity and the particle swarm fitness change rate, adjusting the hyperparameters of the generative question-answering model through the particle swarm optimization algorithm of the multi-state dynamic inertia weight, and outputting the optimal hyperparameters.

7. The electric power industry knowledge question-answering method based on a large language model according to claim 6 is characterized in that: The S605 specifically includes: S6051: Determine the generated loss function value as the objective function; S6052: Initializing hyperparameters of different values ​​as particles to determine particle positions and particle speeds, wherein the particle positions are parameter vectors composed of the hyperparameters; S6053: Use the multi-state dynamic inertia weights determined based on the Fermat principle to determine the particle position update formula and particle velocity update formula: x i (t+1)=x i (t)+v i (t+1) Among them, x i (t) and x i (t+1) represents the particle position at time t and the particle position at time t+1, respectively, v i (t) and v i (t+1) represents the particle velocity at time t and the particle velocity at time t+1 respectively, w(t) represents the multi-state dynamic inertia weight at time t that controls the inertia of particle movement, c1 and c2 both represent learning factors with a value of 2, l1 and l2 represent the first random number and the second random number with a value range of 0 to 1 respectively, represents the best historical position of particle i, g best represents the global best position, i.e., the best historical position found by all particles. d(t) represents the average distance between particles at time t, which represents the diversity of the particle swarm. min(d) = 10 -6 represents the minimum diversity of the particle group, U represents the total number of particles, represents the center position of the particle swarm at time t, ΔFitness(t) represents the fitness difference between the particle swarm at time t and time t-1 in different iterations, and ΔFitness max Indicates the maximum value of the fitness change of the particle swarm during the historical iteration process; S6054: updating the particles using the particle position update formula and the particle velocity update formula respectively, and calculating the update objective function value of the updated particles under the training set; S6055: Determine whether the difference between the updated objective function value and the objective function value before updating is less than a preset difference. If so, proceed to step S6056; otherwise, return to step S6054; S6056: Output the updated particle position as the optimal particle position, and output the hyperparameter corresponding to the optimal particle position as the optimal hyperparameter, and end the hyperparameter adjustment process.

8. The electric power industry knowledge question-answering method based on a large language model according to claim 1 is characterized in that: After S8, the method further includes: The electric power industry data is updated at preset time intervals, and the knowledge graph and the generative question-answering model are updated based on the updated electric power industry data.

9. A knowledge question-answering system for the electric power industry based on a large language model, characterized in that: include: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the electric power industry knowledge question-answering method based on a large language model as described in any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the power industry knowledge question and answer method based on a large language model as described in any one of claims 1 to 8 is implemented.

Images