A knowledge question-answering method and system for the power industry based on a large language model

Abstract

The present invention provides a method and system for answering questions about the electric power industry based on a large language model, which relates to the field of artificial intelligence technology. The method includes: obtaining electric power industry data; constructing a knowledge graph with node weights; obtaining a training set with electric power industry questions as features and electric power industry answers as labels; constructing a generative question-answering model with knowledge graph embedded information; establishing a generation loss function; training the generative question-answering model with the goal of ensuring that the generation loss function value is less than a preset generation loss function value to determine the optimal hyperparameters of the generative question-answering model; obtaining a pending electric power industry question; inputting the pending electric power industry question into the trained generative question-answering model, and outputting an expected electric power industry answer corresponding to the pending electric power industry question. By combining the knowledge graph with node weights and the generative question-answering model, the interference of training set noise on the model's output answers is reduced, and the consistency and accuracy of the generated answers are improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a knowledge question-answering method and system for the power industry based on a large language model. Background Art

[0002] Power industry knowledge refers to professional knowledge related to the operation, maintenance, management and optimization of power systems, covering areas such as power equipment (such as transformers, generators), power dispatching, distribution networks, transmission lines, fault diagnosis, maintenance procedures, safety regulations and energy management. This knowledge is usually highly professional and complex.

[0003] The power industry involves a vast array of complex equipment, operational processes, and fault diagnosis. Traditional information retrieval methods struggle to provide fast, accurate, and professional answers. Large language models can understand natural language questions and, combined with power industry expertise, provide accurate, contextually relevant answers. This improves problem-solving efficiency, reduces human error, and provides intelligent support to power engineers and operators, contributing to improved overall operational efficiency and safety.

[0004] However, existing knowledge question-answering in the power industry only uses the semantic analysis capabilities of the large language model to infer answers when introducing the large language model. This is random, and the generated answers are easily affected by the noise of the training data, resulting in deviations in the accuracy of the question-answering. In addition, due to the limited context window of the large model, when the conversation length exceeds this window, the model cannot remember the previous conversation content. As a result, in long conversations or multi-round question-answering, the model may ignore previous discussions, resulting in a broken context for the generated answers and an inability to maintain long-term memory and consistency. Summary of the Invention

[0005] In order to solve the technical problem that the existing technology of electric power industry knowledge question and answer only uses the semantic analysis ability of the large language model to infer the answer during the introduction of the large language model, which is random and the generated answers are easily disturbed by the noise of the training data, resulting in deviations in the accuracy of question and answer. In addition, due to the limited context window of the large model, when the conversation length exceeds this window, the model cannot remember the previous conversation content. As a result, in long conversations or multi-round question and answer, the model may ignore the previous discussion, resulting in a broken context of the generated answer and an inability to maintain long-term memory and consistency, the present invention provides a method and system for electric power industry knowledge question and answer based on a large language model.

[0006] The technical solutions provided by the embodiments of the present invention are as follows:

[0007] First aspect

[0008] An embodiment of the present invention provides a method for answering questions about the electric power industry based on a large language model, including:

[0009] S1: Obtain power industry data;

[0010] S2: Based on the power industry data, a PageRank algorithm with an adaptive damping coefficient is used to construct a knowledge graph with node weights. The knowledge graph includes nodes representing various power entities and edges representing the relationships between these entities.

[0011] S3: Obtain a training set with power industry questions as features and power industry answers as labels;

[0012] S4: Build a generative question answering model with knowledge graph embedding information in combination with the knowledge graph, where the generative question answering model includes an encoder, a scaled dot product attention layer, and an autoregressive decoder connected in sequence, where the knowledge graph is connected to the scaled dot product attention layer through a graph embedding layer;

[0013] S5: Combined with soft cosine similarity, a generation loss function was established to evaluate the generation loss of the answers generated by the generative question answering model in the power industry.

[0014] S6: Combined with the particle swarm optimization algorithm with multi-state dynamic inertia weights, with the goal of generating a loss function value less than the preset generation loss function value, the generative question answering model is trained using the training set to determine the optimal hyperparameters of the generative question answering model;

[0015] S7: Obtain pending power industry issues;

[0016] S8: Input the power industry question to be processed into the trained generative question-answering model, and output the expected power industry answer corresponding to the power industry question to be processed.

[0017] Second aspect

[0018] An embodiment of the present invention provides a knowledge question-answering system for the electric power industry based on a large language model, comprising:

[0019] processor;

[0020] A memory stores computer-readable instructions, which, when executed by a processor, implement the electric power industry knowledge question-answering method based on a large language model as described in the first aspect.

[0021] The third aspect

[0022] An embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the method for answering questions about the knowledge of the electric power industry based on a large language model according to the first aspect is implemented.

[0023] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0024] In the present invention, by connecting the knowledge graph with node weights to the scaled dot product attention layer of the generative question-answering model, and then using the autoregressive decoder to output the answer to the question, the interference caused by the training process is compensated, the weight relationship between different knowledge nodes is captured more accurately, and the attention to key knowledge points is improved. On the basis of the autoregressive decoder, the model refers to the weight information of the knowledge graph in each step of the generation process to ensure that the generated answers are more logical, accurate and contextually consistent. Even in the case of limited context windows and multi-round conversations, the consistency, logical rationality and accuracy of the answers can still be guaranteed based on the knowledge graph. Combined with the node weights determined by the PageRank algorithm with an adaptive damping coefficient, the importance of the nodes in the graph can be dynamically adjusted to ensure that the correlation and influence between nodes are more accurately reflected in different situations, thereby providing more accurate weight distribution in complex networks and further improving the reasoning ability of the model and the reliability of the answers. During the training process of the generative question-answering model, a generative loss function based on soft cosine similarity is used, which can more finely capture the semantic similarity between texts, especially for long texts and inconsistent word orders. The particle swarm optimization algorithm with multi-state dynamic inertia weights calculated based on particle swarm diversity and particle swarm fitness is combined for hyperparameter optimization. It can adaptively adjust according to the search state at different optimization stages, balance global search and local development, and improve the efficiency of hyperparameter tuning and the semantic consistency and accuracy of model generation results. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 A flowchart of a method for answering questions about the electric power industry based on a large language model provided by an embodiment of the present invention;

[0027] Figure 2 A schematic diagram of the structure of a generative question-answering model provided by an embodiment of the present invention;

[0028] Figure 3 A schematic diagram of the structure of a knowledge question-answering system for the electric power industry based on a large language model provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0029] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0030] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as an "exemplary" in the present invention should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of the word "exemplary" is intended to present concepts in a concrete manner. Furthermore, in the embodiments of the present invention, "and / or" can mean both or either of the two.

[0031] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0032] Reference Manual Figure 1 , which shows a flow chart of a knowledge question-answering method for the power industry based on a large language model provided by an embodiment of the present invention.

[0033] An embodiment of the present invention provides a method for answering questions about the electric power industry based on a large language model. This method can be implemented by a device for answering questions about the electric power industry based on a large language model, which can be a terminal or a server. The processing flow of the method for answering questions about the electric power industry based on a large language model can include the following steps:

[0034] S1: Obtain power industry data.

[0035] Power industry data includes power system equipment information, operating data, maintenance records, operational procedures, fault diagnosis, and repair procedures. This data covers the entire process from power generation to distribution, including the operating status of power equipment, historical operating records, real-time monitoring data, and relevant technical standards and specifications. It forms the foundation for building knowledge graphs and training generative question-answering models, helping them accurately understand and answer specialized questions in the power industry.

[0036] S2: Based on the power industry data, a knowledge graph with node weights is constructed in combination with the PageRank algorithm with an adaptive damping coefficient.

[0037] The knowledge graph includes nodes representing various power entities and edges representing the relationships between various power entities.

[0038] Among them, the adaptive damping coefficient is a parameter in the PageRank algorithm, which is used to control the probability of random jumps. PageRank is a graph-based sorting algorithm that was first used for web page sorting. It iteratively calculates the influence of each node in the graph, taking into account the node's out-degree and connection relationship with other nodes. The higher the PageRank value, the greater the importance of the node in the network. The knowledge graph is a structured data representation that uses nodes and edges to represent entities (such as devices) and their relationships. It can help the model understand the knowledge hierarchy and association relationships in complex fields and provide clear structural information. The node weight indicates the importance of each node in the graph. The larger the weight, the more influential or more important the node is in the knowledge graph.

[0039] By constructing a knowledge graph with node weights through the PageRank algorithm with adaptive damping coefficient adjustment, it can more accurately reflect the connections and influence between entities in the power industry, enabling the generative question-answering model to effectively focus on important knowledge nodes, thereby improving the accuracy of model reasoning and the logic of answer generation.

[0040] In a possible implementation, S2 specifically includes:

[0041] S201: Extracting basic data from power industry data, wherein the basic data includes power entities, power entity attributes, power entity relationships, and power entity relationship attributes.

[0042] Specifically, entities and entity attributes are extracted through the entity extraction model, and relations and relationship attributes are extracted through the relationship extraction model.

[0043] S202: Connect each node according to the basic data to build a primary knowledge graph:

[0044] G=(V,E)

[0045] v i =(e i ,A i ),v i ∈V

[0046]

[0047] Among them, G represents the primary knowledge graph consisting of the node set V and the edge set E, v i The power entity is e i The power entity attribute is A i The i-th node, e ij The power entity relationship between the i-th node and the j-th node is r ij , the power entity relationship attribute between the i-th node and the j-th node is edge.

[0048] S203: Initialize the PageRank value of each node, where the PageRank value is the reciprocal value of the total number of nodes:

[0049]

[0050] Among them, PR(v i ) represents the i-th node v i The PageRank value of , N represents the total number of nodes.

[0051] S204: Update the PageRank value of each node according to the number of adjacent nodes and out-degree of each node:

[0052]

[0053] Among them, PR'(v i ) represents node v i The updated PageRank value, d represents the adaptive damping coefficient representing the probability of random jump to other nodes in each calculation, O(v i ) represents node v i The incoming neighbor set of node v i The adjacent node set, PR(v j ) represents node v j The PageRank value of N represents the total number of nodes. and They represent the average out-degree and average in-degree of nodes in the primary knowledge graph, H and H max They represent the actual information entropy of the primary knowledge graph and the maximum information entropy under the completely uniform distribution of node access, log represents the logarithmic function, P(v i ) represent the nodes v i The probability of being visited before the update is the PageRank value.

[0054] Among them, the average out-degree of a node indicates how many edges each node has pointing to other nodes in the entire primary knowledge graph, reflecting the overall connectivity between nodes. The average in-degree of a node indicates the average in-degree of all nodes in the primary knowledge graph, indicating how many edges each node has pointing to it on average. Specifically, It represents the normalized value of information entropy, ranging from 0 to 1. When H=H maxWhen H is , the access probability of the nodes is completely uniformly distributed, the randomness is maximized, and the damping coefficient should be closer to 1. When H is small, the system is less random and the access between nodes is uneven. In this case, the damping coefficient should be smaller to better utilize random jumps. By combining information entropy, which reflects the randomness and uncertainty of the system, we can ensure that the overall randomness and structural tightness of the system are balanced. Information entropy combines the connectivity of the nodes (out-degree and in-degree) and the access probability distribution to more accurately select the damping coefficient.

[0055] It should be noted that in the calculation process of the damping coefficient, when the out-degree and in-degree are larger, the damping coefficient should tend to 1, indicating that the relationship between the nodes in the graph is stronger. When the out-degree and in-degree are smaller, the damping coefficient should be reduced to better utilize random jumps to avoid falling into an infinite loop. Among them, O(v i ) is the node v i The set of all incoming edge neighbors points to node v i The setting of the damping coefficient avoids the problem of the calculation falling into an infinite loop. Optionally, the specific value of d can be 0.85. By combining the average out-degree, average in-degree and information entropy of the nodes to calculate the PageRank value, it is possible to more accurately balance the structural associations and random jumps between nodes, taking into account the density of node connections in the graph and reflecting the randomness and uncertainty of the system through information entropy. This method effectively avoids local structural traps, ensures the stability and wide applicability of the algorithm, and at the same time improves its adaptability to complex network structures, making the generated PageRank value more accurate and reasonable.

[0056] S205: Add the updated PageRank value as the weight of the corresponding node to the primary knowledge graph to obtain a knowledge graph with node weights.

[0057] It should be noted that the knowledge graph constructed using the PageRank algorithm with an adaptive damping coefficient can more accurately reflect the connections and importance between power entities. This method comprehensively considers the out-degree and in-degree of nodes, as well as the randomness of the system, enabling the knowledge graph to better capture the structural features of the graph and effectively avoid local structural traps, improving the stability and adaptability of the algorithm, thereby generating a more accurate and reliable knowledge graph and enhancing the reliability of power Q&A.

[0058] S3: Obtain a training set with power industry questions as features and power industry answers as labels.

[0059] Features are the model's input data, representing the task context we want the model to learn. In the power industry Q&A model, features refer to questions related to the power industry posed by users. Labels are the model's output data, representing the target the model should learn to generate. Labels are the standard answers or solutions corresponding to the features and are the correct outputs the model should generate through learning. The training set is used to supervise the model's learning. By learning the relationship between questions and corresponding answers, the model gradually masters how to generate accurate answers based on the questions.

[0060] Reference Manual Figure 2 , which shows a structural diagram of a generative question-answering model provided by an embodiment of the present invention.

[0061] S4: Combine knowledge graphs to build a generative question-answering model with knowledge graph embedding information.

[0062] Among them, the generative question answering model includes an encoder, a scaled dot product attention layer and an autoregressive decoder connected in sequence, wherein the knowledge graph is connected to the scaled dot product attention layer through a graph embedding layer.

[0063] The encoder is responsible for processing the input power industry questions and converting natural language into numerical representations (vectors). The scaled dot product attention layer is used to capture the correlation between the various parts of the input features, focusing on important keywords or information in the question. The autoregressive decoder is responsible for gradually generating the answer based on the vector representation generated by the encoder. During the autoregressive process, each time the model generates a word, it will use the previously generated word as input and continue to generate the next word until the complete answer is output. The graph embedding layer embeds the nodes and relationships in the knowledge graph into the vector space. By connecting with the attention layer, the information in the knowledge graph is injected into the model, helping the model better understand the knowledge structure and entity relationships of the power industry.

[0064] It's important to note that by embedding the knowledge graph into the encoder and scaled dot-product attention layer of the generative question-answering model, the model leverages structured knowledge from the power industry to better understand the semantics and context of the question during the generation process, improving its ability to accurately capture and reason about key concepts. Combined with the weight information from the knowledge graph, the autoregressive decoder generates more professional, logically coherent, and contextually consistent answers, significantly improving the accuracy and relevance of answers, particularly in complex power industry question-answering scenarios.

[0065] Specifically, when processing conversations, traditional language models are unable to utilize entire long documents due to input length limitations, which can lead to the loss of important knowledge and affect the quality of the generated conversations. In the power industry, since power analysis and reasoning is complex and requires cascaded reasoning, if the knowledge from the previous stage cannot be accurately recorded and learned, the generated power industry answers will lack depth and will not be able to effectively analyze and give accurate answers. The introduction of knowledge graphs converts key information in documents into triples and constructs graph structures. These knowledge graphs can effectively summarize long documents, ensuring that important information is included in the input, allowing the model to reduce the learning length limit of the model itself when answering, and improve the accuracy and coherence of answer generation. By converting documents into knowledge graphs, the model overcomes the capacity limitation problem of pre-trained language models when processing long documents, that is, by condensing the key information of long documents through graph structures, avoiding the loss of important knowledge caused by truncating documents.

[0066] In practical applications, incorporating a search-based knowledge graph into the generative question-answering model's generation process makes the output answers more logical. The connections in the indicator graph can also increase the accuracy of answers during multi-turn conversations. The knowledge graph can also reference device attributes, processes, and standards to generate more professional responses. Generative models can sometimes generate vague and imprecise answers when handling complex and technical questions. Introducing a knowledge graph can constrain the generation process, making the model's answers more accurate and consistent, and avoiding incorrect or irrelevant responses. When answering questions involving multiple devices or multiple steps, the knowledge graph can help the model disambiguate and generate clear answers based on the relationships between devices and processes. In multi-turn conversations, users may ask multiple consecutive questions with complex contextual relationships. The knowledge graph can preserve the contextual information in the conversation and the logical relationships between devices, helping the model generate coherent answers. This is particularly effective for questions that span multiple devices and operational processes.

[0067] In one possible implementation, the generative question answering model further includes an input layer and an output layer, wherein the input layer is connected to the encoder, and the output layer is connected to the autoregressive decoder.

[0068] S5: Combine soft cosine similarity to establish a generation loss function to evaluate the generation loss of the generative question answering model for the power industry.

[0069] Soft cosine similarity is an extension of cosine similarity. It considers not only the angle between vectors but also the semantic similarity between vector elements (such as words). Traditional cosine similarity only compares the values ​​of two vectors and does not consider the relationship between elements. Soft cosine similarity, however, introduces a similarity matrix between elements. This allows the model to detect similarity even if two words are not exactly the same, as long as they are semantically related. This ensures that the generated answers meet expectations while reducing the training difficulty.

[0070] In one possible implementation, the loss function is generated as follows:

[0071]

[0072] Among them, L oss represents the generated loss function, represents the total probability of obtaining the answer generated by the power industry, log represents the logarithmic function, and Y represents the label, i.e., the true answer of the power industry. Indicates the power industry corresponding to the label generates the answer, Indicates that Y and The soft cosine similarity between p represents the pth word in Y, express The qth word in Y, S(p,q) represents p and The cosine similarity between them, S(p,p) and S(q,q) both represent the cosine similarity of themselves with a value of 1.

[0073] It's important to note that by taking the logarithm of the generation probability, we can amplify the penalty for low-probability generation results, encouraging the model to generate high-quality answers that are more consistent with expectations. This allows the loss function to consider both generation accuracy and the flexibility provided by soft cosine similarity. The inclusion of soft cosine similarity prevents excessive losses caused by single word errors. Even if subtle errors occur during generation, as long as the overall semantics remain similar, the loss function will not impose excessive penalties. This makes the model optimization process more stable and reduces potential fluctuations in the generation process. Furthermore, soft cosine similarity guides the model to focus on overall semantic matching rather than solely on lexical form, thereby improving the quality of generated content. This considers both the accuracy of the generated answer and the ground-truth answer (via log probability) and semantic similarity. Through soft cosine similarity, even if the generated answer differs from the ground-truth answer in vocabulary but is semantically similar, the loss will not be excessively high. This improves the flexibility and robustness of the generative model, enabling more effective model training and improved training results.

[0074] S6: Combined with the particle swarm optimization algorithm with multi-state dynamic inertia weights, with the goal of generating a loss function value less than the preset generation loss function value, the generative question answering model is trained using the training set to determine the optimal hyperparameters of the generative question answering model.

[0075] Among them, the multi-state dynamic inertia weight is the inertia weight calculated based on the particle swarm diversity and particle swarm fitness.

[0076] The multi-state dynamic inertia weight is a dynamic parameter in particle swarm optimization that controls the search behavior of the particle swarm. At different stages, the inertia weight is adjusted based on the diversity and fitness of the particle swarm. A larger inertia weight encourages global exploration, while a smaller weight promotes local search, allowing the algorithm to dynamically balance global search and local optimization. The particle swarm optimization algorithm is an optimization algorithm based on swarm intelligence that simulates the collaborative behavior of a swarm of particles. Each particle represents a solution. By tracking its own optimal solution and the optimal solution in the swarm, the particle continuously adjusts its position and ultimately converges to the optimal solution. The generation loss function represents the error in the model's answer generation. The loss function measures the difference between the generated answer and the correct answer. The smaller the generation loss value, the better the model's generation performance. Optimal hyperparameters are a set of hyperparameter combinations found through methods such as particle swarm optimization that achieve the best performance for a generative question answering model on a specific task. Particle diversity measures the degree of variation between particles in a swarm and reflects the distribution of the swarm in the search space. The swarm fitness evaluates the quality of the solution corresponding to each particle. Fitness is usually calculated based on the value of the generated loss function. The higher the fitness, the closer the particle's solution is to the global optimal solution.

[0077] By combining the particle swarm optimization algorithm with multi-state dynamic inertia weights, the model can adaptively adjust the search range, accelerate local optimization while ensuring global search, improve the optimization efficiency of the hyperparameters of the generative question-answering model, and ensure that the generation loss converges to the optimal solution faster, making model training more efficient and accurate.

[0078] It should be noted that those skilled in the art can set the size of the generated loss function value according to actual needs, and the present invention does not limit this.

[0079] In one possible implementation, the optimal hyperparameters include an optimal learning rate, an optimal batch size, an optimal number of encoder layers, an optimal number of decoder layers, and an optimal dropout rate for the generative question answering model.

[0080] In a possible implementation, S6 specifically includes:

[0081] S601: Input the features of the training set, i.e., the power industry problem, into the encoder to obtain the hidden state of the features:

[0082] h i =Encoder(x i ,θ enc )

[0083] Among them, h i Represents the i-th word x in the input power industry problem i The hidden state of Encoder represents the Transformer encoder, θ enc Represents a Transformer encoder parameter set.

[0084] S602: Convert the knowledge graph into an embedded representation of a numerical vector through the graph embedding layer:

[0085] z v =GAT(h v ,h N(v) ,A,θ gat )

[0086] Among them, z v represents the embedding representation of v nodes in the knowledge graph, GAT represents the graph embedding layer representing the graph attention network, and h N(v) Represents the neighbor node feature representation of node v in the graph embedding layer, h v represents the v-node feature representation in the graph embedding layer, A represents the adjacency matrix of the knowledge graph, and θ gat Represents the graph embedding layer parameters representing the weight matrices of each layer of the graph embedding layer.

[0087] S603: Combined with the node PageRank value, the attention scores of the feature hidden state and the embedded representation are calculated respectively through the scaled dot product attention layer:

[0088]

[0089] Among them, α i,j represents the attention score between the i-th hidden state and the j-th knowledge graph node, q i represents the corresponding query vector obtained by transforming the hidden state of the i-th word, exp represents the natural exponential function, k j and k k They represent the corresponding key vectors obtained by converting the embedding representation of the j-th node and the embedding representation of the k-th node, respectively. k represents k k The key vector dimension of , PageRank(j) represents the PageRank value of node j, and T represents transpose.

[0090] S604: Combined with the attention score, the autoregressive decoder is used in a cascaded manner to sequentially generate answer words based on the feature hidden state and the embedding representation. Each answer word constitutes a generated label, i.e., the generated answer for the power industry:

[0091] y t =Decoder(y t-1 ,h,z,α i,j ,θ dec )

[0092]

[0093] Among them, y t and y t-1 Denote the answer words generated by the autoregressive decoder at the t-th time step and the t-1-th time step, respectively. Decoder represents the autoregressive decoder, h represents the hidden state of the input power industry question, and z represents the embedded representation of the input power industry question. Indicates the answer generated by the power industry, y i Represents the answer word generated at the i-th time step, i = 1, 2, …, T', T' represents the total number of answer words generated.

[0094] Specifically, the cascade method means that after each answer word is generated, all the answer words are converted into hidden representations. When the next answer word is generated, the converted hidden representation is directly re-input into the decoder to intervene in the generation of the next answer word. The model generates only one word at a time. The first generated word will be used as input to generate the second word. After the second word is generated, it will be used as input to generate the third word. This process continues until a complete sentence is generated or a predetermined end mark (such as a special terminator) is reached. Each generated word depends not only on the input context (such as the conversation history or knowledge graph), but also on the previously generated words. In other words, each generated word affects the generation of subsequent words.

[0095] Specifically, the model first receives an input question or conversation context and encodes it. The output of this part is the understanding of the question or the encoding of the background context. During the decoding phase, the model generates the first word. This word is generated based on the input context and the knowledge graph. After the first word is generated, it is used as the input to the decoder in the next step and is passed to the model along with the previous context to generate the next word. Each time a new word is generated, it is input into the decoder along with the previously generated word, and the next word is generated until the model generates a complete answer sentence. The advantage of this autoregressive generation method is that it not only ensures contextual coherence and semantic consistency, but also effectively reduces the risk of error propagation through gradual generation and dynamic adjustment. Autoregressive generation is particularly suitable for tasks that require complex reasoning and multi-round dialogue because it can gradually process information and flexibly respond to changes in the task, thereby improving the accuracy and naturalness of the generated results.

[0096] S605: Taking the generation loss function value less than the preset generation loss function value as the optimization goal, determining the multi-state dynamic inertia weight based on the particle swarm diversity and the particle swarm fitness change rate, adjusting the hyperparameters of the generative question answering model through the particle swarm optimization algorithm of the multi-state dynamic inertia weight, and outputting the optimal hyperparameters.

[0097] In a possible implementation, S605 specifically includes:

[0098] S6051: Determine the generated loss function value as the objective function.

[0099] S6052: Initialize hyperparameters of different values ​​as particles and determine the particle position and particle velocity, where the particle position is a parameter vector composed of the hyperparameters.

[0100] S6053: Use the multi-state dynamic inertia weights determined based on the Fermat principle to determine the particle position update formula and particle velocity update formula:

[0101] x i (t+1)=x i (t)+v i (t+1)

[0102]

[0103] Among them, x i (t) and x i (t+1) represents the particle position at time t and the particle position at time t+1, v i (t) and v i(t+1) represents the particle velocity at time t and the particle velocity at time t+1, w(t) represents the multi-state dynamic inertia weight at time t that controls the inertia of particle movement, c1 and c2 both represent learning factors with a value of 2, l1 and l2 respectively represent the first random number and the second random number with a value range of 0 to 1, represents the best historical position of particle i, g best represents the global best position, i.e., the best historical position found by all particles. d(t) represents the average distance between particles at time t, which represents the diversity of the particle swarm. min(d) = 10 -6 represents the minimum diversity of the particle swarm, U represents the total number of particles, represents the center position of the particle swarm at time t, ΔFitness(t) represents the fitness difference between the particle swarm at time t and time t-1 in different iterative processes, and ΔFitness max Indicates the maximum value of the particle swarm fitness change during the historical iteration process.

[0104] Among them, the maximum value of the particle swarm fitness change will continue to decrease with the increase of the number of iterations. Therefore, selecting the maximum value of the particle swarm fitness change in the historical iteration process can ensure the normalization of the particle swarm fitness difference in the subsequent iteration process. The fitness here is the size of the loss function value of the model under the training set, and the particle swarm fitness difference is the difference between the loss function under the current iteration number and the previous iteration number.

[0105] Specifically, the Fermat Principle's role in calculating multi-state dynamic inertia weights can be viewed as an optimized "shortest path" principle. By adjusting the inertia weights, the system helps particles avoid searching excessive and unnecessary paths when finding the optimal solution, similar to the process of selecting the shortest path for light propagation. By adjusting the inertia weights, the particle swarm can dynamically switch search strategies based on its fitness and diversity at different stages. Similar to the process of light rays searching for the shortest path in Fermat's principle, the particle swarm will quickly reach the global optimal solution via the "optimal inertial path."

[0106] S6054: Update the particles using the particle position update formula and the particle velocity update formula respectively, and calculate the update objective function value of the updated particles under the training set.

[0107] S6055: Determine whether the difference between the updated objective function value and the objective function value before updating is less than the preset difference. If so, proceed to step S6056; otherwise, return to step S6054.

[0108] It should be noted that those skilled in the art can set the size of the preset difference according to actual needs, and the present invention is not limited thereto.

[0109] S6056: Output the updated particle position as the optimal particle position, and output the hyperparameter corresponding to the optimal particle position as the optimal hyperparameter, ending the hyperparameter adjustment process.

[0110] In steps S6051 to S6056, the model first generates the loss function value as the objective function, initializes different hyperparameters to particles, and calculates their positions and velocities. Then, using the dynamic inertia weights based on Fermat's principle, the particle positions and velocities are updated, and the search direction is gradually adjusted, allowing the particle swarm to quickly find the global optimal solution. Next, the model repeatedly calculates the updated objective function value and determines whether it is close to a preset threshold. Finally, the hyperparameters corresponding to the optimal particle position are output as the model's optimal hyperparameters. By adjusting the dynamic inertia weights, global search and local optimization are balanced, improving the efficiency and accuracy of the hyperparameter search and enabling the model to quickly converge to the optimal state.

[0111] It's important to note that by combining a particle swarm optimization algorithm with multi-state dynamic inertia weights for hyperparameter adjustment, the model can adaptively balance global search and local optimization based on changes in particle swarm diversity and fitness. This approach not only accelerates hyperparameter optimization but also avoids falling into local optimality, enabling generative question-answering models to achieve better hyperparameter combinations for complex tasks. This improves the model's generation quality, accuracy, and training efficiency, while ensuring that the generation loss quickly converges to the optimal solution.

[0112] S7: Obtain pending power industry issues.

[0113] S8: Input the power industry question to be processed into the trained generative question-answering model, and output the expected power industry answer corresponding to the power industry question to be processed.

[0114] In a possible implementation manner, after S8, the method further includes:

[0115] The power industry data is updated at preset intervals, and the knowledge graph and generative question-answering model are updated based on the updated power industry data.

[0116] By regularly updating power industry data and adjusting the knowledge graph and generative question-answering model based on this updated data, we ensure that the model always maintains an accurate grasp of the latest technologies, equipment status, and industry standards, thereby improving the real-time and accuracy of question-answering.

[0117] In practical application, the model first extracts information about equipment, operating procedures, and faults from power industry data. It then uses the PageRank algorithm to construct a knowledge graph reflecting the importance of entities. Next, the model uses a training set of questions and answers, combined with the knowledge graph embedding information, to build a generative question-answering model. The model then optimizes the generated loss function using soft cosine similarity. Next, it automatically adjusts model hyperparameters using a particle swarm optimization algorithm. Finally, the model receives power industry questions and generates accurate answers. This combination of structured knowledge and language generation improves the model's reasoning accuracy and question-answer consistency.

[0118] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0119] In the present invention, by connecting the knowledge graph with node weights to the scaled dot product attention layer of the generative question-answering model, and then using the autoregressive decoder to output the answer to the question, the interference caused by the training process is compensated, the weight relationship between different knowledge nodes is captured more accurately, and the attention to key knowledge points is improved. On the basis of the autoregressive decoder, the model refers to the weight information of the knowledge graph in each step of the generation process to ensure that the generated answers are more logical, accurate and contextually consistent. Even in the case of limited context windows and multi-round conversations, the consistency, logical rationality and accuracy of the answers can still be guaranteed based on the knowledge graph. Combined with the node weights determined by the PageRank algorithm with an adaptive damping coefficient, the importance of the nodes in the graph can be dynamically adjusted to ensure that the correlation and influence between nodes are more accurately reflected in different situations, thereby providing more accurate weight distribution in complex networks and further improving the reasoning ability of the model and the reliability of the answers. During the training process of the generative question-answering model, a generative loss function based on soft cosine similarity is used, which can more finely capture the semantic similarity between texts, especially for long texts and inconsistent word orders. The particle swarm optimization algorithm with multi-state dynamic inertia weights calculated based on particle swarm diversity and particle swarm fitness is combined for hyperparameter optimization. It can adaptively adjust according to the search state at different optimization stages, balance global search and local development, and improve the efficiency of hyperparameter tuning and the semantic consistency and accuracy of model generation results.

[0120] Reference Manual Figure 3 , which shows a structural diagram of a power industry knowledge question-answering system based on a large language model provided by the present invention.

[0121] The present invention further provides a large language model-based electric power industry knowledge question-answering system 20, which is applied to the above-mentioned large language model-based electric power industry knowledge question-answering method, comprising:

[0122] Processor 201.

[0123] The memory 202 stores computer-readable instructions. When the computer-readable instructions are executed by the processor 201 , the power industry knowledge question-answering method based on a large language model as described in the method embodiment is implemented.

[0124] The electric power industry knowledge question and answer system 20 based on a large language model provided by the present invention can execute the above-mentioned electric power industry knowledge question and answer method based on a large language model and achieve the same or similar technical effects. To avoid repetition, the present invention will not go into details.

[0125] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0126] In the present invention, by connecting the knowledge graph with node weights to the scaled dot product attention layer of the generative question-answering model, and then using the autoregressive decoder to output the answer to the question, the interference caused by the training process is compensated, the weight relationship between different knowledge nodes is captured more accurately, and the attention to key knowledge points is improved. On the basis of the autoregressive decoder, the model refers to the weight information of the knowledge graph in each step of the generation process to ensure that the generated answers are more logical, accurate and contextually consistent. Even in the case of limited context windows and multi-round conversations, the consistency, logical rationality and accuracy of the answers can still be guaranteed based on the knowledge graph. Combined with the node weights determined by the PageRank algorithm with an adaptive damping coefficient, the importance of the nodes in the graph can be dynamically adjusted to ensure that the correlation and influence between nodes are more accurately reflected in different situations, thereby providing more accurate weight distribution in complex networks and further improving the reasoning ability of the model and the reliability of the answers. During the training process of the generative question-answering model, a generative loss function based on soft cosine similarity is used, which can more finely capture the semantic similarity between texts, especially for long texts and inconsistent word orders. The particle swarm optimization algorithm with multi-state dynamic inertia weights calculated based on particle swarm diversity and particle swarm fitness is combined for hyperparameter optimization. It can adaptively adjust according to the search state at different optimization stages, balance global search and local development, and improve the efficiency of hyperparameter tuning and the semantic consistency and accuracy of model generation results.

[0127] It should be understood that the processor in the embodiments of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0128] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus RAM (DR RAM).

[0129] The above embodiments can be implemented in whole or in part through software, hardware (such as circuits), firmware, or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer program are loaded or executed on a computer, the processes or functions described in accordance with the embodiments of the present invention are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired method (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains a collection of one or more available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, or magnetic tape), an optical medium (such as a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state drive.

[0130] It should be understood that the term "and / or" as used herein simply describes a relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A alone, A and B together, or B alone. A and B can be singular or plural. Furthermore, the character " / " as used herein generally indicates an "or" relationship between the associated objects, but it may also indicate an "and / or" relationship. For specific understanding, please refer to the context.

[0131] In this disclosure, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, "at least one of a, b, or c" can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or plural.

[0132] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0133] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0134] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0135] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interface, indirect coupling or communication connection of the device or unit, which can be electrical, mechanical or other forms.

[0136] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0137] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0138] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0139] An embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the method for answering questions about the knowledge of the electric power industry based on a large language model as described in the method embodiment is implemented.

[0140] The computer-readable storage medium provided by the present invention can implement the steps and effects of the power industry knowledge question and answer method based on a large language model in the above-mentioned method embodiment. To avoid repetition, the present invention will not go into details.

[0141] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0142] In the present invention, by connecting the knowledge graph with node weights to the scaled dot product attention layer of the generative question-answering model, and then using the autoregressive decoder to output the answer to the question, the interference caused by the training process is compensated, the weight relationship between different knowledge nodes is captured more accurately, and the attention to key knowledge points is improved. On the basis of the autoregressive decoder, the model refers to the weight information of the knowledge graph in each step of the generation process to ensure that the generated answers are more logical, accurate and contextually consistent. Even in the case of limited context windows and multi-round conversations, the consistency, logical rationality and accuracy of the answers can still be guaranteed based on the knowledge graph. Combined with the node weights determined by the PageRank algorithm with an adaptive damping coefficient, the importance of the nodes in the graph can be dynamically adjusted to ensure that the correlation and influence between nodes are more accurately reflected in different situations, thereby providing more accurate weight distribution in complex networks and further improving the reasoning ability of the model and the reliability of the answers. During the training process of the generative question-answering model, a generative loss function based on soft cosine similarity is used, which can more finely capture the semantic similarity between texts, especially for long texts and inconsistent word orders. The particle swarm optimization algorithm with multi-state dynamic inertia weights calculated based on particle swarm diversity and particle swarm fitness is combined for hyperparameter optimization. It can adaptively adjust according to the search state at different optimization stages, balance global search and local development, and improve the efficiency of hyperparameter tuning and the semantic consistency and accuracy of model generation results.

[0143] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

[0144] There are a few points to note:

[0145] (1) The drawings of the embodiments of the present invention only relate to the structures related to the embodiments of the present invention. Other structures may refer to conventional designs.

[0146] (2) For the sake of clarity, the thickness of layers or regions in the drawings used to describe the embodiments of the present invention are exaggerated or reduced, that is, these drawings are not drawn to scale. It is understood that when an element such as a layer, film, region, or substrate is referred to as being "on" or "under" another element, the element may be "directly" "on" or "under" the other element or intervening elements may be present.

[0147] (3) In the absence of conflict, the embodiments of the present invention and the features therein may be combined with each other to form new embodiments.

[0148] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. The protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A knowledge question answering method for the power industry based on a large language model, characterized by: include: S1: Obtain power industry data; S2: Based on the power industry data, a knowledge graph with node weights is constructed in combination with a PageRank algorithm with an adaptive damping coefficient, wherein the knowledge graph includes nodes representing various power entities and edges representing relationships between various power entities; S3: Obtain a training set with power industry questions as features and power industry answers as labels; S4: constructing a generative question answering model with knowledge graph embedded information in combination with the knowledge graph, wherein the generative question answering model includes an encoder, a scaled dot product attention layer, and an autoregressive decoder connected in sequence, wherein the knowledge graph is connected to the scaled dot product attention layer through a graph embedding layer; S5: Establish a generation loss function based on soft cosine similarity to evaluate the generation loss of the answers generated by the generative question answering model in the power industry; S6: In combination with a particle swarm optimization algorithm with a multi-state dynamic inertia weight, with the goal of generating a loss function value less than a preset generation loss function value, the generative question answering model is trained using the training set to determine the optimal hyperparameters of the generative question answering model, wherein the multi-state dynamic inertia weight is an inertia weight calculated based on particle swarm diversity and particle swarm fitness; S7: Obtain pending power industry issues; S8: Inputting the pending power industry question into the trained generative question answering model, and outputting an expected power industry answer corresponding to the pending power industry question; Wherein, the S2 specifically includes: S201: extracting basic data from the power industry data, wherein the basic data includes the power entity, power entity attributes, the power entity relationship, and the power entity relationship attributes; S202: Connect each node according to the basic data to construct a primary knowledge graph: G=(V,E) v i =(e i ,A i ),v i ∈V Among them, G represents the primary knowledge graph consisting of the node set V and the edge set E, v i The power entity is e i The power entity attribute is A i The i-th node, e ij The power entity relationship between the i-th node and the j-th node is r ij , the power entity relationship attribute between the i-th node and the j-th node is edge; S203: Initialize the PageRank value of each node, where the PageRank value is the inverse of the total number of nodes: Among them, PR(v i ) represents the i-th node v i The PageRank value of , N represents the total number of nodes; S204: Update the PageRank value of each node according to the number of adjacent nodes and out-degree of each node: Among them, PR′(v i ) represents node v i The updated PageRank value, d represents the adaptive damping coefficient representing the probability of random jump to other nodes in each calculation, O(v i ) represents node v i The incoming neighbor set of node v i The adjacent node set, PR(v j ) represents node v j The PageRank value of N represents the total number of nodes. and They represent the average out-degree and average in-degree of nodes in the primary knowledge graph, H and H max They represent the actual information entropy of the primary knowledge graph and the maximum information entropy under the completely uniform distribution of node access, log represents the logarithmic function, P(ν i ) represent the node ν i The probability of being visited before the update is the PageRank value; S205: Adding the updated PageRank value as the weight of the node to the primary knowledge graph to obtain a knowledge graph with node weights; The generation loss function is specifically: Among them, L oss represents the generated loss function, represents the total probability of obtaining the answer generated by the power industry, log represents the logarithmic function, and Y represents the label, i.e., the true answer of the power industry. Indicates the power industry corresponding to the label generates the answer, Indicates that Y and The soft cosine similarity between p represents the pth word in Y, express The qth word in Y, S(p,q) represents p and The cosine similarity between them, S(p,p) and S(q,q) both represent the cosine similarity of themselves with a value of 1; Wherein, the S4 specifically includes: Inputting the features of the training set, i.e., the power industry problem, into the encoder to obtain a feature hidden state; Converting the knowledge graph into an embedding representation of a numerical vector through the graph embedding layer; Combined with the node PageRank value, the attention scores of the feature hidden state and the embedded representation are calculated respectively through the scaled dot product attention layer: Among them, α i,j represents the attention score between the i-th hidden state and the j-th knowledge graph node, q i represents the corresponding query vector obtained by transforming the hidden state of the i-th word, exp represents the natural exponential function, k j and k k They represent the corresponding key vectors obtained by converting the embedding representation of the jth node and the embedding representation of the kth node, respectively. k represents k k The key vector dimension is , PageRank(j) represents the PageRank value of node j, and T represents transposition; Combined with the attention score, the autoregressive decoder is used to sequentially generate answer words based on the feature hidden state and the embedded representation in a cascade manner, wherein each answer word constitutes a generated label, i.e., the power industry generates an answer, thereby completing the construction of the generative question-answering model.

2. The electric power industry knowledge question answering method based on a large language model according to claim 1 is characterized in that: The generative question answering model further includes an input layer and an output layer, wherein the input layer is connected to the encoder and the output layer is connected to the autoregressive decoder.

3. The electric power industry knowledge question answering method based on a large language model according to claim 1 is characterized in that: The optimal hyperparameters include the optimal learning rate, optimal batch size, optimal number of encoder layers, optimal number of decoder layers, and optimal Dropout rate of the generative question answering model.

4. The electric power industry knowledge question answering method based on a large language model according to claim 1 is characterized in that: The S6 specifically includes: S601: Input the features of the training set, i.e., the power industry problem, into the encoder to obtain the feature hidden state: h i =Encoder(x i ,θ enc ) Among them, h i Represents the i-th word x in the input power industry problem i The hidden state of Encoder represents the Transformer encoder, θ enc Represents the Transformer encoder parameter set; S602: Convert the knowledge graph into an embedding representation of a numerical vector through the graph embedding layer: z v GAT(h v ,h N(v) ,A,θ gat ) Among them, z v represents the embedding representation of v nodes in the knowledge graph, GAT represents the graph embedding layer representing the graph attention network, and h N(v) Represents the neighbor node feature representation of node v in the graph embedding layer, h v represents the v-node feature representation in the graph embedding layer, A represents the adjacency matrix of the knowledge graph, and θ gat Represents the graph embedding layer parameters representing the weight matrices of each layer of the graph embedding layer; S603: Calculate the attention scores of the feature hidden state and the embedded representation respectively through the scaled dot product attention layer in combination with the node PageRank value: Among them, α i,j represents the attention score between the i-th hidden state and the j-th knowledge graph node, q i represents the corresponding query vector obtained by transforming the hidden state of the i-th word, exp represents the natural exponential function, k j and k k They represent the corresponding key vectors obtained by converting the embedding representation of the jth node and the embedding representation of the kth node, respectively. k represents k k The key vector dimension is , PageRank(j) represents the PageRank value of node j, and T represents transposition; S604: Combined with the attention score, the autoregressive decoder is used to sequentially generate answer words based on the feature hidden state and the embedding representation in a cascade manner, wherein each answer word constitutes a generated label, i.e., a generated answer for the power industry: y t =Decoder(y t-1 ,h,z,a i,j ,i dec ) Among them, y t and y t-1 Denote the answer words generated by the autoregressive decoder at the t-th time step and the t-1-th time step, respectively. Decoder represents the autoregressive decoder, h represents the hidden state of the input power industry question, and z represents the embedded representation of the input power industry question. Indicates the power industry generates the answer, y i represents the answer word generated at the i-th time step, i = 1, 2, …, T′, T′ represents the total number of answer words generated; S605: Taking the generation loss function value being less than the preset generation loss function value as the optimization target, determining the multi-state dynamic inertia weight based on the particle swarm diversity and the particle swarm fitness change rate, adjusting the hyperparameters of the generative question-answering model through the particle swarm optimization algorithm of the multi-state dynamic inertia weight, and outputting the optimal hyperparameters.

5. The electric power industry knowledge question answering method based on a large language model according to claim 4 is characterized in that: The S605 specifically includes: S6051: Determine the generated loss function value as the objective function; S6052: Initializing hyperparameters of different values ​​as particles to determine particle positions and particle velocities, wherein the particle positions are parameter vectors composed of the hyperparameters. S6053: Use the multi-state dynamic inertia weights determined based on the Fermat principle to determine the particle position update formula and particle velocity update formula: x i (t+1)=x i (t)+v i (t+1) Among them, x i (t) and x i (t+1) represents the particle position at time t and the particle position at time t+1, v i (t) and v i (t+1) represents the particle velocity at time t and the particle velocity at time t+1, w(t) represents the multi-state dynamic inertia weight at time t that controls the inertia of particle movement, c1 and c2 both represent learning factors with a value of 2, l1 and l2 respectively represent the first random number and the second random number with a value range of 0 to 1, represents the best historical position of particle i, g best represents the global best position, i.e., the best historical position found by all particles. d(t) represents the average distance between particles at time t, which represents the diversity of the particle swarm. min(d) = 10 -6 represents the minimum diversity of the particle swarm, U represents the total number of particles, represents the center position of the particle swarm at time t, ΔFitness(t) represents the fitness difference between the particle swarm at time t and time t-1 in different iterative processes, and ΔFitness max Indicates the maximum value of the particle swarm fitness change during the historical iteration process; S6054: updating the particles using the particle position update formula and the particle velocity update formula respectively, and calculating the update objective function value of the updated particles under the training set; S6055: Determine whether the difference between the updated objective function value and the objective function value before updating is less than a preset difference. If so, proceed to step S6056; otherwise, return to step S6054. S6056: Output the updated particle position as the optimal particle position, and output the hyperparameter corresponding to the optimal particle position as the optimal hyperparameter, ending the hyperparameter adjustment process.

6. The electric power industry knowledge question answering method based on a large language model according to claim 1 is characterized in that: After S8, the method further includes: The power industry data is updated at preset time intervals, and the knowledge graph and the generative question-answering model are updated based on the updated power industry data.

7. A knowledge question-answering system for the power industry based on a large language model, characterized by: include: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method for answering questions about the knowledge of the electric power industry based on a large language model as described in any one of claims 1 to 6 is implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the power industry knowledge question-answering method based on a large language model as described in any one of claims 1 to 6 is implemented.

Images