Power field knowledge question and answer optimization system based on large model retrieval enhancement generation and instruction supervision fine tuning

Abstract

An electric power field knowledge question-answer optimization system based on large model retrieval enhancement generation and instruction supervision fine tuning comprises a knowledge base construction module, a retrieval enhancement generation module, an instruction supervision fine tuning module and a knowledge fusion reasoning module. Core technologies such as multi-source heterogeneous knowledge base construction, multi-level retrieval architecture design, low-rank parameter efficient fine tuning and knowledge fusion reasoning are innovatively and organically combined, and high precision, high reliability and high real-time performance of knowledge questions and answers in the electric power professional field are achieved. The method is not only suitable for an intelligent question-answering system in the electric power field, but also can be popularized and applied to knowledge service systems in other professional technical fields, and has important theoretical value and wide application prospects. According to the invention, the knowledge coverage and timeliness of the model are improved; according to the method, an efficient and accurate knowledge retrieval and fusion mechanism is realized, and the accuracy and reliability of answers are improved; the training cost is reduced, and the effect is improved; according to the invention, the specialty and credibility of the model are enhanced.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an electric power field knowledge question-answering optimization system based on large model retrieval enhancement generation and instruction supervision fine-tuning. Background Art

[0002] With the rapid development of artificial intelligence technology, large language models (LLMs) have shown great potential in the field of knowledge question answering. Especially in applications in professional fields such as electricity, large language models can initially understand and answer professional questions through the pre-training-fine-tuning paradigm. However, existing technologies still face many challenges in practical applications, which are specifically manifested in the following aspects:

[0003] 1) Insufficient professional knowledge coverage

[0004] The existing large language models have obvious blind spots in knowledge coverage in the field of electric power. For example, the pre-training data mainly comes from public corpus on the Internet, and the coverage of electric power professional knowledge is low, especially the lack of grasp of key information such as equipment parameters, technical specifications, and operating procedures.

[0005] 2) Lack of real-time and timeliness of knowledge

[0006] There are significant deficiencies in knowledge updating and maintenance. For example, model knowledge is solidified in parameters, making it difficult to adapt to the rapid selection of new technologies, new equipment, and new specifications in the power industry. Specifically, it is unable to obtain and integrate the latest technical standards and specifications in a timely manner, the understanding of new equipment and technologies is seriously lagging behind, and it is difficult to reflect the latest industry development dynamics and trends.

[0007] 3) Lack of accuracy and reliability of answers

[0008] Existing models have serious accuracy issues in professional question-answering. For example, the problem of knowledge hallucination is prominent, which manifests itself in: generating professional parameters that do not conform to the facts, fabricating non-existent regulatory requirements, and confusing the technical characteristics of different devices.

[0009] 4) Insufficient scenario adaptability and knowledge relevance

[0010] It exhibits obvious limitations in actual application scenarios, for example: insufficient understanding of the specific professional context in the power sector, specifically: inability to accurately understand industry abbreviations and professional abbreviations, inaccurate understanding of the meaning of terms in specific scenarios, and difficulty in processing implicit professional background information.

[0011] 5) Inefficient model training and optimization

[0012] There are obvious shortcomings in model optimization. For example, traditional fine-tuning methods consume a lot of computing resources, which is specifically manifested in: requiring a lot of GPU resource support, high training time cost, and high parameter storage overhead. Summary of the invention

[0013] The purpose of the present invention is to provide a knowledge question-answering optimization system in the electric power field based on large model retrieval enhancement generation and instruction supervision fine-tuning, including: a knowledge base construction module, a retrieval enhancement generation module, an instruction supervision fine-tuning module, and a knowledge fusion reasoning module.

[0014] The knowledge base construction module is used to establish a multi-source heterogeneous knowledge base containing professional knowledge in the power field.

[0015] The multi-source heterogeneous knowledge base includes a document-level knowledge base, a paragraph-level knowledge base, and an entity-level knowledge base.

[0016] The retrieval enhancement generation module uses a large model of knowledge retrieval in the electric power field to perform multi-level information retrieval on a multi-source heterogeneous knowledge base based on a user query question, thereby generating an initial answer.

[0017] The instruction supervision fine-tuning module adjusts the parameters of the large model of power field knowledge retrieval through instruction supervision.

[0018] The knowledge fusion reasoning module integrates the user query question and the initial answer output by the retrieval enhancement generation module through multi-hop reasoning and cross-modal knowledge fusion technology to generate an explanatory answer.

[0019] Furthermore, the steps of the knowledge base construction module establishing a multi-source heterogeneous knowledge base containing professional knowledge in the power field include:

[0020] a11 obtains the original power domain knowledge dataset D.

[0021] a12 preprocesses the original power field knowledge dataset D to obtain the normalized knowledge dataset D′, as shown below:

[0022]

[0023] In the formula, i is the knowledge data serial number, X is the total number of knowledge data, and D i is the i-th knowledge data. To standardize the knowledge space.

[0024] The preprocessing includes data cleaning and normalization.

[0025] a13 normalizes the knowledge data D in the knowledge dataset D′ i Convert to structured data D struct , as shown below:

[0026] D struct ={(e i ,r i ,a i )||i∈{1,…,X}} (2)

[0027] In the formula, e i Represents an entity, r i is the relationship type, a i is the attribute value.

[0028] a14 builds a hierarchical knowledge storage system K, as shown below:

[0029]

[0030] In the formula, K d is the document-level knowledge base, K p is the paragraph-level knowledge base, K e It is an entity-level knowledge base.

[0031] Among them, the index structure of the hierarchical knowledge storage system K As shown below:

[0032]

[0033] Where M is the maximum number of neighbors per layer, efConstruction is the size of the candidate set during construction, HNSW is the indexing algorithm, l is the level number, K1 is the knowledge base at level l, and d, p, and e represent the document level, paragraph level, and entity level, respectively.

[0034] a15 uses natural language processing technology to extract struct Extract key knowledge units and store them hierarchically into the hierarchical knowledge storage system K, as shown below:

[0035] K d ={(d i ,m i ,t i )||i∈{1,…,N d}} (5)

[0036] K p ={(p i ,c i ,q i )||i∈{1,…,N p}} (6)

[0037] K e ={(e i ,r i ,a i )||i∈{1,…,Ne}} (7)

[0038] Where, d i is the document content, m i is metadata, t i is the timestamp. i For paragraph content, c i is the context information, q i is the relevance score. N d is the total number of document-level knowledge data. p is the total number of paragraph-level knowledge data. N e is the total amount of entity-level knowledge data.

[0039] Furthermore, the knowledge base construction module is also used to update the multi-source heterogeneous knowledge base in real time, and the steps are as follows:

[0040] a21 computing new knowledge dataset Importance score As shown below:

[0041]

[0042] In the formula, k is the knowledge data, ω k is the knowledge item weight coefficient, and ∑ k ω k =1.

[0043] Among them, the knowledge quality scoring function Q(k) is as follows:

[0044] Q(k)=α·Accuracy(k)+β·Timeliness(k)+γ·Completeness(k)(9)

[0045] Where α, β, γ are weight coefficients, and α+β+γ=1. Accuracy(k) is the accuracy score of knowledge data k, Timeliness(k) is the timeliness score of knowledge data k, and Completeness(k) is the attribute completeness of knowledge data k.

[0046] a22 The knowledge base K at time step t t Perform the update operation to obtain the knowledge base K at time step t+1 t+1 , as shown below:

[0047]

[0048] In the formula, θ t Get the threshold for the knowledge at time step t.

[0049] Among them, the update function Update is as follows:

[0050]

[0051] In the formula, Score(k) is the importance score of knowledge data k.

[0052] a23 updates the threshold parameter by gradient descent method as follows:

[0053]

[0054] In the formula, θ t+1 is the knowledge acquisition threshold at time step t+1. η is the learning rate, is the loss function, is the gradient function.

[0055] Furthermore, the knowledge base construction module is also used to clean up the remaining data, the steps are as follows:

[0056] a31 calculates the similarity between knowledge data pairs as follows:

[0057] Similarity(k i1 ,k i2 )=cos(E(k i1 ),E(k i2 ))+λ·JaccardSim(k i1 ,k i2 )(13)

[0058] Where E(·) is the knowledge encoding function and JaccardSim is the Jaccard similarity. Similarity(k i1 ,k i2 ) is the knowledge data pair (k i1 ,k i2 ). i1 , k i2 are all knowledge data. λ is the weight.

[0059] a32 merges the knowledge data whose similarity exceeds the threshold and constructs the merged knowledge data k merged , as shown below:

[0060]

[0061] In the formula, i is the knowledge data serial number, n is the total number of knowledge data to be merged, is the knowledge data set to be merged. Q(k) is the knowledge quality scoring function. k is the knowledge data, and Merge is the merging function.

[0062] a33 builds the knowledge base after cleaning the remaining data As shown below:

[0063]

[0064] In the formula, k′ is the knowledge data. Similarity is the similarity function, θ sim is the similarity threshold. K is the hierarchical knowledge storage system.

[0065] Furthermore, the search enhancement generation module uses the large model of knowledge retrieval in the electric power field to perform multi-level information retrieval on multi-source heterogeneous knowledge bases based on the user query question, so as to generate the initial answer in the following steps:

[0066] b1 builds a document-level retrieval layer and uses it to search the input query q in the document-level knowledge base to obtain the document-level retrieval result R d .

[0067] The document-level retrieval result R d As shown below:

[0068]

[0069] In the formula, R d is the document-level retrieval result, i is the knowledge data serial number, doc i is the i-th related document. is the standardized knowledge space. d The document-level retrieval threshold.

[0070] Among them, the search score score (doc i ) is as follows:

[0071] score(doc i )=λ s BM25(q,doc i )+λ d ·cos(E q (q),E d (doc i ))(17)

[0072] In the formula, λ s , d are the weight coefficients of sparse retrieval and dense retrieval respectively, and λ s +λ d =1. q (q), E d (doc i ) are query encoder and document encoder respectively. BM25(q,doc i ) is the document-level retrieval relevance score.

[0073] b2 builds a paragraph-level retrieval layer and introduces a paragraph context enhancement mechanism.

[0074] The calculation formula of the paragraph context enhancement mechanism is as follows:

[0075]

[0076] In the formula, j is the paragraph number, p j-1 、p j 、p j+1 Represent the word representations of the j-1th, jth, and j+1th paragraphs respectively. Represents text concatenation. BERT is the text concatenation function. PE(j) is the positional encoding. Enhance results for paragraph context.

[0077] b3 Use the paragraph-level retrieval layer to search in the document-level retrieval results to obtain the paragraph-level retrieval result R p .

[0078] The paragraph-level retrieval result R p As shown below:

[0079] R p =TopK({p j |p j ∈Split(doc i ),doc i ∈R d},K' p ) (18)

[0080] In the formula, R p K' is the paragraph-level retrieval result. p is the number of paragraphs to be retained. TopK is the search function. Split is the segmentation function.

[0081] b4 builds an entity-level retrieval layer and introduces a multi-head cross-attention mechanism.

[0082] The calculation formula of the multi-head cross attention mechanism is as follows:

[0083] MultiHead(Q,K,V)=Concat(head1,…,head h )W O (19)

[0084] Where MultiHead(Q,K,V) is the enhanced result of the multi-head cross attention mechanism. o is the output projection matrix. Concat is the fusion concatenation function. h is the total number of attention heads. Q, K, and V represent the query matrix, key matrix, and value matrix, respectively.

[0085] Among them, the attention head i As shown below:

[0086]

[0087] In the formula, head u is the u-th attention head, Both are projection matrices.

[0088] Among them, the cross attention mechanism Attention(Q,K,V) is as follows:

[0089]

[0090] In the formula, softmax is the normalization function, d k is the dimension of the key vector. K T is the transposed matrix of the key-value matrix.

[0091] b5 Use the entity-level retrieval layer to extract information from the paragraph-level retrieval results to obtain the entity-level retrieval result R e .

[0092] b6 dynamically fuses the retrieval results of each level through residual connection and layer normalization, as shown below:

[0093] R final =LayerNorm(∑ l∈{d,p,e} w l R l +FFN(∑ l∈{d,p,e} w l R l ))(twenty two)

[0094] In the formula, R final is the fusion retrieval result. l is the level number, d, p, and e are document level, paragraph level, and entity level, respectively. l is the l-th level search result. LayerNorm is the layer normalization function. l is the fusion weight. FFN is a two-layer feedforward network.

[0095] Furthermore, the step of adjusting the parameters of the large power field knowledge retrieval model by the instruction supervision fine-tuning module through instruction supervision includes:

[0096] c1 builds the instruction tag set.

[0097] c2 adds instruction labels to the knowledge data and constructs a training set.

[0098] c3 takes instruction labels and knowledge data as input, and the knowledge data corresponding to the instruction labels as output. It uses the training set to train the large model to obtain a large model with directionality.

[0099] The large model selects a BERT model or a GPT model.

[0100] Furthermore, the step of using the training set to train the large model to obtain a large model with directionality includes:

[0101] c31 performs a structured decomposition of the projection matrix of the large model and introduces adaptive updates as follows:

[0102]

[0103] In the formula, is the projection matrix of the large model. W is the basic projection matrix. ΔW is the low-rank structure. B and A are both low-rank matrices, α is the adaptive coefficient, and Adapt is the input-related adaptive adjustment item.

[0104] c32 builds an optimization model, and the objective function of the optimization model is as follows:

[0105]

[0106] In the formula, is the objective function of the optimization model. is the main task performance function. λ1, λ2, and λ3 are all trade-off coefficients. ∥.∥ F is the norm.

[0107] c33 uses the optimization model to update the low-rank matrix B and the low-rank matrix A to obtain a large model with optimized parameters.

[0108] c34 performs quantitative training on the large model after parameter optimization to obtain a large model with directionality.

[0109] Furthermore, the step of performing quantization training on the large model after parameter optimization includes:

[0110] c341 performs b-bit quantization on the weights of the large model after parameter optimization, as shown below:

[0111]

[0112] In the formula, w q is the quantized weight. b is a constant, w is the weight of the large model after parameter optimization. clip is the truncation function. round is the rounding function.

[0113] The mean μ and standard deviation σ of the weights are as follows:

[0114]

[0115] In the formula, g i is the weight sequence number. g represents the weight grouping, and |g| is the grouping size. For g i Weights. g , σ g are the mean and standard deviation of the weights of group g respectively.

[0116] c342 quantifies the loss function of the large model after parameter optimization, and dynamically adjusts the weight of the large model based on the loss function.

[0117] The loss function of the large model after parameter optimization is as follows:

[0118]

[0119] Where β is the quantization loss weight and dequant(·) is the dequantization operation. This is the loss function of the large model after parameter optimization. is the task loss function.

[0120] c343 calculates the accuracy of the large model after parameter optimization and dynamically adjusts the weight of the large model based on the accuracy.

[0121] The accuracy of the large model after parameter optimization is as follows:

[0122]

[0123] In the formula, l is the level number, p l is the accuracy of the lth layer. high and θ low Toggle threshold for accuracy.

[0124] Among them, the layer sensitivity (l) is as follows:

[0125]

[0126] In the formula, O l is the output of the lth layer. l is the weight of the lth layer. Represents the gradient of the loss function with respect to the network weights. Loss represents the total loss function.

[0127] c344 builds an adaptive adjustment system and uses it to adjust the weights of large models.

[0128] The learning rate of the adaptive adjustment system is as follows:

[0129]

[0130] Where η t is the adjusted learning rate, η0 is the original learning rate. β1 and β2 are momentum parameters. is the initial gradient norm.

[0131] The adaptive adjustment system also introduces Warm-up and Cosine annealing methods, as shown below:

[0132]

[0133] Where η max is the learning rate extreme value, t warmup is the number of warm-up steps, t total is the total number of training steps. k is the training step.

[0134] Furthermore, the knowledge fusion reasoning module integrates the user query question and the initial answer output by the retrieval enhancement generation module through multi-hop reasoning to generate an explanatory answer, including the following steps:

[0135] d11 uses a path search algorithm to find knowledge paths related to user query questions in multi-source heterogeneous knowledge bases.

[0136] d12 calculates the probability of each reasoning path and determines the optimal direction of the reasoning path.

[0137] The probabilities of the reasoning paths are as follows:

[0138]

[0139] In the formula, i v is the inference node number, n v is the total number of inference nodes. v ) is the path inference probability. Represents the reasoning path. To assess the reliability of the relationship. Indicates that in the reasoning path, the i v nodes and the i-th v +1 Relationship between nodes. Indicates the reasoning path To the reasoning path The relevant knowledge subgraph.

[0140] Among them, the relationship transfer probability As shown below:

[0141]

[0142] In the formula, j vis the inference node number. Softmax is the normalization function. is the relation mask matrix, E pos Encodes relative position. Indicates the jth v +1 The transpose of the key vector of the node. k is the dimension of the key vector. Indicates the jth v The query vector of nodes.

[0143] d13 generates explanatory answers based on the optimal direction of the reasoning path, using the logical connections between path nodes and integrating the knowledge data on different path nodes.

[0144] Furthermore, the knowledge fusion reasoning module uses cross-modal knowledge fusion technology to reason and integrate the user query question and the initial answer output by the retrieval enhancement generation module to generate an explanatory answer, including the following steps:

[0145] d21 builds a multimodal feature extraction model.

[0146] d22 extracts features from knowledge data through a multimodal feature extraction model, and uses feature alignment technology to map features of different modalities into the same semantic space.

[0147] The knowledge data includes text data, image data and structured data.

[0148] The features of different modalities include semantic vectors, visual features, and numerical vectors.

[0149] The feature alignment technique is as follows:

[0150]

[0151] In the formula, f aligned is the feature after being mapped to the same semantic space. OT is the optimal transmission algorithm. t is the text feature. g is the graph feature. C is the feature distance metric. For the i t Text features With the jth g Spectral features The distance measure between .

[0152] d23 performs weighted fusion of features from different modalities through a gating mechanism to construct a fused semantic space.

[0153] The calculation formula for weighted fusion of features of different modalities through the gating mechanism is as follows:

[0154] Z=σg (W g [f t ;f g ]+b g ),·f out =Z⊙f fusion +(1-Z)⊙f aligned (35)

[0156] In the formula, f out is the weighted fusion feature. Z is the gate vector. σ g is the sigmoid function. g is the weight matrix. g is the bias vector.

[0157] Among them, the cross-modal feature fusion network f fusion As shown below:

[0158] f fusion =LayerNorm(MLP([f t ;f g ])+λ f Cross-Attention(f t ,f g )) (36)

[0159] Where LayerNorm is the layer normalization function. MLP is the multi-layer perceptron. Cross-Attention is the correlation function. f is the scaling parameter.

[0160] d24 generates explanatory answers in the fused semantic space through associative reasoning and information supplementation.

[0161] The technical effect of the present invention is unquestionable. The present invention proposes a large-model knowledge question-answering optimization system in the power field based on the Retrieval-Augmented Generation (RAG) paradigm and instruction-tuning, which focuses on solving the key technical problems faced by large language models when applied in professional fields such as electricity, such as insufficient knowledge coverage, poor timeliness, low accuracy, poor scenario adaptability, and weak knowledge relevance.

[0162] The present invention innovatively combines core technologies such as multi-source heterogeneous knowledge base construction, multi-level retrieval architecture design, efficient low-rank parameter fine-tuning, and knowledge fusion reasoning to achieve high precision, high reliability, and high real-time performance in knowledge question and answering in the power professional field. This method is not only applicable to intelligent question and answer systems in the power field, but can also be extended to knowledge service systems in other professional and technical fields, and has important theoretical value and broad application prospects.

[0163] The present invention constructs a comprehensive and real-time updated electric power professional knowledge system to improve the knowledge coverage and timeliness of the model; the present invention realizes an efficient and accurate knowledge retrieval and fusion mechanism to improve the accuracy and reliability of the answers; the present invention designs a model optimization scheme with low resource consumption to reduce training costs and improve the effect; the present invention establishes an explainable knowledge reasoning system to enhance the professionalism and credibility of the model.

[0164] The present invention has high adaptability and professional guidance capabilities in different application scenarios. Whether it is equipment fault diagnosis, operation and maintenance, safe operation, or preventive maintenance, the system of the present invention can provide comprehensive, accurate, and explanatory answers according to specific task requirements, thereby providing an efficient solution to the knowledge question and answer needs in the power field.

[0165] Combined with instruction supervision and low-rank parameter optimization technology, the present invention not only improves the domain knowledge adaptability of the model, but also significantly reduces the consumption of computing resources, providing technical support for large-scale power knowledge question and answer applications. The fine-tuned model is more adaptable to the professional needs of the power field, and can accurately understand and answer professional questions related to power equipment, fault diagnosis, operation and maintenance, etc. Its answer content not only has the terminology standardization of the field, but also can reflect the logical structure that meets industry standards. Low-rank parameter optimization and quantization training significantly reduce the computational burden of the model during the reasoning process, so that the model can still run efficiently under the condition of limited hardware resources. Compared with the unoptimized model, the fine-tuned model has been greatly improved in terms of generation speed and response time, meeting the performance requirements in practical applications. The introduction of instruction supervision makes the model more directional when answering, ensuring that the output content is highly matched with the query requirements. Whether it is the grammatical structure of the answer content or the use of professional terms, it reflects a high consistency and accuracy, thereby improving the user experience.

[0166] The beneficial effects of the present invention include: 1) In terms of knowledge acquisition and organization, a method for constructing a professional power knowledge base based on multi-source heterogeneity is proposed; 2) In terms of knowledge retrieval technology, a multi-level retrieval architecture and context enhancement technology are innovatively designed; 3) In terms of model optimization, a low-rank parameter efficient fine-tuning scheme based on LoRA and QLoRA is proposed; 4) In terms of knowledge reasoning, a multi-hop reasoning and explainability analysis system based on knowledge graph is realized. BRIEF DESCRIPTION OF THE DRAWINGS

[0167] Figure 1 This is the architecture diagram of the knowledge question answering system in the power field based on large model retrieval enhancement generation and instruction supervision fine-tuning;

[0168] Figure 2 It is the overall architecture diagram of the system of the present invention;

[0169] Figure 3 Construct a flow chart for a multi-source heterogeneous knowledge base;

[0170] Figure 4 This is a schematic diagram of a multi-level retrieval architecture;

[0171] Figure 5 This is a schematic diagram of instruction supervision fine-tuning and parameter optimization;

[0172] Figure 6 This is a diagram of knowledge fusion reasoning and interpretability analysis. DETAILED DESCRIPTION

[0173] The present invention is further described below in conjunction with the embodiments, but it should not be understood that the above subject matter of the present invention is limited to the following embodiments. Without departing from the above technical ideas of the present invention, various substitutions and changes are made according to the common technical knowledge and customary means in the art, which should all be included in the protection scope of the present invention.

[0174] Embodiment 1:

[0175] See also Figures 1 to 6 , a knowledge question answering optimization system in the power field based on large model retrieval enhancement generation and instruction supervision fine-tuning, including: knowledge base construction module, retrieval enhancement generation module, instruction supervision fine-tuning module, knowledge fusion reasoning module.

[0176] The knowledge base construction module is used to establish a multi-source heterogeneous knowledge base containing professional knowledge in the power field.

[0177] The professional knowledge in the power field includes professional documents such as substation operation and maintenance procedures, power grid dispatching operation specifications, and equipment maintenance standards.

[0178] The multi-source heterogeneous knowledge base includes a document-level knowledge base, a paragraph-level knowledge base, and an entity-level knowledge base.

[0179] The document-level knowledge base contains the content of the paragraph-level knowledge base, and the paragraph-level knowledge base contains the content of the entity-level knowledge base.

[0180] The retrieval enhancement generation module uses a large model of knowledge retrieval in the electric power field to perform multi-level information retrieval on a multi-source heterogeneous knowledge base based on a user query question, thereby generating an initial answer.

[0181] The instruction supervision fine-tuning module adjusts the parameters of the large model of power field knowledge retrieval through instruction supervision.

[0182] The knowledge fusion reasoning module integrates the user query question and the initial answer output by the retrieval enhancement generation module through multi-hop reasoning and cross-modal knowledge fusion technology to generate an explanatory answer.

[0183] Embodiment 2:

[0184] The power field knowledge question answering optimization system based on large model retrieval enhancement generation and instruction supervision fine-tuning, the main technical content is shown in Example 1, further, the steps of the knowledge base construction module to establish a multi-source heterogeneous knowledge base containing professional knowledge in the power field include:

[0185] a11 obtains the original power domain knowledge dataset D.

[0186] a12 preprocesses the original power field knowledge dataset D to obtain the normalized knowledge dataset D′, as shown below:

[0187]

[0188] In the formula, i is the knowledge data serial number, X is the total number of knowledge data, and D i is the i-th knowledge data. To standardize the knowledge space.

[0189] The preprocessing includes data cleaning and normalization.

[0190] a13 normalizes the knowledge data D in the knowledge dataset D′ i Convert to structured data D struct , as shown below:

[0191] D struct ={(e i ,r i ,a i )||i∈{1,…,X}} (2)

[0192] In the formula, e i Represents an entity, r i is the relationship type, a i is the attribute value.

[0193] a14 builds a hierarchical knowledge storage system K, as shown below:

[0194]

[0195] In the formula, K d is the document-level knowledge base, K pis the paragraph-level knowledge base, K e It is an entity-level knowledge base.

[0196] Among them, the index structure of the hierarchical knowledge storage system K As shown below:

[0197]

[0198] Where M is the maximum number of neighbors per layer, efConstruction is the size of the candidate set during construction, HNSW is the indexing algorithm, l is the level number, K l is the lth level knowledge base. d, p, e represent document level, paragraph level, and entity level respectively.

[0199] a15 uses natural language processing technology to extract struct Extract key knowledge units and store them hierarchically into the hierarchical knowledge storage system K, as shown below:

[0200] K d ={(d i ,m i ,t i )||i∈{1,…,N d}} (5)

[0201] K p ={(p i ,c i ,q i )||i∈{1,…,N p}} (6)

[0202] K e ={(e i ,r i ,a i )||i∈{1,…,N e}} (7)

[0203] Where, d i is the document content, m i is metadata, t i is the timestamp. i For paragraph content, c i is the context information, q i is the relevance score. N d is the total number of document-level knowledge data. p is the total number of paragraph-level knowledge data. N e is the total amount of entity-level knowledge data.

[0204] Embodiment 3:

[0205] The power field knowledge question answering optimization system based on large model retrieval enhanced generation and instruction supervision fine-tuning, the main technical content is shown in any one of embodiments 1 to 2, further, the knowledge base construction module is also used to update the multi-source heterogeneous knowledge base in real time, the steps are as follows:

[0206] a21 computing new knowledge dataset Importance score As shown below:

[0207]

[0208] In the formula, k is the knowledge data, ω k is the knowledge item weight coefficient, and ∑ k ω k =1.

[0209] Among them, the knowledge quality scoring function Q(k) is as follows:

[0210] Q(k)=α·Accuracy(k)+β·Timeliness(k)+γ·Completeness(k) (9)

[0212] Where α, β, γ are weight coefficients, and α+β+γ=1. Accuracy(k) is the accuracy score of knowledge data k, Timeliness(k) is the timeliness score of knowledge data k, and Completeness(k) is the attribute completeness of knowledge data k.

[0213] a22 The knowledge base K at time step t t Perform the update operation to obtain the knowledge base K at time step t+1 t+1 , as shown below:

[0214]

[0215] In the formula, θ t Get the threshold for the knowledge at time step t.

[0216] Among them, the update function Update is as follows:

[0217]

[0218] In the formula, Score(k) is the importance score of knowledge data k.

[0219] a23 updates the threshold parameter by gradient descent method as follows:

[0220]

[0221] In the formula, θt+1 is the knowledge acquisition threshold at time step t+1. η is the learning rate, is the loss function, is the gradient function.

[0222] Embodiment 4:

[0223] The power field knowledge question answering optimization system based on large model retrieval enhanced generation and instruction supervision fine-tuning, the main technical content is shown in any one of embodiments 1 to 3, further, the knowledge base construction module is also used to clean up the remaining data, the steps are as follows:

[0224] a31 calculates the similarity between knowledge data pairs as follows:

[0225] Similarity(k ii ,k i2 )=cos(E(k i1 ),E(k i2 ))+λ·JaccardSim(k ii ,k i2 )(13)

[0226] Where E(·) is the knowledge encoding function and JaccardSim is the Jaccard similarity. Similarity(k i1 ,k i2 ) is the knowledge data pair (k i1 ,k i2 ). i1 , k i2 are all knowledge data. λ is the weight.

[0227] a32 merges the knowledge data whose similarity exceeds the threshold and constructs the merged knowledge data k merged , as shown below:

[0228]

[0229] In the formula, i is the knowledge data serial number, n is the total number of knowledge data to be merged, is the knowledge data set to be merged. Q(k) is the knowledge quality scoring function. k is the knowledge data, and Merge is the merging function.

[0230] a33 builds the knowledge base after cleaning the remaining data As shown below:

[0231]

[0232] In the formula, k′ is the knowledge data. Similarity is the similarity function, θ simis the similarity threshold. K is the hierarchical knowledge storage system.

[0233] Embodiment 5:

[0234] The power field knowledge question answering optimization system based on large model retrieval enhancement generation and instruction supervision fine-tuning, the main technical content is shown in any one of embodiments 1 to 4, further, the retrieval enhancement generation module uses the power field knowledge retrieval large model to perform multi-level information retrieval on multi-source heterogeneous knowledge bases based on user query questions, so as to generate the initial answer in the following steps:

[0235] b1 builds a document-level retrieval layer and uses it to search the input query q in the document-level knowledge base to obtain the document-level retrieval result R d .

[0236] The document-level retrieval result R d As shown below:

[0237]

[0238] In the formula, R d is the document-level retrieval result, i is the knowledge data serial number, doc i is the i-th related document. is the standardized knowledge space. d The document-level retrieval threshold.

[0239] Among them, the search score score (doc i ) is as follows:

[0240] score(doc i )=λ s BM25(q,doc i )+λ d ·cos(E q (q),E d (doc i ))(17)

[0241] In the formula, λ s , d are the weight coefficients of sparse retrieval and dense retrieval respectively, and λ s +λ d =1. q (q), E d (doc i ) are query encoder and document encoder respectively. BM25(q,doc i ) is the document-level retrieval relevance score.

[0242] b2 builds a paragraph-level retrieval layer and introduces a paragraph context enhancement mechanism.

[0243] The calculation formula of the paragraph context enhancement mechanism is as follows:

[0244]

[0245] In the formula, j is the paragraph number, p j-1 、p j 、p j+1 Represent the word representations of the j-1th, jth, and j+1th paragraphs respectively. Represents text concatenation. BERT is the text concatenation function. PE(j) is the positional encoding. Enhance results for paragraph context.

[0246] b3 Use the paragraph-level retrieval layer to search in the document-level retrieval results to obtain the paragraph-level retrieval result R p .

[0247] The paragraph-level retrieval result R p As shown below:

[0248] R p =TopK({p j |p j ∈Split(doc i ),doc i ∈R d},K' p ) (18)

[0249] In the formula, R p K' is the paragraph-level retrieval result. p is the number of paragraphs to be retained. TopK is a search function that searches for the first few most relevant fragments. Split is a segmentation function.

[0250] b4 builds an entity-level retrieval layer and introduces a multi-head cross-attention mechanism.

[0251] The calculation formula of the multi-head cross attention mechanism is as follows:

[0252] MultiHead(Q,K,V)=Concat(head1,…,head h )W O (19)

[0253] Where MultiHead(Q,K,V) is the enhanced result of the multi-head cross attention mechanism. O is the output projection matrix. Concat is the fusion concatenation function. h is the total number of attention heads. Q, K, and V represent the query matrix (Query), the key matrix (Key), and the value matrix (Value), respectively.

[0254] Among them, the attention head u As shown below:

[0255]

[0256] In the formula, head u is the u-th attention head, Both are projection matrices.

[0257] Among them, the cross attention mechanism Attention(Q,K,V) is as follows:

[0258]

[0259] In the formula, softmax is the normalization function, d k K is the dimension of the key vector, used for scaling the attention scores. T is the transposed matrix of the key-value matrix.

[0260] b5 Use the entity-level retrieval layer to extract information from the paragraph-level retrieval results to obtain the entity-level retrieval result R e .

[0261] b6 dynamically fuses the retrieval results of each level through residual connection and layer normalization, as shown below:

[0262] R final =LayerNorm(∑ l∈{d,p,e} w l R l +FFN(∑ l∈{d,p,e} w l R l ))(twenty two)

[0263] In the formula, R fimal is the fusion retrieval result. l is the level number, d, p, and e are document level, paragraph level, and entity level, respectively. l is the l-th level retrieval result. LayerNorm is the layer normalization function, which is used to standardize the features and introduce learnable scaling and offset parameters to improve the stability and effect of model training. l is the fusion weight. FFN is a two-layer feedforward network.

[0264] Embodiment 6:

[0265] The power field knowledge question answering optimization system based on large model retrieval enhancement generation and instruction supervision fine-tuning, the main technical content is shown in any one of embodiments 1 to 5, further, the instruction supervision fine-tuning module adjusts the parameters of the power field knowledge retrieval large model through instruction supervision, including:

[0266] c1 builds the instruction tag set.

[0267] c2 adds instruction labels to the knowledge data and constructs a training set.

[0268] c3 takes instruction labels and knowledge data as input, and the knowledge data corresponding to the instruction labels as output. It uses the training set to train the large model to obtain a large model with directionality.

[0269] The large model selects a BERT model or a GPT model.

[0270] Embodiment 7:

[0271] The power field knowledge question answering optimization system based on large model retrieval enhancement generation and instruction supervision fine-tuning, the main technical content of which is shown in any one of embodiments 1 to 6, further, the step of using the training set to train the large model to obtain a large model with directionality includes:

[0272] c31 performs a structured decomposition of the projection matrix of the large model and introduces adaptive updates as follows:

[0273]

[0274] In the formula, is the projection matrix of the large model. W is the basic projection matrix. ΔW is the low-rank structure. B and A are both low-rank matrices, α is the adaptive coefficient, and Adapt is the input-related adaptive adjustment item.

[0275] c32 builds an optimization model, and the objective function of the optimization model is as follows:

[0276]

[0277] In the formula, is the objective function of the optimization model. is the main task performance function. λ1, λ2, and λ3 are all trade-off coefficients. ∥.∥ F is the norm.

[0278] c33 uses the optimization model to update the low-rank matrix B and the low-rank matrix A to obtain a large model with optimized parameters.

[0279] c34 performs quantitative training on the large model after parameter optimization to obtain a large model with directionality.

[0280] Embodiment 8:

[0281] The power field knowledge question answering optimization system based on large model retrieval enhancement generation and instruction supervision fine-tuning, the main technical content of which is shown in any one of embodiments 1 to 7, further, the step of performing quantization training on the large model after parameter optimization includes:

[0282] c341 performs b-bit quantization on the weights of the large model after parameter optimization, as shown below:

[0283]

[0284] In the formula, w q is the quantized weight. b is a constant, and w is the weight of the large model after parameter optimization. clip is a truncation function, which is used to limit the value within a given range and map the out-of-range value to the range boundary. round is a rounding function.

[0285] The mean μ and standard deviation σ of the weights are as follows:

[0286]

[0287] In the formula, g i is the weight sequence number. g represents the weight grouping, and |g| is the grouping size. For the g i Weights. g , σ g are the mean and standard deviation of the weights of group g respectively.

[0288] c342 quantifies the loss function of the large model after parameter optimization, and dynamically adjusts the weight of the large model based on the loss function.

[0289] The loss function of the large model after parameter optimization is as follows:

[0290]

[0291] Where β is the quantization loss weight and dequant(·) is the dequantization operation. This is the loss function of the large model after parameter optimization. is the task loss function. In the quantization-aware training process, two goals need to be considered simultaneously:

[0292] 1) Task performance: We hope that the quantized model can still maintain good performance on the original task.

[0293] It is the loss function used to measure the performance of the model on the original task. For example, if the task is image classification, It can be a cross entropy loss function; if the task is machine translation, It can be a negative log-likelihood loss function.

[0294] 2) Quantization loss: We hope that the quantized model parameters are as close to the original model parameters as possible to avoid the accuracy loss caused by quantization. Part of it is used to measure the quantitative loss.

[0295] The summary is: It is the loss function of the model’s original task, which is used to measure the performance of the model on the original task and ensure that the quantized model is still effective on the main task.

[0296] c343 calculates the accuracy of the large model after parameter optimization and dynamically adjusts the weight of the large model based on the accuracy.

[0297] The accuracy of the large model after parameter optimization is as follows:

[0298]

[0299] In the formula, l is the level number, p l is the accuracy of the lth layer. high and θ low Toggle threshold for accuracy.

[0300] Among them, the layer sensitivity (l) is as follows:

[0301]

[0302] In the formula, O l is the output of the lth layer. l is the weight of the lth layer. Represents the gradient of the loss function with respect to the network weights. Loss represents the total loss function.

[0303] c344 builds an adaptive adjustment system and uses it to adjust the weights of large models.

[0304] The learning rate of the adaptive adjustment system is as follows:

[0305]

[0306] Where η t is the adjusted learning rate, η0 is the original learning rate. β1 and β2 are momentum parameters. is the initial gradient norm.

[0307] The adaptive adjustment system also introduces Warm-up and Cosine annealing methods, as shown below:

[0308]

[0309] Where η max is the learning rate extreme value, t warmup is the number of warm-up steps, t total is the total number of training steps. k is the training step.

[0310] Embodiment 9:

[0311] The power field knowledge question answering optimization system based on large model retrieval enhancement generation and instruction supervision fine-tuning, the main technical content of which is shown in any one of embodiments 1 to 8. Further, the knowledge fusion reasoning module integrates the user query question and the initial answer output by the retrieval enhancement generation module through multi-hop reasoning, and the step of generating an explanatory answer includes:

[0312] d11 uses a path search algorithm to find knowledge paths related to user query questions in multi-source heterogeneous knowledge bases.

[0313] d12 calculates the probability of each reasoning path and determines the optimal direction of the reasoning path.

[0314] The probabilities of the reasoning paths are as follows:

[0315]

[0316] In the formula, i v is the inference node number, n v is the total number of inference nodes. v ) is the path inference probability. Represents the reasoning path. To assess the reliability of the relationship. Indicates that in the reasoning path, the i v nodes and the i-th v +1 relationship between nodes. In the reasoning path of a multi-hop knowledge graph, multiple nodes are connected through different relationships to form a reasoning link. It represents the specific relationship between two adjacent nodes in this link. Indicates the reasoning path To the reasoning path In multi-hop reasoning, not all information in the knowledge graph is relevant to the current reasoning task. Represents the current reasoning step (that is, arrive ) related knowledge subgraph. It can be a subset containing related entities and relations. It can narrow the search space and improve the efficiency and accuracy of reasoning.

[0317] Among them, the relationship transfer probability As shown below:

[0318]

[0319] In the formula, j v is the inference node number. Softmax is the normalization function. is the relation mask matrix, E pos Encodes relative position. Indicates the jth v +1 Transpose of the key vector of the node. k is the dimension of the key vector, used to scale the dot product result. Indicates the jth v The query vector of nodes.

[0320] d13 generates explanatory answers based on the optimal direction of the reasoning path, using the logical connections between path nodes and integrating the knowledge data on different path nodes.

[0321] Embodiment 10:

[0322] The power field knowledge question answering optimization system based on large model retrieval enhancement generation and instruction supervision fine-tuning, the main technical content of which is shown in any one of embodiments 1 to 9, further, the knowledge fusion reasoning module reasoning and integrating the user query question and the initial answer output by the retrieval enhancement generation module through cross-modal knowledge fusion technology, and the step of generating an explanatory answer includes:

[0323] d21 builds a multimodal feature extraction model.

[0324] d22 extracts features from knowledge data through a multimodal feature extraction model, and uses feature alignment technology to map features of different modalities into the same semantic space.

[0325] The knowledge data includes text data, image data and structured data.

[0326] The features of different modalities include semantic vectors, visual features, and numerical vectors.

[0327] The feature alignment technique is as follows:

[0328]

[0329] In the formula, f aligned is the feature after being mapped to the same semantic space. OT is the optimal transmission algorithm. t is the text feature. g is the graph feature. C is the feature distance metric. For the i t Text features With the jth g Spectral features The distance measure between .

[0330] d23 performs weighted fusion of features from different modalities through a gating mechanism to construct a fused semantic space.

[0331] The calculation formula for weighted fusion of features of different modalities through the gating mechanism is as follows:

[0332] Z=σ g (W g [f t ;f g ]+b g ),f out =Z⊙f fusion +(1-Z)⊙f aligned (35)

[0334] In the formula, f out is the weighted fusion feature. Z is the gate vector, which controls the fusion ratio of different modal features. The value of Z is between 0 and 1, which is determined by the Sigmoid function σ g Activation is obtained. g is a sigmoid function, which maps the result of the linear transformation to between 0 and 1 through the sigmoid function to obtain the gate vector Z. g is the weight matrix. g is the bias vector.

[0335] Among them, the cross-modal feature fusion network f fusion As shown below:

[0336] f fusion =LayerNorm(MLP([f t ;f g ])+λ f Cross-Attention(f t ,f g )) (36)

[0337] In the formula, LayerNorm is the layer normalization function, which is a commonly used normalization technique that can stabilize the training process and improve model performance. MLP is a multi-layer perceptron, which is a fully connected neural network used to perform nonlinear transformations on input features. Here, the input of MLP is the text feature vector f t and the graph feature vector f g Cross-Attention is the correlation function, which is a cross attention mechanism. This is an attention mechanism that calculates the correlation between text features and graph features. It allows the model to focus on relevant parts of the graph based on text information and vice versa. In this formula, it uses the text feature f t As the query vector, the graph feature f gAs the key vector and value vector, the attention weight of the text feature on the graph feature is calculated, and the graph feature is weighted by this, and finally a context vector is obtained, which reflects the attention of the text information to the graph information. f is a learnable scaling parameter that controls the contribution of the cross-attention mechanism.

[0338] d24 generates explanatory answers in the fused semantic space through associative reasoning and information supplementation.

[0339] Embodiment 11:

[0340] See also Figures 1 to 6 , a knowledge question-answering optimization system in the power field based on large model retrieval enhancement generation and instruction supervision fine-tuning, the main technical contents include:

[0341] (1) Constructing a multi-source heterogeneous power professional knowledge base, which specifically includes the following steps:

[0342] 1.1) Knowledge acquisition and preprocessing: Based on the original power field knowledge dataset D, data cleaning, standardization and structured processing are performed to obtain a standardized knowledge dataset D′=[D1,D2,…,D x ]; where X is the total number of knowledge samples, D x represents the Xth knowledge sample; for each knowledge sample D i , construct its vector representation:

[0343]

[0344] Where Encoder(·) is the pre-trained Transformer encoder, and d is the vector dimension;

[0345] 1.2) Knowledge structured modeling: Construct the knowledge graph G = (V, E, R) in the power field, where V represents the entity set, E represents the edge set, and R represents the relationship type set; for any entity pair (v i ,v j )∈V×V, through the relation extraction function f rel Calculate the probability distribution of their relationship:

[0346] P(r ij v i ,v j )=softmax(f rel (h i ,h j )),r ij ∈R#(2)

[0347] Among them, h i ,h jis the hidden layer representation of the entity, obtained through the graph neural network:

[0348]

[0349] in, represents the neighbor set of node i, α ij is the attention weight:

[0350]

[0351] 1.3) Multi-level knowledge organization: Building a hierarchical knowledge storage system K, including a document-level knowledge base K d , paragraph-level knowledge base K p and entity-level knowledge base K e , satisfying the inclusion relation:

[0352]

[0353] For each level of knowledge, establish an index structure I l :

[0354]

[0355] Among them, HNSW is a hierarchical navigable small-world graph indexing algorithm, M is the maximum number of neighbors per layer, and efConstructior is the size of the candidate set during construction;

[0356] (2) Implementing a search enhancement generation module, specifically including the following steps:

[0357] 2.1) Construct a multi-level retrieval architecture: For the input query q, perform parallel retrieval in each level of knowledge base:

[0358]

[0359] Among them, E q (·) is the query encoder, k is the number of retrievals; the retrieval similarity is calculated using the dual-tower model:

[0360]

[0361] Among them, sparse(q,k i ) is the sparse matching score:

[0362]

[0363] 2.2) Implementing context enhancement technology: Using a multi-head cross-attention mechanism to fuse query q and retrieval result R:

[0364] MultiHead(Q,K,V)=Concat(head1,…,headh )W O #(10)

[0365] Each attention head is calculated as follows:

[0366]

[0367] Introducing relative position encoding B ij Enhanced context awareness:

[0368]

[0369] 2.3) Design a fusion retrieval strategy: Dynamically integrate multi-level retrieval results through an adaptive attention network:

[0370] w l =AdaptiveAttention(h l ,c),l∈{d,p,e}#(14)

[0371] Among them, h l is the hierarchical feature, c is the context vector, and the weight is calculated as follows:

[0372]

[0373] The final fusion result is obtained through residual connection and layer normalization:

[0374]

[0375] (3) Execute instruction supervision and fine-tuning, which specifically includes the following steps:

[0376] 3.1) Construct a low-rank adapter (LoRA) structure to pre-train the model weight matrix Perform structured decomposition:

[0377]

[0378] Among them, r<<min(d,k) is the rank parameter; the optimization goal is:

[0379]

[0380] Initialize the low-rank matrix by SVD decomposition:

[0381]

[0382] 3.2) Introduce adapter layer normalization and process each layer input x:

[0383]

[0384] Among them, μ, σ are the mean and standard deviation, γ, β are learnable parameters;

[0385] 3.3) Design residual adapter structure:

[0386] h=x+Dropout(W2GeLU(W1LayerNorm(x)))#(21)

[0387] in is the adapter parameter, m is the bottleneck dimension;

[0388] 3.4) Construct a multi-view data enhancement pipeline to train the samples (x i ,y i ) applies a sequence of transformations:

[0389]

[0390] Among them, the transformation set Include:

[0391]

[0392] 3.5) Constructing contrastive learning objectives based on InfoNCE loss:

[0393]

[0394] where s(·,·) is the similarity function, is the temperature parameter;

[0395] 3.6) Implement online hard example mining strategy:

[0396]

[0397] in is the loss value of sample i;

[0398] 3.7) Use mixed precision training strategy to set different calculation precisions for different layers:

[0399]

[0400] 3.8) Construct a multi-objective loss function:

[0401]

[0402] The definitions of the losses are as follows:

[0403]

[0404] 3.9) Implement gradient clipping and accumulation:

[0405]

[0406] (4) Perform knowledge fusion reasoning, which specifically includes the following steps:

[0407] 4.1) Construct a multi-hop reasoning path. For a given query path p = (v1, ..., v n ), and calculate the path probability through the attention mechanism:

[0408]

[0409] The single-step transition probability is calculated by relation-aware attention:

[0410]

[0411] in is the relation mask matrix;

[0412] 4.2) Implement path scoring and sorting:

[0413] score(p)=αP(p)+βCoverage(p)+γDiversity(p)#(34)

[0414] 4.3) Construct a cross-modal attention network to integrate text features f t and knowledge graph feature f g

[0415] f fusion =LayerNorm(MLP([f t ;f g ])+λCross-Attention(f t ,f g ))#(35)

[0416] 4.4) Implement feature alignment and calibration:

[0417]

[0418] Among them, OT is the optimal transmission algorithm;

[0419] 4.5) Introduce a gating mechanism to control information flow:

[0420] g=σ(W g [f t ;f g ]),f ouit =g⊙f fusuon +(1-g)⊙f aligned #(37)

[0421] 4.6) Generate multi-granularity reasoning process explanation:

[0422]

[0423] 4.7) Realize attention visualization and attribution:

[0424]

[0425] 4.8) Establish an evidence chain tracking mechanism:

[0426]

[0427] where s i As the source of evidence, i is the inference rule, c i For confidence.

[0428] Embodiment 12:

[0429] The power field knowledge question answering optimization system based on large model retrieval enhanced generation and instruction supervision fine-tuning, the main technical content is shown in Example 11, and further, the step of constructing the multi-source heterogeneous power professional knowledge base also includes:

[0430] 1) Implementing a real-time knowledge update mechanism, which includes incremental knowledge acquisition and dynamic update strategy. Incremental knowledge acquisition is implemented by the following steps:

[0431] 1.1) New knowledge Perform quality assessment and calculate its importance score:

[0432]

[0433] Among them, Q(k) is the knowledge quality scoring function, ω k is the weight coefficient of the knowledge item, satisfying ∑ k ω k =1

[0434] 1.2) Establish a knowledge increment update strategy. For the knowledge base at time step t To perform an update operation:

[0435]

[0436] The update operation includes:

[0437]

[0438] In the formula, θ acquire is the knowledge acquisition threshold, which is used to control the quality standard of new knowledge;

[0439] 1.3) Implement dynamic update optimization algorithm and update model parameters through gradient descent method:

[0440]

[0441] Where η is the learning rate, is the loss function used to evaluate the effect of knowledge updating;

[0442] 2) Construct a knowledge quality assessment system, which includes accuracy assessment, timeliness assessment and completeness assessment.

[0443] The specific implementation steps are as follows:

[0444] 2.1) Implement the accuracy assessment module to score the accuracy of knowledge item k:

[0445]

[0446] in, is the reference knowledge set, Verify(k,r) is the verification function, and the return value is in the interval [0,1];

[0447] 2.2) Establish a timeliness evaluation mechanism to calculate the timeliness score of knowledge

[0448] Timeliness(k)=exp(-λ(t current -t update (k)))#(46)

[0449] Where λ is the time-dependent attenuation coefficient, t current is the current timestamp, t update (k) is the last update time of the knowledge item;

[0450] 2.3) Perform completeness assessment to evaluate the completeness of the attributes of the knowledge item:

[0451]

[0452] Among them, Required is the required attribute set, Attributes(k) is the actual attribute set of knowledge item k;

[0453] 2.4) Calculation of comprehensive evaluation score:

[0454] Q(k)=α·Accuracy(k)+β·Timeliness(k)+γ·Completeness(k)#(48)

[0455] In the formula, α, β, γ are weight coefficients, and they satisfy α+β+γ=1

[0456] 3) Execute the remaining data cleaning process, which includes two main steps: duplicate identification and merging optimization:

[0457] 3.1) Implement a duplicate identification mechanism and calculate the similarity between pairs of knowledge items:

[0458] Similarity(k u ,k j )=cos(E(k i ),E(k j ))+λ·JaccardSim(k i ,k j )#(49)

[0459] Among them, E(·) is the knowledge encoding function, and JaccardSim is the Jaccard similarity:

[0460] 3.2) Establish a knowledge merging strategy to merge knowledge items whose similarity exceeds the threshold:

[0461]

[0462] In the formula, is the set of knowledge items to be merged;

[0463] 3.3) Build the cleaned knowledge base:

[0464]

[0465] Among them, θ sim is the similarity threshold, which is used to control the strictness of knowledge deduplication.

[0466] Embodiment 13:

[0467] The power field knowledge question answering optimization system based on large model retrieval enhancement generation and instruction supervision fine-tuning, the main technical content is shown in any one of embodiments 11 to 12, further, the multi-level retrieval architecture of the retrieval enhancement generation module also includes the following steps:

[0468] 3.1) Build a document-level retrieval layer for filtering a wide range of relevant documents, including:

[0469]

[0470] The retrieval score is calculated by a weighted combination of sparse representation and dense representation:

[0471] score(doc i )=λ s BM25(q,doc i )+λ d ·cos(E q (q),E d (doci ))#(53)

[0472] In the formula, θ d is the document-level retrieval threshold, λ s and λ d are the weight coefficients of sparse retrieval and dense retrieval respectively and satisfy λ s +λ d =1,E q and E d for query and document encoders respectively; the BM25 score is calculated as follows:

[0473]

[0474] Where k1 and b are the hyperparameters of the BM25 algorithm, |d| is the document length, and avgdl is the average document length; 3.2) Implement a paragraph-level retrieval layer for accurate paragraph positioning, the retrieval layer includes:

[0475] R p =TopK({p j |p j ∈Split(doc i ),doc i ∈R d},K p )#(55)

[0476] The paragraph similarity is calculated through a fine-grained interaction matrix:

[0477]

[0478] In the formula, q i and p j Represent the word representations of query and paragraph respectively, K p The number of paragraphs to be retained; at the same time, a paragraph context enhancement mechanism is introduced:

[0479]

[0480] in represents text concatenation, PE(j) is the positional encoding;

[0481] 3.3) Design a sentence-level retrieval layer for specific information extraction, which is achieved through the cross-attention mechanism

[0482]

[0483] And based on the multi-head mechanism, the feature extraction capability is enhanced:

[0484] MultiHead(Q,K,V)=Concat(head1,…,head h)W o #(59)

[0485] The calculation of each attention head is as follows:

[0486]

[0487] In the formula, is the learnable projection matrix, is the output projection matrix;

[0488] 3.4) Construct an inter-layer dynamic fusion mechanism to optimize retrieval results, including feature interaction and adaptive weighting:

[0489] R final =DynamicFusion(R d ,R p ,R s )#(61)

[0490] The fusion weights are dynamically calculated through the gating network:

[0491] w l =Gate(h l ,c)=σ(W g [h l ;c]+b g )#(62)

[0492] In the formula, h l is the feature of each level, c is the global context vector, W g and b q is the gating network parameter; the final fusion result is obtained through residual connection and layer normalization:

[0493]

[0494] Among them, FFN is a two-layer feedforward network for feature conversion and nonlinear mapping:

[0495] FFN(x)=max(0,xW1+b1)W2+b2#(64)

[0496] The multi-level retrieval architecture realizes a hierarchical retrieval process from coarse-grained to fine-grained through the synergistic effect of the above four modules, ensuring the accuracy and completeness of the retrieval results.

[0497] Embodiment 14:

[0498] The power field knowledge question answering optimization system based on large model retrieval enhancement generation and instruction supervision fine-tuning, the main technical content is shown in any one of embodiments 11 to 13, further, the instruction supervision fine-tuning also includes the following steps

[0499] 4.1) Implement low-rank parameter optimization technology based on LoRA, including:

[0500] Pre-trained model weight matrix Perform structured decomposition and introduce adaptive updates:

[0501]

[0502] in is a low-rank matrix that satisfies the rank constraint r<<min(d,k), α is the adaptive coefficient, and Adapt(x) is the input-related adaptive adjustment term; the optimization objective function is designed as:

[0503]

[0504] Where λ1, λ2, λ3 are regularization coefficients used to balance the contribution of each loss;

[0505] 4.2) Implement quantization training technology based on QLoRA. The specific steps are:

[0506] Perform b-bit quantization on the model weights:

[0507]

[0508] Among them, μ and σ are the mean and standard deviation of the weights, respectively, calculated by group statistics:

[0509]

[0510] In the formula, g represents weight grouping, |g| is the grouping size; the quantization-aware training strategy is introduced:

[0511]

[0512] Where β is the quantization loss weight, dequant(·) is the dequantization operation;

[0513] 4.3) Build a mixed precision training framework, which includes the following components: Precision adaptive adjustment mechanism:

[0514]

[0515] Where Sensitivity(l) is the layer sensitivity evaluation function:

[0516]

[0517] Where O l is the layer output, θ high and θ low Switch threshold for accuracy;

[0518] 4.4) Implement dynamic optimization strategy adjustment mechanism, including: adaptive adjustment of learning rate:

[0519]

[0520] Where β1, β2 are momentum parameters, is the initial gradient norm;

[0521] Gradient clipping and accumulation strategy:

[0522]

[0523] Where K is the number of gradient accumulation steps, c is the clipping threshold; Warmup and Cosine annealing strategies are introduced at the same time:

[0524]

[0525] where t warmup is the number of warm-up steps, t total is the total number of training steps.

[0526] The finger-supervised fine-tuning achieves efficient optimization of model parameters and dynamic adjustment of accuracy through the organic combination of the above four core technical modules, which significantly reduces the computing and storage overhead while ensuring the model performance.

[0527] Embodiment 15:

[0528] The power field knowledge question answering optimization system based on large model retrieval enhanced generation and instruction supervision fine-tuning, the main technical content is shown in any one of embodiments 11 to 14, further, the knowledge fusion reasoning also includes the following steps:

[0529] 5.1) Implement knowledge graph multi-hop reasoning technology, including: building a multi-level reasoning path search mechanism:

[0530]

[0531] Where V q To query the related entity set, V a is a set of candidate answer entities, and the path probability is calculated through multi-step transfer:

[0532]

[0533] Introducing attention-aware path selection mechanism:

[0534]

[0535] In the formula is the relation mask matrix, E pos For relative position encoding, the calculation is as follows:

[0536]

[0537] 5.2) Perform multimodal information collaborative processing, including: Building a cross-modal feature fusion network:

[0538] f fusion =LayerNorm(MLP([f t ;f g ])+λ·Cross-Attention(f t ,f g ))#(79)

[0539] where f t and f q They are text and graph features respectively, and the correlation between modalities is calculated through the attention mechanism:

[0540]

[0541] Achieve feature alignment and complementary enhancement:

[0542]

[0543] Where d(·,·) is the feature distance metric, and is the feature distribution, γ is the distribution difference weight;

[0544] Introducing dynamic gating mechanism:

[0545] g=σ(W g [f t ;f g ]+b g ),f out =g⊙f fusion +(1-g)⊙f aligned #(82)

[0546] 5.3) Construct an explainable reasoning link, including: generating multi-granularity reasoning process explanations:

[0547]

[0548] The reasoning process coherence score is:

[0549]

[0550] Implement gradient-based attention visualization:

[0551]

[0552] where y cScore the target category, is the activation value of the feature map;

[0553] 5.4) Establish an answer reliability assessment mechanism, including: Building a multi-dimensional evidence scoring system:

[0554]

[0555] The confidence calculation function is defined as:

[0556] f conf (s,r)=α·P source (s)+β·P rule (r)+γ·Consistency(s,r)#(87)

[0557] Where P source (s) is the reliability of the evidence source, P rule (r) is the reliability of the inference rule, and Consistency(s,r) is the consistency of evidence. The complementary verification mechanism of the evidence chain is introduced:

[0558]

[0559] Where E i represents the i-th evidence chain, and Support(·,·,·) is the complementary support of evidence.

[0560] Embodiment 16:

[0561] See also Figures 1 to 6 , a knowledge question-answering optimization system in the power field based on large model retrieval enhancement generation and instruction supervision fine-tuning, the main technical contents include:

[0562] 1.1 Construction of multi-source heterogeneous knowledge base: This module adopts a hierarchical knowledge acquisition and organization method, which mainly includes the following technical features:

[0563] 1.1.1) Knowledge acquisition and preprocessing technology: Based on the original power field knowledge data set D, perform data cleaning, standardization and structured processing to build a standardized knowledge data set:

[0564]

[0565] Among them, X is the total number of knowledge samples, is the standardized knowledge space. For each knowledge sample D i Vectorized representation:

[0566]

[0567] Where Encoder(·) is the pre-trained Transformer encoder and d is the vector dimension.

[0568] 1.1.2) Knowledge graph construction technology: Construct the knowledge graph G = (V, E, R) in the power field, where V represents the entity set, E represents the edge set, and R represents the relationship type set. For any entity pair (v i ,v j )∈V×V, and calculate its relationship probability distribution through the multi-head attention mechanism:

[0569]

[0570] Among them, Q i ,K j ,V j is the query, key, and value matrix of the entity, is the relation-specific attention mask, d k The dimension of attention.

[0571] 1.1.3) Knowledge hierarchical organization technology: Establish a three-level knowledge storage system Contains document-level, paragraph-level, and entity-level knowledge, satisfying the inclusion relationship:

[0572]

[0573] Building an efficient index structure Using the HNSW (HierarchicalNavigableSmallWorld) algorithm:

[0574]

[0575] Among them, M is the maximum number of neighbors per layer, and efConstruction is the size of the candidate set during construction.

[0576] 1.1.4) Real-time knowledge updating mechanism: Design an incremental updating strategy based on importance scoring to update new knowledge Calculate its importance score:

[0577]

[0578] Among them, Q(k) is the knowledge quality scoring function:

[0579] Q(k)=α·Accuracy(k)+β·Timeliness(k)+γ·Completeness(k)

[0580] In the formula, α, β, γ are weight coefficients and satisfy α+β+γ=1

[0581] 1.2 Multi-level retrieval architecture design: This module adopts a hierarchical retrieval strategy, which mainly includes the following technical features:

[0582] 1.2.1) Document-level retrieval technology: Implementing a hybrid retrieval strategy that combines sparse representation and dense representation

[0583] score(q,d)=λ s BM25(q,d)+λ d ·cos(E q (q),E d (d))

[0584] Among them, q is the query, d is the document, and λ s ,λ d is the weight coefficient, E q ,E d For query and document encoders. BM25 score calculation:

[0585]

[0586] 1.2.2) Paragraph-level retrieval technology: construct a fine-grained interaction matrix and calculate paragraph similarity:

[0587]

[0588] Introducing context enhancement mechanism:

[0589]

[0590] in, represents text concatenation, and PE(j) is the positional encoding.

[0591] 1.2.3) Fusion layer design: Implementing dynamic weighted attention mechanism:

[0592] w l =Gate(h l ,c)=σ(W g [h l ;c]+b g )

[0593] Among them, h l are the features of each level, and c is the global context vector.

[0594] The final fusion result is obtained through residual connection and layer normalization:

[0595]

[0596] 1.3 Instruction supervision fine-tuning optimization: In response to the specialized optimization needs of large language models in the power field, this paper proposes a systematic model optimization solution. This solution fully considers the balance between computing resource constraints and model performance, and significantly reduces computing overhead while ensuring the optimization effect. The specific technical solution includes the following key parts:

[0597] 1.3.1) Parameter optimization method based on improved low-rank adaptation: This paper deeply studies the problem of large model parameter optimization and proposes an improved low-rank adaptation method. The core idea of ​​this method is to reduce the parameter scale through structured decomposition and introduce a dynamic update mechanism to maintain the expressiveness of the model. Specifically: First, the weight matrix of the pre-trained model is Conduct innovative structural decomposition:

[0598]

[0599] In this decomposition framework: the basic weight matrix W remains unchanged to ensure the basic capabilities of the model, and the low-rank matrix and (where r<<min(d,k)) captures domain-specific knowledge, and the dynamic adaptation term Adapt(x) is responsible for handling input-related feature adjustments. In order to ensure the controllability and effectiveness of the optimization process, a multi-objective optimization function is designed:

[0600]

[0601] This optimization function cleverly balances multiple key factors: main task performance, parameter structure preservation, regularization constraints, and adaptability. By carefully adjusting the trade-off coefficients λ1, λ2, and λ3, the best optimization effect can be achieved in different scenarios.

[0602] 1.3.2) Quantization-aware precision training technology: To address the resource constraints in model deployment, this paper proposes a high-precision quantization training framework based on NF4. This framework not only considers the accuracy loss during the quantization process, but also pays special attention to maintaining the distribution characteristics of the model. The quantization process is performed in a grouped statistical manner:

[0603]

[0604] Among them, the calculation of group statistical parameters pays special attention to accuracy:

[0605]

[0606] To ensure that the quantization process does not affect model performance, a special quantization-aware training strategy is designed:

[0607]

[0608] The uniqueness of this training strategy lies in: strictly controlling the information loss caused by quantization by reconstructing the error term, introducing the KL divergence term to maintain the distribution characteristics of the model, and adopting a dynamic weight adjustment mechanism to ensure the stability of the training process.

[0609] 1.3.3) Adaptive precision dynamic control mechanism: Considering the differences in precision requirements of different layers of the model, the present invention creatively proposes a hierarchical adaptive precision control mechanism. This mechanism achieves the optimal allocation of precision resources by dynamically evaluating the importance of each layer:

[0610]

[0611] Layer sensitivity assessment adopts an innovative comprehensive evaluation method:

[0612]

[0613] The advantages of this evaluation method are as follows: it comprehensively considers gradient information, parameter norm and output influence, dynamically captures the mutual influence relationship between layers, and adaptively adjusts the evaluation weights to ensure the accuracy of the evaluation.

[0614] 1.3.4) Optimization strategy adaptive adjustment system

[0615] In order to deal with various uncertainties in the training process, the present invention designs a complete set of optimization strategy adaptive adjustment system, including: learning rate dynamic adjustment mechanism:

[0616]

[0617] The characteristics of this mechanism include: making full use of historical gradient information to guide current updates, ensuring training stability through adaptive step size, and introducing warm-up and cosine attenuation strategies to optimize the training process.

[0618] Gradient processing uses an innovative accumulation and clipping strategy:

[0619]

[0620] The advantages of this strategy are: effectively reducing the variance of gradient estimation; preventing gradient explosion and vanishing problems; and achieving selective updates through a dynamic mask mechanism.

[0621] 1.3.5) Multi-task collaborative learning optimization system: Based on the in-depth analysis of the knowledge system in the power field, this paper constructs an innovative multi-task collaborative learning framework. This framework breaks through the limitations of traditional single-task optimization and significantly improves the comprehensive performance of the model through knowledge transfer and feature sharing between tasks. The specific implementation includes:

[0622] Task representation learning mechanism:

[0623] h task =TaskEncoder(t)=MLP(Embedding(t)+Context(t))

[0624] The innovation of this mechanism lies in: integrating explicit features and contextual information of the task; enhancing representation capabilities through nonlinear mapping; achieving adaptive extraction of task features; and designing a special task adaptation layer to ensure effective collaboration between different tasks:

[0625] TaskAdapter(x,t)=LayerNorm(x+FFN(x⊙h task ))

[0626] The characteristics of this adaptation layer are: achieving task-specific adaptation through feature modulation; maintaining the core information of the original features; and introducing residual connections to ensure training stability.

[0627] The design of multi-task loss function is particularly critical:

[0628]

[0629] This loss function cleverly balances multiple objectives: through the dynamic weight w t Regulate the importance of each task; use the mutual information term MI(·,·) to promote knowledge sharing; introduce the diversity metric Diversity(·) to prevent feature convergence

[0630] 1.4 Knowledge fusion reasoning mechanism: Aiming at the special needs of professional knowledge question and answer in the power field, the present invention proposes a complete knowledge fusion reasoning mechanism. This mechanism achieves high-quality answer generation through multi-level knowledge integration and reasoning analysis. Specifically, it includes the following core technologies:

[0631] 1.4.1) Multi-hop knowledge reasoning engine: The present invention designs a multi-hop reasoning system based on knowledge graph, which can effectively handle complex reasoning links and path reasoning probability calculation:

[0632]

[0633] Where: p=(v1,…,v n ) represents the reasoning path; is the relationship transition probability; Reliability(r i,i+1 ) is the relationship reliability evaluation

[0634] To enhance the accuracy of reasoning, an innovative attention-aware mechanism is introduced:

[0635]

[0636] The advantages of this mechanism are: considering the semantic association between entities; integrating the constraint information specific to the relationship; introducing position encoding to enhance path perception

[0637] 1.4.2) Cross-modal knowledge fusion system: In view of the multi-modal characteristics of knowledge in the power field, the present invention constructs an efficient cross-modal fusion framework:

[0638] Feature fusion uses an improved attention network:

[0639] f fusion =LayerNorm(MLP([f t ;f g ])+λ·Cross-Attention(f t ,f g ,C))

[0640] Among them, the cross-modal attention calculation introduces conditional information C:

[0641]

[0642] To ensure the accuracy of feature alignment, an alignment mechanism based on optimal transmission is designed:

[0643]

[0644] Its characteristics are: comprehensive consideration of feature distance and distribution difference; ensuring global optimal alignment through optimal transmission; introducing KL divergence to maintain distribution characteristics

[0645] 1.4.3) Evidence chain reasoning and interpretability analysis system: Based on an in-depth study of the characteristics of professional knowledge in the power field, this paper constructs a complete set of evidence chain reasoning and interpretability analysis system. This system not only ensures the reliability of the reasoning process, but also realizes the clear display of the reasoning path. Specifically, it includes:

[0646] Reasoning process explains the generation mechanism:

[0647]

[0648] The reasoning coherence score adopts an innovative assessment method:

[0649]

[0650] The characteristics of this evaluation method are: considering the semantic coherence between adjacent evidence; introducing logical reasoning scores to ensure the rationality of reasoning; integrating domain knowledge constraints to ensure professionalism

[0651] 1.4.4) Dynamic evidence enhancement and verification mechanism: In order to improve the reliability of reasoning results, the present invention designs an innovative evidence enhancement and verification mechanism:

[0652] Evidence chain reliability assessment:

[0653]

[0654] The confidence calculation function is designed as:

[0655] Confidence(E)=α·P source (E)+β·P rule (E)+γ·Consistency(E)

[0656] In particular, the reliability of evidence sources is assessed using multi-dimensional analysis:

[0657]

[0658] The advantages of this mechanism are: improving reliability through multi-source cross-validation; considering the timeliness and authority of evidence; introducing consistency checks to ensure the coordination of evidence

[0659] 1.4.5) Attention attribution and visualization system: This invention builds a complete attention attribution analysis system and realizes the transparent display of the reasoning process:

[0660] Gradient-based attention visualization:

[0661]

[0662] The design of the scale adjustment function takes into account local and global information:

[0663]

[0664] To enhance interpretability, multi-level feature attribution is introduced:

[0665]

[0666] The innovations of this system include: realizing fine-grained attention analysis; supporting multi-level feature importance evaluation; and providing an intuitive visual display interface.

[0667] Embodiment 17:

[0668] See also Figures 1 to 6 , a knowledge question-answering optimization system in the power field based on large model retrieval enhancement generation and instruction supervision fine-tuning, the main technical contents include:

[0669] 1. System overall architecture

[0670] A knowledge question-answering optimization method for the power field based on large model retrieval enhancement generation and instruction supervision fine-tuning is proposed. Its system architecture consists of multiple core modules, including knowledge base construction module, retrieval enhancement generation module, instruction supervision fine-tuning module and knowledge fusion reasoning module. Each module is organically integrated through data flow and control flow to form a closed-loop knowledge question-answering optimization system.

[0671] The operating mechanism of this system can be summarized as follows: the knowledge base construction module collects, processes and structures the multi-source heterogeneous data from the power field, so as to establish a high-quality knowledge base containing professional knowledge in the power field; the retrieval enhancement generation module, driven by user input, adopts a multi-level retrieval strategy to comprehensively query the knowledge base, and extracts relevant information based on a combination of sparse and dense retrieval; the instruction supervision fine-tuning module improves the model's knowledge generation and semantic understanding capabilities in the power field by fine-tuning and optimizing the parameters of the large model, enabling it to respond more accurately to complex queries in the field; the knowledge fusion reasoning module integrates the retrieved multi-level knowledge, and combines contextual information and reasoning strategies to generate question-answer content with high accuracy and explanatory power.

[0672] Through the collaborative work of the above modules, the present invention can provide efficient, accurate and in-depth answers in the professional knowledge question-answering scenarios of the power industry, significantly enhancing the application effect and adaptability of traditional question-answering systems. The large model fine-tuning method of the present invention in the retrieval and generation process realizes the organic integration of the knowledge base and the generation module, significantly improving the system's responsiveness and reasoning depth in a complex knowledge environment, thereby meeting the increasingly complex knowledge needs in the power industry.

[0673] 2. Knowledge base building module

[0674] The knowledge base construction module is responsible for establishing a multi-source heterogeneous knowledge base containing professional knowledge in the power field. Through steps such as data collection, data preprocessing, knowledge extraction and knowledge organization, this module structurally integrates multi-source power professional knowledge to meet the needs of the knowledge question-answering system in the power field.

[0675] 2.1 Data acquisition and preprocessing

[0676] The knowledge base construction module first collects professional knowledge in the power field from multiple sources through the data acquisition unit, including but not limited to: technical specifications and standard documents of power equipment, historical fault cases and handling experience, real-time operation and maintenance data and operation logs, as well as expert knowledge base and related technical literature. Diverse data sources provide a solid knowledge foundation for the construction of the knowledge base.

[0677] After data collection is completed, the preprocessing unit performs a series of standardization processes on the raw data to ensure the consistency and quality of the data. Specifically, the following steps are included:

[0678] 1) Data cleaning: The preprocessing unit cleans the collected raw data to remove noise, redundant information and erroneous data, thereby improving data quality. The result set of data cleaning can be expressed as:

[0679] D clean ={x∈D raw ∣∣Q(x)>θ quality}

[0680] Among them, D raw is the original data set, Q(x) is the data quality assessment function, θ quality The quality threshold is preset, and only data items that meet the quality standard are retained.

[0681] 2) Text normalization: After data cleaning, the preprocessing unit performs text normalization on the data to ensure uniform terminology and format. This step includes unified encoding of text format, standardized conversion of professional terminology, and character standardization to make data more consistent during storage and retrieval.

[0682] 3) Structural processing: The preprocessed data is usually unstructured text and needs to be further converted into structured data for knowledge extraction. Specifically, unstructured data is decomposed into entity, relationship and attribute triples, expressed as:

[0683] D struct ={(e i ,r i ,a i )||i∈{1,…,N}}

[0684] Among them, e i Represents an entity, r i is the relationship type, a i This structuring process transforms complex text data into standardized knowledge representation, laying the foundation for subsequent knowledge extraction and organization.

[0685] Through data acquisition and preprocessing, the knowledge base construction module can ensure the high quality and consistency of the original data, providing reliable data support for the system's knowledge questions and answers.

[0686] 2.2 Knowledge Extraction and Organization

[0687] After completing the preprocessing, the knowledge base construction module enters the knowledge extraction and organization stage, extracting key knowledge units from the data through professional natural language processing (NLP) technology, and storing and organizing them in a hierarchical manner.

[0688] 1) Entity recognition: An improved bidirectional LSTM-CRF model is used for entity recognition to extract professional terms and key concepts in the power field. This model combines the long and short-term memory capabilities of LSTM and the sequence labeling capabilities of CRF, and can effectively cope with the diversity of professional terms while ensuring recognition accuracy.

[0689] 2) Relationship extraction: After entity recognition is completed, a graph neural network (GNN) is used for relationship extraction to identify the association between entities. Graph neural networks can transfer information based on the dependency between nodes and calculate the relationship weights between nodes through the attention mechanism, thereby effectively extracting the semantic association between entities.

[0690] 3) Knowledge organization: The entities and relationships extracted by the knowledge extraction unit are processed by the knowledge organization unit and are systematically stored as three-level knowledge bases at the document level, paragraph level, and entity level to support knowledge retrieval needs at different levels.

[0691] Document-level knowledge base: stores overall document information so that relevant documents can be quickly located when performing extensive searches. The structure of the document-level knowledge base can be expressed as:

[0692] K d ={(d i ,m i ,t i )||i∈{1,…,N d}}

[0693] Among them, d i is the document content, m i is metadata, t i Is the timestamp.

[0694] Paragraph-level knowledge base: knowledge is stored in paragraph units to support more refined retrieval. The structure of the paragraph-level knowledge base is:

[0695] K p ={(p i ,c i ,r i )||i∈{1,…,N p}}

[0696] Among them, p i For paragraph content, c i is the context information, r i Score for relevance.

[0697] Entity-level knowledge base: stores all identified entities and their related information for accurate matching. The structure of the entity-level knowledge base is:

[0698] Ke ={(e i ,r i ,a i )||i∈{1,…,N e}}

[0699] Among them, e i For entity, r i is an entity relationship, a i Attribute information

[0700] Through the above process, the knowledge base construction module realizes the systematic storage and hierarchical organization of knowledge in the power field, so that subsequent modules can effectively retrieve and reason based on knowledge at different levels. The knowledge base constructed by this module is high-quality, structured and hierarchical, laying a solid foundation for the efficient operation of the knowledge question answering system.

[0701] 3. Retrieval Enhancement Generation Module

[0702] The retrieval enhancement generation module is used in the present invention to realize multi-level information retrieval of the knowledge base, and combined with the ability of large model generation, to provide accurate and semantically rich answers. This module significantly improves the response speed of the question-answering system and the relevance and accuracy of the answers through the organic integration of sparse retrieval and dense retrieval, thereby realizing efficient query response and intelligent answer generation in knowledge question-answering applications in the power field.

[0703] 3.1 Multi-level search strategy

[0704] The retrieval enhancement generation module of the present invention adopts a multi-level retrieval strategy to achieve efficient screening and accurate positioning of user queries. The core of this module is to process user queries at different semantic levels through a hierarchical combination of sparse retrieval and dense retrieval to meet diverse query needs and adapt to complex question-answering scenarios.

[0705] 1) Sparse retrieval: Sparse retrieval uses the classic BM25 model for keyword matching and relevance calculation, mainly used to quickly filter out the documents most relevant to the query. The BM25 model is a traditional retrieval algorithm based on word frequency and inverse document frequency, suitable for processing information screening tasks in large-scale knowledge bases. Its relevance calculation formula is

[0706]

[0707] Where q represents the query, d represents the document, N is the total number of documents, and n i is the number of documents containing query word i, f i,d is the frequency of word i in document d, k iand b are the adjustment parameters of the model. Through this formula, the system can filter out highly relevant documents containing keywords in a short time to meet the basic needs of user queries.

[0708] The role of sparse search in the present invention is mainly to preliminarily filter the candidate document set that has obvious vocabulary matching with the query, which is particularly suitable for queries with dense vocabulary matching and clear keywords. This method can quickly narrow the search space in the early stage of the query, provide high-quality candidate sets for subsequent intensive search, and thus improve the overall search efficiency.

[0709] 2) Dense retrieval: After preliminary candidate documents are screened out through sparse retrieval, dense retrieval further uses the semantic vector representation generated by large models (such as BERT or GPT models) to perform deep semantic matching.

[0710] The advantage is that it can go beyond the surface matching limitations of vocabulary and identify content that is semantically related to the query, even if it is not exactly the same in vocabulary. Intensive retrieval measures the relevance of queries and documents through vector similarity, which is calculated as:

[0711]

[0712] in, and denote the vector representations of query and document respectively, · is the dot product operation, and Respectively represent the norm of the vector. Through this calculation formula, dense retrieval can capture the deep connection between the query and the document at the semantic level, allowing the question-answering system to answer questions with different vocabulary but similar semantics, further improving the accuracy and coverage of retrieval.

[0713] The multi-level retrieval strategy of the present invention combines the preliminary filtering of sparse retrieval with the semantic matching of dense retrieval to build an efficient hierarchical screening framework. In different types of query scenarios, the multi-level retrieval strategy can adapt to diverse user needs, ensure the best balance between query efficiency and answer quality, and provide a highly relevant candidate document set for the generation module.

[0714] 3.2 Retrieval Enhancement Generation

[0715] On the basis of multi-level retrieval strategy, the retrieval enhancement generation module of the present invention further combines the generation technology of large models to provide users with coherent and semantically rich answers. The generation module takes the generative model as the core, integrates and reconstructs the information in the candidate documents for the knowledge base content in the power field, and generates accurate answers that meet the query intent.

[0716] The generation process consists of the following three key steps:

[0717] 1) Contextual understanding: The generation module first performs contextual analysis on the retrieved candidate document set to extract background information and key points related to the query. The contextual understanding process relies on the semantic modeling capability of the generation model, which enables the system to accurately grasp the intent and context of the user's query and reflect the content that is highly consistent with the user's needs in the answer. In the power field, this process can identify key contextual factors such as the technical background, equipment type, and operating conditions involved in the query.

[0718] 2) Content generation and organization: Based on the results of context understanding, the generation module organically integrates and reconstructs the answer content through the generation model. The generation model extracts and summarizes the key information in the candidate documents according to the query intent and context information, and generates logically coherent answer content that conforms to the terminology and expression habits in the power field. The generation process is not just a simple combination of information, but also includes reasoning and interpretation of knowledge. For example, for complex equipment fault diagnosis queries, the generation module can provide detailed operation steps and theoretical basis, thereby enhancing the practicality and professionalism of the answer.

[0719] 3) Optimization of generated results: To ensure the accuracy, consistency and readability of the answers, the generation module further optimizes the generated results, including grammar correction, logical consistency check and terminology standardization. This process ensures the accuracy of the generated answers in terms of professional terminology, grammatical structure and logical order. Especially for question-and-answer scenarios in the power sector, the optimization step of generated results can effectively improve the professionalism of the answers, allowing users to obtain clear and easy-to-understand answers.

[0720] Through the design of the retrieval enhancement generation module, the knowledge question-answering system of the present invention achieves rapid response and high-quality answers to user queries in the power field, overcoming the problem of disconnection between retrieval and generation in traditional knowledge question-answering systems. While improving the accuracy of information retrieval, this module further enriches the answer content through generation technology. It is not only suitable for simple knowledge query scenarios, but also can cope with complex problem reasoning and professional interpretation needs, providing effective technical support for the practical application of knowledge question-answering systems in the power field.

[0721] 4. Instruction supervision fine-tuning module

[0722] The instruction supervision fine-tuning module is used in this invention to optimize and fine-tune the parameters of the large model in the professional field to improve its accuracy and understanding depth in the power knowledge question-answering scenario. By introducing instruction supervision and low-rank parameter optimization technology, this module enables the large model to efficiently adapt to the specific question-answering needs in the power field and maintain domain consistency and professionalism during the generation process.

[0723] 4.1 Instruction Supervision Technology

[0724] The instruction supervision technology in the present invention applies task instructions to the model, so that the model can accurately capture the professional needs of the power field when understanding and generating content. The core of instruction supervision is to define task goals and output requirements through a set of clear instructions, thereby guiding the model to generate answer content with goal consistency.

[0725] Specifically, the instruction supervision technology includes the following steps:

[0726] 1) Task instruction design: Based on the characteristics and requirements of power knowledge questions and answers, a series of instruction sets are designed to enable the model to focus on specific tasks during the answering process. For example, for questions such as "equipment fault diagnosis", task instructions may include "identify equipment type", "provide diagnostic steps", "suggest repair methods", etc. These instructions define the content logic and output format that the model needs to follow in answering, ensuring that the generated results meet the standards of the professional field.

[0727] 2) Instruction label generation: After the task instructions are designed, instruction labels are added to the training data to associate each data sample with the corresponding instruction. This process not only improves the model's ability to understand specific instructions, but also enables the model to automatically match the most suitable instruction logic when generating. The addition of instruction labels makes the model more task-oriented and domain-adaptable in practical applications.

[0728] 3) Supervised learning: Supervised learning is performed on training data with instruction labels, so that the model can gradually learn how to execute various instructions. During the supervised learning process, the model optimizes parameters based on the input instruction labels and training samples to improve the accuracy and consistency of the generated content. The introduction of instruction supervision makes the model more directional when generating question-and-answer content in the power field, ensuring that the generated content conforms to the professional logic and terminology specifications in the power field.

[0729] 4.2 Low-rank parameter optimization techniques

[0730] In the present invention, the instruction supervision fine-tuning module also combines the low-rank parameter optimization (Low-Rank Adaptation LoRA) technology to reduce the computational overhead of model fine-tuning while maintaining its performance and expressiveness. Low-rank parameter optimization effectively reduces the complexity of parameter updates by decomposing the model's weight matrix into a low-rank matrix, enabling the model to be fine-tuned with lower resource consumption.

[0731] The low-rank parameter optimization process includes the following steps:

[0732] 1) Weight matrix decomposition: Assuming that the weight matrix of the large model is W, the matrix can be decomposed into two low-rank matrices A and B, thereby obtaining a low-rank approximation:

[0733] W=W0+ΔW,ΔW=A·B

[0734] Among them, W0 is the original weight matrix, A and B are low-rank matrices, satisfying rank(A)=rank(B)<<rank(W). This decomposition makes the model parameter update concentrated in the low-rank subspace, thereby reducing the computational burden during fine-tuning.

[0735] 2) Parameter update: After the weight matrix is ​​decomposed, low-rank parameter optimization only updates matrices A and B, avoiding adjusting the entire weight matrix library. By optimizing parameters in a low-dimensional space, the model can efficiently learn the knowledge characteristics of the power field without generating excessive computational load. This process ensures that the model can balance accuracy and resource efficiency when performing power knowledge question-answering tasks.

[0736] 3) Quantization training: After parameter optimization is completed, the model is further quantized to compress the model weights to a lower precision (such as 8-bit floating point or fixed point), thereby further reducing the consumption of memory and computing resources. Quantization training combined with the strategy of low-rank optimization enables the model to achieve high performance in a resource-constrained environment when generating power knowledge question-and-answer content.

[0737] 4.3 Performance Improvement after Model Fine-tuning

[0738] Through the dual support of instruction supervision and low-rank parameter optimization, the instruction supervision fine-tuning module can significantly improve the performance of the model in knowledge question answering in the power field. Specifically, the fine-tuned model has significant advantages in the following aspects:

[0739] 1) Domain adaptability: The fine-tuned model is more adaptable to the professional needs of the power sector, and can accurately understand and answer professional questions related to power equipment, fault diagnosis, operation and maintenance, etc. Its answers not only have the terminology of the field, but also reflect the logical structure that conforms to industry standards.

[0740] 2) Generation efficiency: Low-rank parameter optimization and quantization training significantly reduce the computational burden of the model during inference, allowing the model to run efficiently under limited hardware resources. Compared with the unoptimized model, the fine-tuned model has been greatly improved in generation speed and response time, meeting the performance requirements in practical applications.

[0741] 3) Answer accuracy and consistency: The introduction of instruction supervision makes the model more directional when answering, ensuring that the output content is highly matched with the query requirements. Whether it is the grammatical structure of the answer content or the use of professional terms, it reflects a high degree of consistency and accuracy, thereby improving the user experience.

[0742] The instruction supervision fine-tuning module achieves efficient adaptation of the large model in the power knowledge question-answering task through the above steps. Combining instruction supervision and low-rank parameter optimization technology, the present invention not only improves the domain knowledge adaptability of the model, but also significantly reduces the consumption of computing resources, providing technical support for large-scale power knowledge question-answering applications.

[0743] 5. Knowledge fusion reasoning module

[0744] The knowledge fusion reasoning module in this invention is responsible for deep reasoning and integration of retrieved and generated knowledge to meet the multi-level and complex knowledge question-answering needs in the power field. Through multi-hop reasoning and cross-modal knowledge fusion technology, this module can not only provide accurate answers, but also generate explanatory answers, so that users can understand the reasoning process behind the answers.

[0745] 5.1 Multi-hop Reasoning Mechanism

[0746] Multi-hop reasoning is a reasoning method that spans multiple levels of knowledge nodes and is suitable for the step-by-step solution of complex problems. The knowledge fusion reasoning module of the present invention uses a multi-hop reasoning mechanism to associate and transfer information between multiple knowledge nodes to achieve reasoning across multiple knowledge links. The specific implementation steps are as follows:

[0747] 1) Path search and node screening: Multi-hop reasoning first searches for knowledge paths related to the query in the knowledge base through a path search algorithm. Path search uses a heuristic search algorithm (such as the A* algorithm or breadth-first search) to screen out key nodes that meet the logical reasoning link based on the query goal. This process ensures that the model can cover all knowledge levels related to the query during the reasoning process.

[0748] 2) Calculation of conditional probability between nodes: Based on the path search, the conditional probability between each node is calculated to determine the optimal direction of the reasoning path. The calculation formula of conditional probability is:

[0749]

[0750] Among them, e i represents the i-th node on the reasoning path, P(e i |e i-1 ) represents the slave node e i-1 To node e i Through this probability calculation, the system can automatically select the path with the most reasoning value to ensure the logical rigor and rationality of the answer.

[0751] 3) Answer generation through multi-hop reasoning: After the reasoning path is determined, the system combines the key information in the path to generate the final answer. This process uses the logical associations between the path nodes to integrate the knowledge distributed on different nodes to answer the complex needs of the query. Multi-hop reasoning can effectively answer questions that need to span multiple knowledge points and involve multiple concepts or processes, ensuring the comprehensiveness and depth of the answer.

[0752] 5.2 Cross-modal Knowledge Fusion

[0753] In order to cope with the diversity of knowledge in the power field, the knowledge fusion reasoning module of the present invention introduces cross-modal knowledge fusion technology to uniformly process multi-modal information such as text, images and structured data to generate more complete answer content. Cross-modal knowledge fusion integrates information of different modalities in the same semantic space through feature alignment and gating mechanisms to achieve cross-modal knowledge reasoning and question-answer generation.

[0754] The specific implementation steps include:

[0755] 1) Modal feature extraction and alignment: First, the system extracts features from multimodal information such as text, images, and structured data, and converts features of different modalities into vector representations. For text data, a pre-trained language model (such as BERT) is used to generate semantic vectors; for image data, a convolutional neural network (CNN) is used to extract visual features; for structured data, embedding technology is used to convert the data into a numerical vector representation. Subsequently, feature alignment technology is used to map these multimodal features to the same semantic space to ensure that features of different modalities can understand and relate to each other.

[0756] 2) Gated fusion mechanism: After feature alignment, the system performs weighted fusion of information from different modalities through a gating mechanism. The gating mechanism automatically assigns weights to different modalities by learning the contribution of different modalities to the current query to achieve optimal information fusion. The calculation formula of the gating mechanism is as follows:

[0757] z=σ(W x x+W y y+b)

[0758] Among them, x and y represent the eigenvectors of different modes, W x and W y is the weight matrix for inter-modal fusion, b is the bias term, and σ is the activation function (such as Sigmoid or ReLU). Through this mechanism, the system can filter information and assign weights between different modalities, so that the answer content can fully present the query results from the perspective of multiple modalities.

[0759] 3) Cross-modal reasoning and answer generation: In the fused semantic space, the system combines multimodal information for cross-modal reasoning to generate answers that meet the query requirements. The cross-modal reasoning process makes full use of the knowledge in text, images, and structured data to generate more explanatory and contextual answers through associative reasoning and information supplementation. For example, for fault diagnosis queries involving equipment images, the system can extract the operating status information of the equipment from the image features, and combine the fault mode described in the text to generate a diagnostic result with a logical explanation.

[0760] 5.3 Improvement of Knowledge Fusion Reasoning

[0761] Through multi-hop reasoning and cross-modal knowledge fusion technology, the knowledge fusion reasoning module has the following advantages when answering complex questions:

[0762] 1) Ability to cope with multi-level problems: The multi-hop reasoning mechanism can handle problems that require step-by-step analysis and multi-level reasoning. It is particularly suitable for complex queries involving multiple subsystems or operation steps in the power field, enabling the system to have higher accuracy and comprehensiveness in answering complex knowledge reasoning needs.

[0763] 2) Ability to integrate multimodal information: Cross-modal knowledge fusion enables the system to unify multi-source information from text, images, and structured data into the same answer, ensuring the multi-angle and comprehensiveness of the answer. It is particularly suitable for scenarios that require cross-modal information support, such as fault analysis of power equipment and operating status monitoring.

[0764] 3) High interpretability and credibility: Through the integration of multi-hop reasoning paths and cross-modal information, the answers generated by the knowledge fusion reasoning module are highly interpretable, allowing users to understand the reasoning process and the logical connections behind the answers, thereby improving the credibility of the system.

[0765] The knowledge fusion reasoning module realizes the accurate reasoning and generation of complex knowledge questions and answers in the power field through the combination of multi-hop reasoning and cross-modal knowledge fusion technology, so that the system of the present invention can provide professional, accurate and explanatory answers in a multi-level information and multi-modal data environment.

[0766] Embodiment 18:

[0767] See also Figures 1 to 6 , a knowledge question-answering optimization system in the power field based on large model retrieval enhancement generation and instruction supervision fine-tuning, the main technical contents include:

[0768] Power equipment fault diagnosis

[0769] In the fault diagnosis scenario of power equipment, the knowledge question answering optimization method provided by the present invention significantly improves the efficiency and accuracy of engineers in locating the cause of the fault and provides a reasonable solution. Power equipment such as transformers, circuit breakers and generators have complex structures and involve a variety of fault types. Traditional knowledge base retrieval is difficult to efficiently and accurately retrieve.

[0770] The system of the present invention utilizes large model retrieval enhancement generation and instruction supervision fine-tuning to achieve multi-level positioning and comprehensive analysis of fault information.

[0771] The specific operation process is as follows:

[0772] 1) Query input: The user enters a query, such as "Transformer oil temperature is abnormal, how to diagnose the cause of the fault?"

[0773] 2) Knowledge base search: The system first uses a multi-level search mechanism to filter out documents related to keywords such as "transformer" and "abnormal oil temperature" from the knowledge base. Sparse search quickly locks in document sets containing relevant keywords, and dense search further accurately filters out content related to the semantics of the problem through semantic matching. The system extracts a variety of possible faults from the search results, such as cooling system failure, oil aging, and excessive load.

[0774] 3) Instruction supervision fine-tuning: Under the guidance of the model knowledge integrated into instruction supervision fine-tuning, the system identifies the user's query owner's "fault diagnosis" type of questions, and organizes the candidate content based on the preset prior knowledge to ensure that the answer content is logically clear and focuses on the core of the problem. The answer content output by the system includes the cause, troubleshooting steps and recommended solutions for each possible fault. For example, for a cooling system fault, the answer content includes specific operating instructions such as coolant inspection and circulating pump status inspection.

[0775] 4) Multi-hop reasoning and integrated generation: The system combines multi-hop reasoning technology to comprehensively analyze key information extracted from different documents and integrate multi-level answer content. The reasoning process gradually screens and analyzes possible fault causes and generates multi-step answers that conform to the actual diagnostic process, so that users can quickly follow up and troubleshoot.

[0776] Through the above steps, the present invention not only provides multiple possibility analyses in the fault diagnosis task, but also combines specific diagnosis steps and repair suggestions to ensure the logical rigor and practicality of the answer.

[0777] Embodiment 19:

[0778] See also Figures 1 to 6 , a knowledge question-answering optimization system in the power field based on large model retrieval enhancement generation and instruction supervision fine-tuning, the main technical contents include:

[0779] Operation and maintenance scenario: In the operation and maintenance scenario of the power system, operation and maintenance personnel often face complex operating procedures and strict standard requirements. Through the knowledge question and answer system of the present invention, operation and maintenance personnel can obtain clear guidance in actual operation, improve work efficiency and reduce the error rate of operation. The application of the present invention in this scenario is particularly suitable for tasks such as daily inspections, equipment monitoring and data recording.

[0780] The application process is as follows:

[0781] 1) Question input: The operation and maintenance personnel enter a query, such as "What are the key steps in daily inspection of substations?".

[0782] 2) Knowledge retrieval and screening: The system first quickly extracts the standard operating procedures and precautions related to "substation inspection" through multi-level retrieval, and further matches the specific inspection steps and standards based on intensive retrieval. This process ensures that the answer content is comprehensive and can meet the operational details required for substation inspection.

[0783] 3) Instruction-based answer generation: Through instruction supervision, the domain knowledge integrated into the model is fine-tuned. The fine-tuned autoregressive model predicts prior knowledge about substations and daily inspections, and generates detailed answers that comply with the operation specifications in the power field. The answer details key steps such as equipment inspection, environmental monitoring, and abnormal recording to ensure that operation and maintenance personnel can strictly follow the specifications in actual operations.

[0784] 4) Cross-modal information fusion: During the inspection process, maintenance personnel may need to obtain equipment images or video guidance (such as how to check the contact status of the contactor). The system combines text information with relevant image information through the cross-modal fusion module, and provides detailed diagrams and operating instructions, so that maintenance personnel can understand the inspection steps more intuitively. For example, in the scenario of contactor contact inspection, the system provides diagrams of the normal and abnormal states of the contactor, and explains in detail the judgment criteria and subsequent processing methods for each state through text.

[0785] The output results include a complete list of daily inspection steps, image examples of key parts and suggestions for handling abnormal situations, ensuring that operation and maintenance personnel can receive clear and standardized guidance during operations.

[0786] Embodiment 20:

[0787] See also Figures 1 to 6 , a knowledge question-answering optimization system in the power field based on large model retrieval enhancement generation and instruction supervision fine-tuning, the main technical contents include:

[0788] Power safety operation knowledge: In the safe operation scenario of the power system, the accurate implementation of safety regulations and operation procedures is crucial to ensure the safety of equipment and personnel. The complexity of power operation requires operators to strictly abide by standard procedures. Through the knowledge question and answer system of the present invention, users can obtain targeted safety operation guidance to ensure that each step of the operation complies with safety regulations and reduce operational risks.

[0789] The operation process is as follows:

[0790] 1) User query: The user enters a query, such as "How to safely perform high voltage switch grounding operations?".

[0791] 2) Knowledge retrieval and generation: The system recognizes that the query belongs to the "safe operation" instruction set and retrieves all steps and precautions related to the safe operation of high-voltage equipment in the knowledge base. Sparse retrieval quickly screens relevant documents, and dense retrieval further matches semantically related specific operation steps.

[0792] 3) Instructed answer generation: Based on the keyword “high-voltage switch grounding”, the generation module generates detailed operation steps that meet the safety standards of the power industry, including preparation before grounding, safety inspection during operation, and verification after grounding. The system ensures that each step of the output content has logical rigor and completeness of the operation instructions.

[0793] 4) Multimodal reasoning and explanation: Through multi-hop reasoning, the system adds risk warnings during the operation to the answer and explains each step of the grounding operation. For content such as "how to use the grounding rod" or "risks in the grounding steps", the system not only provides detailed operation descriptions, but also uses cross-modal information to generate graphic and text explanations, so that operators can more intuitively understand the operation specifications and potential risks.

[0794] The system's output results include detailed steps for high-voltage switch grounding operations, possible risk warnings, and necessary safety inspection instructions, ensuring that users can complete safe operations of high-voltage equipment in accordance with standardized processes.

[0795] Embodiment 21:

[0796] See also Figures 1 to 6 , a knowledge question-answering optimization system in the power field based on large model retrieval enhancement generation and instruction supervision fine-tuning, the main technical contents include:

[0797] Preventive maintenance scenario: In the power system, preventive maintenance is crucial for the long-term stable operation of equipment. The application of this invention in preventive maintenance tasks can help operation and maintenance personnel formulate reasonable maintenance plans, discover and deal with potential problems in advance, reduce failure rates and extend equipment life.

[0798] 1) Query input: The user enters a query, such as "How to develop a preventive maintenance plan for generators?"

[0799] 2) Knowledge retrieval and screening: The system screens out knowledge content related to generator preventive maintenance through multi-level retrieval, including maintenance frequency, key inspection areas, early warning indicators and other information, to ensure that the output content covers comprehensive maintenance requirements.

[0800] 3) Instruction generation and logical reasoning: The instruction supervision fine-tuning module classifies the user query as a "preventive maintenance" task, and combines multi-hop reasoning to integrate knowledge from multiple documents to generate key steps covering generator maintenance. The system will provide detailed instructions on regular inspections, data recording, performance monitoring, etc. to ensure that users receive comprehensive maintenance guidance.

[0801] 4) Result optimization and output: In the generated maintenance plan, the system provides some actual inspection indicators, such as temperature, vibration, noise, etc., and explains the judgment criteria and maintenance interval of each indicator. The maintenance plan output by the system includes the regular maintenance frequency, specific operation steps and reference indicators for each task, providing scientific support for the daily maintenance of operation and maintenance personnel.

Claims

1. A knowledge question-answering optimization system for the power field based on large model retrieval enhancement generation and instruction supervision fine-tuning, characterized by: include: Knowledge base construction module, retrieval enhancement generation module, instruction supervision fine-tuning module, knowledge fusion reasoning module; The knowledge base construction module is used to establish a multi-source heterogeneous knowledge base containing professional knowledge in the power field. The multi-source heterogeneous knowledge base includes a document-level knowledge base, a paragraph-level knowledge base, and an entity-level knowledge base. The retrieval enhancement generation module uses a large model of knowledge retrieval in the electric power field to perform multi-level information retrieval on a multi-source heterogeneous knowledge base based on the user's query question, thereby generating an initial answer; The instruction supervision fine-tuning module adjusts the parameters of the large model of power field knowledge retrieval through instruction supervision; The knowledge fusion reasoning module integrates the user query question and the initial answer output by the retrieval enhancement generation module through multi-hop reasoning and cross-modal knowledge fusion technology to generate an explanatory answer.

2. The power field knowledge question answering optimization system based on large model retrieval enhancement generation and instruction supervision fine-tuning according to claim 1 is characterized in that: The steps of the knowledge base construction module to establish a multi-source heterogeneous knowledge base containing professional knowledge in the power field include: a11 obtains the original power domain knowledge dataset D; a12 preprocesses the original power field knowledge dataset D to obtain the normalized knowledge dataset D′, as shown below: In the formula, i is the knowledge data serial number, X is the total number of knowledge data, and D i is the i-th knowledge data; To standardize knowledge space; The preprocessing includes data cleaning and normalization processing; a13 normalizes the knowledge data D in the knowledge dataset D′ i Convert to structured data D struct , as shown below: D struct ={(e i ,r i ,a i )∣∣i∈{1,…,X}} (2) In the formula, e i Represents an entity, r i is the relationship type, a i is the attribute value; a14 builds a hierarchical knowledge storage system K, as shown below: In the formula, K d is the document-level knowledge base, K p is the paragraph-level knowledge base, K e It is an entity-level knowledge base; Among them, the index structure of the hierarchical knowledge storage system K As shown below: Where M is the maximum number of neighbors per layer, efConstruction is the size of the candidate set during construction, HNSW is the indexing algorithm, l is the level number, K l is the lth level knowledge base; d, p, e represent document level, paragraph level, and entity level respectively; a15 uses natural language processing technology to extract struct Extract key knowledge units and store them hierarchically into the hierarchical knowledge storage system K, as shown below: K d ={(d i ,m i ,t i )∣∣i∈{1,…,N d }} (5) K p ={(p i ,c i ,q i )∣∣i∈{1,…,N p }} (6) K e ={(e i ,r i ,a i )∣∣i∈{1,…,N e }} (7) Where, d i is the document content, m i is metadata, t i is the timestamp; p i For paragraph content, c i is the context information, q i is the correlation score; N d is the total number of document-level knowledge data; N p is the total number of paragraph-level knowledge data; N e is the total amount of entity-level knowledge data.

3. The power field knowledge question answering optimization system based on large model retrieval enhancement generation and instruction supervision fine-tuning according to claim 2 is characterized in that: The knowledge base construction module is also used to update the multi-source heterogeneous knowledge base in real time, and the steps are as follows: a21 computing new knowledge dataset Importance score As shown below: In the formula, k is the knowledge data, ω k is the knowledge item weight coefficient, and ∑ k ω k =1; Among them, the knowledge quality scoring function Q(k) is as follows: Q(k)=α·Accuracy(k)+β·Timeliness(k)+γ·Completeness(k) (9) Where α, β, γ are weight coefficients, and α+β+γ=1; Accuracy(k) is the accuracy score of knowledge data k, Timeliness(k) is the timeliness score of knowledge data k; Completeness(k) is the attribute completeness of knowledge data k; a22 The knowledge base K at time step t t Perform the update operation to obtain the knowledge base K at time step t+1 t+1 , as shown below: In the formula, θ t is the knowledge acquisition threshold for time step t; Among them, the update function Update is as follows: In the formula, Score(k) is the importance score of knowledge data k; a23 updates the threshold parameter by gradient descent method as follows: In the formula, θ t+1 is the knowledge acquisition threshold at time step t+1; η is the learning rate, is the loss function, is the gradient function.

4. The power field knowledge question answering optimization system based on large model retrieval enhancement generation and instruction supervision fine-tuning according to claim 2 is characterized in that: The knowledge base building module is also used to clean up the remaining data, the steps are as follows: a31 calculates the similarity between knowledge data pairs as follows: Similarity(k i1 ,k i2 )=cos(E(k i1 ),E(k i2 ))+λ·JaccardSim(k i1 ,k i2 ) (13) Where E(·) is the knowledge encoding function, JaccardSim is the Jaccard similarity; Similarity(k i1 ,k i2 ) is the knowledge data pair (k i1 ,k i2 ) similarity between i1 , k i2 All are knowledge data; λ is the weight; a32 merges the knowledge data whose similarity exceeds the threshold and constructs the merged knowledge data k merged , as shown below: In the formula, i is the knowledge data serial number, n is the total number of knowledge data to be merged, is the knowledge data set to be merged; Q(k) is the knowledge quality scoring function; k is the knowledge data, Merge is the merging function; a33 builds the knowledge base after cleaning the remaining data As shown below: In the formula, k′ is the knowledge data; Similarity is the similarity function, θ sim is the similarity threshold; K is the hierarchical knowledge storage system.

5. The power field knowledge question answering optimization system based on large model retrieval enhancement generation and instruction supervision fine-tuning according to claim 1 is characterized in that: The retrieval enhancement generation module uses the large model of power field knowledge retrieval to perform multi-level information retrieval on multi-source heterogeneous knowledge bases based on user query questions, so as to generate the initial answer in the following steps: b1 builds a document-level retrieval layer and uses it to search the input query q in the document-level knowledge base to obtain the document-level retrieval result R d ; The document-level retrieval result R d As shown below: In the formula, R d is the document-level retrieval result, i is the knowledge data serial number, doc i is the i-th related document; is the standardized knowledge space; θ d is the document-level retrieval threshold; Among them, the search score score (doc i ) is as follows: score(doc i )=λ s ·BM25(q,doc i )+λ d ·cos(E q (q),E d (doc i )) (17) In the formula, λ s , d are the weight coefficients of sparse retrieval and dense retrieval respectively, and λ s +λ d =1; E q (q), E d (doc i ) are query encoder and document encoder respectively; BM25(q,doc i ) is the document-level retrieval relevance score; b2 builds a paragraph-level retrieval layer and introduces a paragraph context enhancement mechanism; The calculation formula of the paragraph context enhancement mechanism is as follows: In the formula, j is the paragraph number, p j-1 、p j 、p j+1 Respectively represent the word representation of paragraphs j-1, j, and j+1; ⊕ represents text concatenation; BERT is the text concatenation function; PE(j) is the position encoding; Enhance results for paragraph context; b3 Use the paragraph-level retrieval layer to search in the document-level retrieval results to obtain the paragraph-level retrieval result R p ; The paragraph-level retrieval result R p As shown below: R p =TopK({p j |p j ∈Split(doc i ),doc j ∈R d },K’ p ) (18) In the formula, R p is the paragraph-level retrieval result; K' p is the number of paragraphs to be retained; TopK is the retrieval function; Split is the segmentation function; b4 builds an entity-level retrieval layer and introduces a multi-head cross-attention mechanism; The calculation formula of the multi-head cross attention mechanism is as follows: MultiHead(Q,K,V)=Concat(head1,…,head h )W O (19) Where MultiHead(Q,K,V) is the enhanced result of the multi-head cross attention mechanism; W O is the output projection matrix; Concat is the fusion concatenation function; h is the total number of attention heads; Q, K, and V represent the query matrix, key value matrix, and value matrix, respectively; Among them, the attention head u As shown below: In the formula, head u is the u-th attention head, All are projection matrices; Among them, the cross attention mechanism Attention(Q,K,V) is as follows: In the formula, softmax is the normalization function, d k is the dimension of the key vector; K T is the transposed matrix of the key-value matrix; b5 Use the entity-level retrieval layer to extract information from the paragraph-level retrieval results to obtain the entity-level retrieval result R e ; b6 dynamically fuses the retrieval results of each level through residual connection and layer normalization, as shown below: R final =LayerNorm(∑ l∈{d,p,e} w l R l +FFN(∑ l∈{d,p,e} w l R l )) (22) In the formula, R final is the fusion retrieval result; l is the level number, d, p, e are document level, paragraph level, and entity level respectively; R l is the l-th level retrieval result; LayerNorm is the layer normalization function; w l is the fusion weight; FFN is a two-layer feedforward network.

6. The power field knowledge question answering optimization system based on large model retrieval enhancement generation and instruction supervision fine-tuning according to claim 1 is characterized in that: The step of adjusting the parameters of the large power field knowledge retrieval model by the instruction supervision fine-tuning module through instruction supervision includes: c1 builds the instruction tag set; c2 adds instruction labels to the knowledge data and constructs a training set; c3 takes instruction labels and knowledge data as input, and outputs the knowledge data corresponding to the instruction labels. It uses the training set to train the large model to obtain a large model with directionality. The large model selects a BERT model or a GPT model.

7. The power field knowledge question answering optimization system based on large model retrieval enhancement generation and instruction supervision fine-tuning according to claim 6 is characterized in that: The step of using the training set to train the large model to obtain a large model with directionality includes: c31 performs a structured decomposition of the projection matrix of the large model and introduces adaptive updates as follows: In the formula, is the projection matrix of the large model; W is the basic projection matrix; ΔW is the low-rank structure; B and A are both low-rank matrices, α is the adaptive coefficient, and Adapt is the input-related adaptive adjustment item; c32 builds an optimization model, and the objective function of the optimization model is as follows: In the formula, is the objective function of the optimization model; is the main task performance function; λ1, λ2, λ3 are all trade-off coefficients; ∥.∥ F is the norm; c33 uses the optimization model to update the low-rank matrix B and the low-rank matrix A to obtain a large model with optimized parameters; c34 performs quantitative training on the large model after parameter optimization to obtain a large model with directionality.

8. The power field knowledge question answering optimization system based on large model retrieval enhancement generation and instruction supervision fine-tuning according to claim 7 is characterized in that: The step of performing quantization training on the large model after parameter optimization includes: c341 performs b-bit quantization on the weights of the large model after parameter optimization, as shown below: In the formula, w q is the quantized weight; b is a constant, w is the weight of the large model after parameter optimization; clip is the truncation function; round is the rounding function; The mean μ and standard deviation σ of the weights are as follows: In the formula, g i is the weight sequence number; g represents the weight grouping, and |g| is the grouping size; For g i Weights; μ g , σ g are the mean and standard deviation of the weight of group g, respectively; c342 quantifies the loss function of the large model after parameter optimization, and dynamically adjusts the weight of the large model based on the loss function; The loss function of the large model after parameter optimization is as follows: In the formula, β is the quantization loss weight, dequant(·) is the dequantization operation; The loss function of the large model after parameter optimization; is the task loss function; c343 calculates the accuracy of the large model after parameter optimization and dynamically adjusts the weight of the large model according to the accuracy; The accuracy of the large model after parameter optimization is as follows: In the formula, l is the level number, p l is the accuracy of the lth layer; θ high and θ low Switch threshold for accuracy; Among them, the layer sensitivity (l) is as follows: In the formula, O l is the output of the lth layer; W l is the weight of the lth layer; Represents the gradient of the loss function with respect to the network weight; Loss represents the total loss function; c344 builds an adaptive adjustment system and uses it to adjust the weights of large models; The learning rate of the adaptive adjustment system is as follows: Where η t is the adjusted learning rate, η0 is the original learning rate; β1 and β2 are momentum parameters, is the initial gradient norm; The adaptive adjustment system also introduces Warm-up and Cosine annealing methods, as shown below: Where η max is the learning rate extreme value, t warmup is the number of warm-up steps, t total is the total number of training steps; t k is the training step.

9. The power field knowledge question answering optimization system based on large model retrieval enhancement generation and instruction supervision fine-tuning according to claim 1 is characterized in that: The knowledge fusion reasoning module integrates the user query question and the initial answer output by the retrieval enhancement generation module through multi-hop reasoning to generate an explanatory answer, including the following steps: d11 uses a path search algorithm to find knowledge paths related to user query questions in multi-source heterogeneous knowledge bases; d12 calculates the probability of each reasoning path and determines the optimal direction of the reasoning path; The probabilities of the reasoning paths are as follows: In the formula, i v is the inference node number, n v is the total number of inference nodes; P(p v ) is the path reasoning probability; represents the reasoning path; To assess the reliability of the relationship; Indicates that in the reasoning path, the i v nodes and the i-th v +1 relationship between nodes; Indicates the reasoning path To the reasoning path ; Among them, the relationship transfer probability As shown below: In the formula, j v is the inference node number; softmax is the normalization function, is the relation mask matrix, E pos Encode for relative position; Indicates the jth v +1 transpose of the key vector of the node; d k is the dimension of the key vector; Indicates the jth v The query vector of nodes; d13 generates explanatory answers based on the optimal direction of the reasoning path, using the logical connections between path nodes and integrating the knowledge data on different path nodes.

10. The power field knowledge question answering optimization system based on large model retrieval enhancement generation and instruction supervision fine-tuning according to claim 1 is characterized in that: The knowledge fusion reasoning module uses cross-modal knowledge fusion technology to reason and integrate the user query question and the initial answer output by the retrieval enhancement generation module to generate an explanatory answer, including the following steps: d21 builds a multimodal feature extraction model; d22 extracts features from knowledge data through a multimodal feature extraction model, and uses feature alignment technology to map features of different modalities into the same semantic space; The knowledge data includes text data, image data and structured data; The features of different modalities include semantic vectors, visual features, and numerical vectors; The feature alignment technique is as follows: In the formula, f aligned is the feature after being mapped to the same semantic space; OT is the optimal transmission algorithm; f t is the text feature; f g is the graph feature; C is the feature distance metric; For the i t Text features With the jth g Spectral features The distance measure between d23 uses a gating mechanism to perform weighted fusion of features from different modalities and construct a fused semantic space; The calculation formula for weighted fusion of features of different modalities through the gating mechanism is as follows: Z=σ g (W g [f t ;f g ]+b g ),f out =Z⊙f fusion +(1-Z)⊙f aligned (35) In the formula, f out is the feature after weighted fusion; Z is the gate vector; σ g is the sigmoid function; W g is the weight matrix; b g is the bias vector; Among them, the cross-modal feature fusion network f fusion As shown below: f fusion =LayerNorm(MLP([f t ;f g ])+λ f ·Cross-Attention(f t ,f g )) (36) In the formula, LayerNorm is the layer normalization function; MLP is the multi-layer perceptron; Cross-Attention is the correlation function; λ f is the scaling parameter; d24 generates explanatory answers in the fused semantic space through associative reasoning and information supplementation.

Images