A method and system for power system operation and maintenance based on retrieval enhancement technology

Abstract

The present invention discloses a method and system for operation and maintenance of an electric power system based on retrieval enhancement technology. The method comprises the following steps: performing weight analysis on each layer of a preset large model during training of the preset large model, and dynamically freezing the layers in the preset large model that meet preset conditions, so as to obtain a full-parameter fine-tuning large model; optimizing the parameters of each layer in the base retriever based on a Mamba architecture, and introducing a KAN network, so as to obtain a first retriever; integrating and optimizing the first retriever through a gated expert integration network, so as to obtain a second retriever; then, constructing an enhanced data set according to the output result of the full-parameter fine-tuning large model, and training the second retriever according to the enhanced data set, so as to obtain a target retriever; integrating the target retriever and the full-parameter fine-tuning large model, so as to obtain a target operation and maintenance large model, and executing a matching operation and maintenance strategy with the operation and maintenance result output by the target operation and maintenance large model, so as to maintain the stability of the electric power system.

Description

Technical Field

[0001] The present invention relates to the technical field of electric power operation and maintenance, and in particular to a method and system for electric power system operation and maintenance based on retrieval enhancement technology. Background Art

[0002] With the rapid development of artificial intelligence, especially in the field of natural language processing, large language models (LLM) have made significant progress. In many industries such as physics, chemistry, and medicine, vertical expert large models (Expert LLM) have emerged. These models usually form specialized models through technical means such as full parameter fine-tuning, continuous pre-training, or efficient parameter fine-tuning. Based on the knowledge of domain experts, they are fine-tuned on specific question-and-answer data sets to produce more accurate answers in the corresponding fields, so that each field can generate relevant operation and maintenance strategies according to the results output by the large model, thereby promoting the intelligent process in the vertical field of power operation and maintenance.

[0003] When applying large models to power operation and maintenance, due to the particularity of power operation and maintenance data, the power operation and maintenance results output by the constructed large model are inaccurate, which in turn affects the accuracy of the generation of subsequent operation and maintenance strategies, thereby affecting the stability of the power system.

[0004] It can be seen that how to optimize the training process of large models and improve the reliability of large model output has become a technical problem that technical personnel in this field need to solve urgently. Summary of the invention

[0005] The present invention provides a method and system for constructing a large model of power operation and maintenance based on retrieval enhancement technology, so as to solve the problem of how to enhance the retrieval capability and training efficiency in the large model training process, and improve the accuracy of the large model output content and the reliability of subsequent maintenance strategies.

[0006] In order to solve the above technical problems, an embodiment of the present invention provides a power system operation and maintenance method based on retrieval enhancement technology, comprising:

[0007] In the process of training the preset large model based on the acquired power operation and maintenance training sample data, weight analysis is performed on each layer of the preset large model, and the layers in the preset large model that meet the preset conditions are dynamically frozen according to the weight analysis results to obtain a trained full-parameter fine-tuning large model;

[0008] The base retriever body is constructed based on the state space model of the Mamba architecture, and the Kolmogorov-Arnold network is introduced to optimize the parameters of each level in the base retriever body to obtain the first retriever body;

[0009] Performing integrated optimization on the first retriever body through a gated expert integration network to obtain a second retriever body;

[0010] During the retriever training process, an enhanced data set is constructed based on the output of the large model with all-parameter fine-tuning, and the second retriever body is trained according to the enhanced data set to obtain a target retriever;

[0011] The target retriever and the full-parameter fine-tuning large model are integrated by using a collaborative filtering algorithm to obtain a target operation and maintenance large model;

[0012] During the power operation and maintenance process of the target power grid, a matching operation and maintenance strategy is executed based on the operation and maintenance results output by the target operation and maintenance large model.

[0013] Furthermore, the weight analysis of each layer of the preset large model includes:

[0014] According to the weights of each layer of the preset large model, the principal component analysis technology is used to decompose the eigenvalues ​​corresponding to each layer and generate the corresponding principal component matrix;

[0015] Based on the principal component matrix, projecting the weights of each layer by a sparse sub-projection technique;

[0016] In the projected space, the feature importance score of each layer is calculated based on the feature value.

[0017] Furthermore, the layers that meet the preset conditions in the preset large model are dynamically frozen according to the weight analysis result to obtain a trained full-parameter fine-tuning large model, including:

[0018] Analyze the importance of the features of each layer to determine the priority order, so as to sort the layers, freeze the layers that meet the preset conditions according to the sorting results and perform iterative updates;

[0019] During each iteration, the frozen layers are re-determined based on the weighted average norm of all layers in the preset large model currently calculated, so as to train and obtain the full-parameter fine-tuning large model.

[0020] Furthermore, the state space model based on the Mamba architecture constructs a base retriever body, and introduces a Kolmogorov-Arnold network to optimize the parameters of each level in the base retriever body to obtain a first retriever body, including:

[0021] Under the Mamba architecture, the weights of each base layer and the input power operation and maintenance training sample data are recursively applied layer by layer to form the base retriever body; the architecture of the base retriever body is represented by the following formula:

[0022] in, is the weight matrix of the Mamba architecture layer; For the The input of the layer; It is a recursive calculation function layer by layer; The basic structure of the base search engine.

[0023] The architecture of the first retriever body is represented by the following formula:

[0024]

[0025] in, is the weight of the Kolmogorov-Arnold network layer; An expression layer for nonlinearly reconstructing features of the Mamba architecture layer; This is the architecture of the first retriever body.

[0026] Furthermore, the process of acquiring the enhanced data set includes:

[0027] Inputting the pre-built power operation and maintenance query sample into the full-parameter fine-tuning large model, and performing pre-defined query operations through the full-parameter fine-tuning large model to generate initial enhanced data;

[0028] The initial enhanced data is integrated and classified and annotated to obtain the enhanced data set.

[0029] Furthermore, the training of the second retriever body according to the enhanced data set to obtain a target retriever includes:

[0030] According to the matching characteristics between the query text and the related text, defining the positive and negative sample pairs in the enhanced data set;

[0031] Under the preset loss conditions of the positive and negative samples, the second retriever body is trained by an optimizer to obtain the target retriever.

[0032] Furthermore, in the power operation and maintenance process of the target power grid, the operation and maintenance results output by the target operation and maintenance large model are used to execute a matching operation and maintenance strategy, including:

[0033] In response to external power operation and maintenance question and answer instructions, control the target operation and maintenance large model to output the operation and maintenance results composed of power operation and maintenance question and answer knowledge;

[0034] Execute an operation and maintenance strategy that matches the operation and maintenance result.

[0035] Furthermore, in response to the external power operation and maintenance question and answer instruction, controlling the target operation and maintenance large model to output the operation and maintenance result consisting of power operation and maintenance question and answer knowledge includes:

[0036] Inputting the power operation and maintenance question and answer instruction into the full parameter fine-tuning large model to generate a corresponding query vector;

[0037] In the target retriever, the query vector is used to retrieve knowledge items from the power operation and maintenance standard knowledge base to obtain several recommended knowledge sets;

[0038] The several recommended knowledge sets are fed back into the full-parameter fine-tuning large model to generate the operation and maintenance results that meet the conditions.

[0039] Furthermore, after the first retriever body is integrated and optimized by the gated expert integration network to obtain the second retriever body, the method further includes:

[0040] The weights of the base retriever body, the first retriever body and the second retriever body are respectively initialized randomly with uniform distribution.

[0041] Another embodiment of the present invention provides a power system operation and maintenance system based on retrieval enhancement technology, including:

[0042] A full-parameter fine-tuning large model training module is used to perform weight analysis on each layer of the preset large model during the training of the preset large model based on the acquired power operation and maintenance training sample data, and dynamically freeze the layers in the preset large model that meet the preset conditions according to the weight analysis results to obtain a trained full-parameter fine-tuning large model;

[0043] The first-level search optimization module is used to construct a base searcher body based on the state space model of the Mamba architecture, and introduce the Kolmogorov-Arnold network to optimize the parameters of each level in the base searcher body to obtain the first searcher body;

[0044] A secondary search optimization module, used for integrating and optimizing the first search engine body through a gated expert integration network to obtain a second search engine body;

[0045] A retriever training module, used to construct an enhanced data set based on the output of the large model with all-parameter fine-tuning during the retriever training process, and train the second retriever body according to the enhanced data set to obtain a target retriever;

[0046] A model integration module, used to integrate the target retriever and the full-parameter fine-tuning large model using a collaborative filtering algorithm to obtain a target operation and maintenance large model;

[0047] The operation and maintenance strategy generation module is used to execute a matching operation and maintenance strategy based on the operation and maintenance results output by the target operation and maintenance large model during the power operation and maintenance process of the target power grid.

[0048] Compared with the prior art, the embodiments of the present invention have the following advantages:

[0049] (1) The embodiment of the present invention utilizes a two-stage retrieval mechanism. In the retrieval stage of the full-parameter large model, the KAN network is used to effectively optimize the parameters of each level of the Mamba architecture retriever, which significantly improves the retrieval accuracy and compliance of the generated answers of the model when facing specific power operation and maintenance problems, thereby enabling accurate formulation of maintenance strategies.

[0050] (2) The present invention adopts the method of freezing some layers and introducing a learnable matrix, which greatly reduces the unnecessary computational burden in the training process. At the same time, the weighted average norm sorting method is used to effectively determine the frozen layers, ensuring the efficiency and sparsity of training, reducing the time and computing resources required for training, and improving the model output effect on the basis of realizing a fine-tuning adaptive retrieval mechanism system. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is a flowchart of a method for operation and maintenance of a power system based on retrieval enhancement technology in one embodiment of the present invention;

[0052] Figure 2 It is a schematic diagram of the process of constructing a large model with all-parameter fine-tuning in one embodiment of the present invention;

[0053] Figure 3 It is a schematic diagram of a process of completing large model question answering by a dual-stage retrieval enhancement generation mechanism in one embodiment of the present invention;

[0054] Figure 4 It is a schematic diagram of the maintenance process of power equipment in one embodiment of the present invention;

[0055] Figure 5 It is a schematic diagram of the structure of an electric power system operation and maintenance system based on retrieval enhancement technology in one embodiment of the present invention. DETAILED DESCRIPTION

[0056] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. The purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0057] In the description of this application, the terms "first", "second", "third", etc. are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first", "second", "third", etc. may explicitly or implicitly include one or more of the feature. In the description of this application, unless otherwise specified, "plurality" means two or more.

[0058] In the description of the present application, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be a connection between the two elements. The terms "vertical", "horizontal", "left", "right", "upper", "lower" and similar expressions used herein are only for illustrative purposes, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0059] In the description of this application, it should be noted that, unless otherwise defined, all technical and scientific terms used in the present invention have the same meaning as those commonly understood by those skilled in the art. The terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood by specific circumstances.

[0060] In order to solve the problems in the background technology, the present invention proposes a mechanism called (PoD-RAG) two-stage retrieval enhancement generation. It adopts a variety of technical means to train a large expert model in the field of power operation and maintenance, uses a special algorithm to improve the performance and reliability of the model, and combines a special RAG method to update the knowledge base to ensure that the retrieval results are closer to and more relevant to the real scene. Documents or information fragments.

[0061] Based on this, an embodiment of the present invention provides a power system operation and maintenance method based on retrieval enhancement technology. For details, see Figure 1 , Figure 1 The flowchart of the power system operation and maintenance method based on the retrieval enhancement technology in one embodiment of the present invention is shown, which includes the following steps:

[0062] S1. In the process of training the preset large model based on the acquired power operation and maintenance training sample data, weight analysis is performed on each layer of the preset large model, and the layers in the preset large model that meet the preset conditions are dynamically frozen according to the weight analysis results to obtain a trained full-parameter fine-tuning large model.

[0063] In some embodiments of the present invention, electric power professional data can be collected from the electric power operation and maintenance data center, which may include electric power operation and maintenance data records, electric power system operation manuals, electric power operation and maintenance fault history, equipment monitoring data, etc. The collected data is used to construct electric power operation and maintenance training sample data for training the large model.

[0064] It can be understood that first, a full-parameter fine-tuning power operation and maintenance vertical domain expert model for knowledge question and answer is trained. As an example, the general large model selected in the embodiment of the present invention, that is, the preset large model, is the Qwen2.5-7B-Instruct large model.

[0065] Through the sparse fast Fourier transform (Spar-FFT) technology, the general large model fine-tuning training is transformed into an expert large model for the field of power operation and maintenance, that is, the preset large model described in the embodiment of the present invention.

[0066] This embodiment sets the fine-tuning large model to be divided into Model layer, the weight matrix of each layer is ,in , whose serial number represents the corresponding network model layer that represents the fine-tuning training of the general large model; and the power operation and maintenance training sample data is defined as , , indicating that it is used for The input data of the layer, is the sample size, It is The number of features of the layer input data.

[0067] In order to let the model learn the complex knowledge in the power operation and maintenance data and adapt to the search enhancement rules of power operation and maintenance experts under the condition of minimum computing consumption when fine-tuning all parameters, this step combines principal component analysis technology and sparse subspace projection matrix technology to analyze and fine-tune the weights and features of the large model. Specifically,

[0068] According to the weights of each layer of the preset large model, the covariance matrix is ​​calculated using the principal component analysis technology, and the covariance matrix is ​​subjected to eigenvalue decomposition to obtain the eigenvalues ​​and eigenvectors corresponding to each layer, thereby accumulating and generating the corresponding principal component matrix.

[0069] As an example, for each layer weight and the input data , the covariance matrix constructed It is expressed by the following formula:

[0070]

[0071] in, , reflecting the covariance relationship between weights and inputs.

[0072] And for the covariance matrix The eigenvalues ​​obtained by decomposition and the eigenvector , it is expressed by the following formula:

[0073]

[0074] in, Indicates the number of principal components whose cumulative eigenvalues ​​account for 95%; Indicates The covariance matrix of the layer eigenvalues; Indicates The covariance matrix of the layer feature vectors.

[0075] Next, based on the principal component matrix, the weights of each layer are projected by sparse subprojection technology. The purpose of this step is to project the weights into a low-dimensional space to reduce the amount of calculation and improve the generalization ability of the model. As an example, the principal component matrix is ​​set as , the process of projecting the weights is expressed by the following formula:

[0076]

[0077] in, , which represents the representation of weights in the sparse principal component space.

[0078] In the projected space, the feature importance score of each layer is calculated based on the feature value. It can be understood that the importance score here is the feature importance. For example, the importance score It is expressed as follows:

[0079]

[0080] After completing the weight analysis of each layer of the preset large model, the feature importance of each layer is analyzed to determine the priority order according to the obtained importance score, so as to sort the layers. In some embodiments of the present invention, the importance score of the weight of each layer is sorted from low to high, which can be expressed as:

[0081]

[0082] in, represents a set of layers sorted by importance score; Represents the function used to rank the set of importance scores for layer weights.

[0083] According to the above sorting results, the layers that meet the preset conditions are frozen and iteratively updated. It can be understood that the smaller the eigenvalue of the layer, the smaller the contribution of the representative to the model, and the more suitable it is to be frozen. Based on this, in some embodiments of the present invention, the top 25% of the layers are selected to be frozen, and the model will not update the weights of these layers during training. Specifically, the frozen layer can be represented as follows:

[0084]

[0085]

[0086] in, Indicates the set of selected layers that need to be frozen; It means to select the first 25% of elements from the sorted set Avg-Sorted. represents the set of the top 25% of layers with the lowest scores after sorting by weight importance score, and 0.25*L represents the number of layers to be frozen; , which means that for all frozen layers , its weight gradient Set to 0.

[0087] It is worth noting that in each round of training, the frozen layers need to be dynamically updated, and the importance scores of each layer need to be recalculated and sorted. The frozen layer set is updated according to the latest sorting, and the weights of the frozen layers will remain unchanged during the training process, and only other non-frozen layers will be updated. Specifically, the above process can be expressed by the following formula:

[0088]

[0089] in, represents the collection of updated frozen layers; It means to sort the importance scores of each layer after recalculation and update, and also select the first 25% of the elements in the set.

[0090] It is understandable that, since the above training involves multiple rounds of iterative processes, in some embodiments of the present invention, the norm can be used to reselect the frozen layers. Specifically, in each iteration update process, the frozen layers are re-determined according to the weighted average norm of all layers in the preset large model currently calculated, so that during the training process, the fine-tuning large model is frozen layer by layer and optimized to obtain a full-parameter fine-tuning large model.

[0091] Specifically, in practical applications, the specific process of building a large power operation and maintenance expert model through full parameter fine-tuning can be referred to Figure 2 As shown, Figure 2 Shown is a schematic diagram of the process of constructing a large model with full parameter fine-tuning in one embodiment of the present invention.

[0092] from Figure 2 It can be seen that the process of constructing the power operation and maintenance expert big model in the embodiment of the present invention involves three processes. The first is to extract technical manuals / operating procedures in the field of power operation and maintenance, historical fault cases in the field of power operation and maintenance, power operation and maintenance professional knowledge, and power operation and maintenance service summaries / operation records as expert data in the field of power operation and maintenance. Then, these expert data are subjected to data cleaning, information extraction, attribution analysis and other operations to remove redundant and noisy information, and these pre-processed data are used for fine-tuning of the power operation and maintenance big model. During the fine-tuning process, a general large model base is selected, such as the Qwen2.5-7B-Instruct large model exemplified in S1 above, and a sparse fast Fourier transform is performed on this general large model in combination with the pre-processed expert data, involving weight analysis of the large model layer. It is worth noting that in this process, the embodiment of the present invention also introduces a fine-tuning method of freezing and optimizing layer by layer, and finally obtains a large model of power operation and maintenance expert with full parameter fine-tuning.

[0093] S2. Construct a base retriever body based on the state space model of the Mamba architecture, and introduce the Kolmogorov-Arnold network to optimize the parameters of each level in the base retriever body to obtain the first retriever body.

[0094] It can be understood that the embodiment of the present invention combines the state space model of the Mamba architecture (Mamba) and the Kolmogorov-Arnold network (KAN) to construct a high-quality retriever Mamba-KAN.

[0095] Since Mamba does not need to use the self-attention mechanism for calculation, it is more lightweight for the vector retriever. Based on this, the embodiment of the present invention first constructs the Mamba base layer, that is, the base retriever body. Under the Mamba architecture, the weights of each base layer and the input power operation and maintenance training sample data are recursively layer by layer to form the base retriever body. The architecture of the base retriever body can be expressed by the following formula:

[0096] in, The basic structure of the base search engine. is the weight matrix of the Mamba architecture layer; For the The input of the layer; It is a layer-by-layer recursive calculation function, including block matrix multiplication, nonlinear activation function ReLU and pooling operation.

[0097] In this embodiment, KAN is introduced as an auxiliary basic layer into each base layer of the Mamba architecture to obtain a first retriever body, which can adapt to a more complex and redundant power operation and maintenance data knowledge base, which is specifically expressed by the following formula:

[0098]

[0099] in, is the weight of the Kolmogorov-Arnold network layer; An expression layer for nonlinearly reconstructing features of the Mamba architecture layer; This is the architecture of the first retriever body.

[0100] S3. The first retriever body is integrated and optimized through a gated expert integration network to obtain a second retriever body.

[0101] It is understandable that not all contents in the power operation and maintenance knowledge base are complex, and the shallow knowledge that can be directly indexed requires an independent model to process. If only one Mamba-KAN is used, it may not be possible to take into account the different situations of "simple knowledge extraction" and "hidden complex content output".

[0102] Based on this, in some implementations of the present invention, a Mix of Experts (MoE) is introduced, i.e., a gated expert integrated network, which can combine multiple different Mamba-KANs, use gate circuits for control, and assign "simple" or "complex" tasks to the corresponding models, thereby ensuring the accuracy of the model output. Specifically, as an example, the second retriever body obtained by introducing MoE again It is expressed by the following formula:

[0103]

[0104]

[0105] Among them, E represents the number of experts; represents the weight of the gating network; Indicates The output of an expert.

[0106] S4. During the retriever training process, an enhanced data set is constructed based on the output results of the full-parameter fine-tuning large model, and the second retriever body is trained according to the enhanced data set to obtain a target retriever.

[0107] It is worth noting that after completing the above architecture, in some embodiments of the present invention, in order to fully ensure the stability of the model parameters, the weights of the base searcher body, the first searcher body and the second searcher body can be uniformly distributed and randomly initialized, that is, , and For the weights of MoE, the parameters of each expert need to be randomly initialized and the output of the gating network is normalized to further obtain the final architecture.

[0108] Next is the process of training the final retriever architecture obtained by optimizing the second retriever. First, it is necessary to obtain a data set for training, that is, an enhanced data set. Specifically, this embodiment inputs the pre-built power operation and maintenance query sample into the full-parameter fine-tuning large model, and performs pre-defined query operations through the full-parameter fine-tuning large model to generate initial enhanced data, integrate the initial enhanced data and classify and annotate it to obtain the enhanced data set.

[0109] As an example, the present invention provides an example of generating an enhanced data set, including the following steps:

[0110] 1. Fine-tune the large model using all parameters Processing the predefined query “What is the operational process for power equipment fault diagnosis?”;

[0111] 2. Based on the ability to generate large models with full parameter fine-tuning, the query is automatically expanded to generate relevant training data, i.e., fault diagnosis steps and simulated maintenance records;

[0112] 3. Combine the generated text with the original power operation and maintenance documents to form recommended classification annotations and construct an enhanced data set, which is defined as .

[0113] The dataset will be enhanced Input into the second retriever body for training, and then the target retriever can be obtained. Specifically, first, according to the matching characteristics between the query text (i.e., the query question input by the user) and the related text, the positive and negative sample pairs in the enhanced data set are defined. It can be understood that the positive sample pair is defined as "a high similarity match between the query and the related text"; and the negative sample pair is "a low similarity match between the query and the irrelevant text".

[0114] Next, supervised learning is performed, and under the preset loss conditions of the positive and negative samples, the second retriever body is trained by the optimizer to obtain the target retriever. It can be understood that the application process of the operation and maintenance large model that integrates the target retriever and the full parameter fine-tuning large model fully embodies the dual-stage retrieval enhancement generation mechanism proposed by the present invention.

[0115] In some embodiments of the present invention, positive and negative samples are used for training; (a loss function commonly used in ranking tasks) as the standard, and the optimization objectives are "minimizing the loss of positive sample pairs" and "maximizing the loss of negative sample pairs" to improve the accuracy of retrieval.

[0116] Specifically, the loss function It can be expressed by the following formula:

[0117]

[0118] in, is the score of the positive sample; is the score of the negative sample; is a hyperparameter used to control the gap between positive and negative samples; , It is an augmented dataset The number of samples in .

[0119] Exemplarily, the optimizer may be selected as AdamW, which is a variant of the Adam optimizer and can be used for training deep learning models.

[0120] According to the above retrieval optimization process, after evaluating the final retriever, it is confirmed that the training of the retriever is completed, and the target retriever is constructed.

[0121] S5~S6, using collaborative filtering algorithm to integrate the target retriever and the full-parameter fine-tuning large model to obtain a target operation and maintenance large model, and in the power operation and maintenance process of the target power grid, the operation and maintenance results output by the target operation and maintenance large model are used to execute matching operation and maintenance strategies.

[0122] After obtaining the improved full-parameter fine-tuning large model and the target retriever, the embodiment of the present invention first needs to integrate the two models in order to facilitate the subsequent intelligent operation and maintenance process. As an example, the two models can be integrated by a joint means such as a collaborative filtering algorithm.

[0123] After integrating the target operation and maintenance model, the next step is the final model application process and subsequent strategy formulation process, both of which are targeted at the power operation and maintenance field. For details, please refer to Figure 3 , Figure 3FIG. 1 is a flow chart of a two-stage retrieval enhancement generation mechanism in one embodiment of the present invention to complete large model question answering. As an example, from Figure 3 It can be seen that three processes from left to right are involved. First, based on the target operation and maintenance big model, when the professional operation and maintenance engineer asks questions to the full-parameter fine-tuning big model constructed by the embodiment of the present invention, the target retriever will be used to retrieve the most relevant content from the power operation and maintenance knowledge base. During the retrieval process, the embodiment of the present invention constructs a power operation and maintenance knowledge base by using a text classifier, performs self-learning knowledge updates, extracts knowledge from the generated answers and dynamically updates the knowledge base to ensure the professionalism, accuracy and timeliness of the content, and provide more intelligent and efficient decision support. Finally, according to the results output by the big model, targeted maintenance, inspection and fault handling operations are performed on the power equipment in the power operation and maintenance operating system.

[0124] During the power operation and maintenance process of the target power grid, in response to the power operation and maintenance question and answer instructions input from the outside, the target operation and maintenance large model is controlled to output the operation and maintenance results composed of power operation and maintenance question and answer knowledge, that is, to output relevant power operation and maintenance knowledge responses.

[0125] Based on the above process, the premise of using the target operation and maintenance big model to output accurate results is to build a knowledge base that can be used for query and retrieval. Specifically, various types of power operation and maintenance knowledge sets are collected, including objective knowledge information based on power operation and maintenance, historical case information, operation requirement information, and national standard information, which can cover various operation requirements and compliance requirements.

[0126] In some embodiments of the present invention, the power operation and maintenance knowledge set needs to be structured to generate a standardized knowledge base, which is defined as .in, Each piece of data contains a piece of knowledge , and label its content for subsequent retrieval. The knowledge base structure can be expressed as:

[0127]

[0128] in, Represents a standardized knowledge entry, including metadata such as text, source, and tags.

[0129] The knowledge base can be cleaned, redundant or repeated information can be removed, and finally the power operation and maintenance standard knowledge base can be obtained.

[0130] After the knowledge base is constructed, the query question, i.e., the power operation and maintenance question and answer instruction, is input into the full-parameter fine-tuning large model, and the full-parameter fine-tuning large model will generate a corresponding query vector.

[0131] Next, in the target retriever retrieval link in the target operation and maintenance big model, the query vector is used to retrieve knowledge items from the power operation and maintenance standard knowledge base to obtain a number of recommended knowledge sets. At that time, the target operation and maintenance big model will feed back the several recommended knowledge sets into the full-parameter fine-tuning big model, thereby generating an operation and maintenance result that meets the conditions and executing an operation and maintenance strategy that matches the operation and maintenance result.

[0132] The present invention will provide a specific embodiment below to describe the above steps in detail:

[0133] Specifically, please refer to Figure 4 As shown, Figure 4 Shown is a schematic diagram of a power equipment maintenance process in one embodiment of the present invention.

[0134] It can be seen that this embodiment involves a power equipment maintenance scenario, and is also suitable for implementing professional operation and maintenance knowledge question and answer services, helping engineers improve work efficiency and form a good operation and maintenance work model.

[0135] like Figure 4 As shown, the scenario of this embodiment sets up four entities, namely, the power operation and maintenance data center, the power operation and maintenance information center, the power information integrated system and the professional engineer group. Among them, the power operation and maintenance data center is responsible for collecting professional power data, performing data processing and providing corpus for the training of the power operation and maintenance expert big model, including professional data such as power operation and maintenance data records, power system operation manuals and power operation and maintenance fault history; the power operation and maintenance information center is responsible for using the above-mentioned full parameter fine-tuning and retrieval enhancement technology of the present invention to train and maintain the power operation and maintenance expert big model, and use the constructed expert big model to provide power operation and maintenance work services and power professional knowledge question and answer services for the expert engineer group, assisting the professional engineer group to perform power engineering operation and maintenance work, power system infrastructure construction and power talent training / data upgrade; the power information integrated system includes "power system monitoring service", "power communication system", "power supply system", "power electronic database" and other contents, which are the operation and maintenance and service goals of the professional engineer group, that is, the professional engineer group will formulate and implement effective maintenance strategies for the power information integrated system with the assistance of the expert big model constructed by the embodiment of the present invention.

[0136] The specific steps include:

[0137] 1. Data collection and organization

[0138] With the support of the power operation and maintenance data center, power professional data is collected, including power operation and maintenance data records, power system operation manuals, power operation and maintenance fault history, equipment monitoring data, etc. The data is sorted and archived through the power operation and maintenance information center to ensure the integrity and high quality of the data. This embodiment scenario sets up the collection and sorting tasks of different types of data, as shown in the following table. It can be seen that it covers question and answer data, system identification data, professional knowledge base and operation manual data.

[0139]

[0140] 2: Construct optimized training data

[0141] Building a training set is a very important part. Through data cleaning, high-quality data after information extraction can facilitate the fine-tuning training process of the general large model. The data training center will use the Json data format to unify the data structure. This embodiment uses the data construction process of service feedback as the standard.

[0142] In this example, the data involves the management of performance bond and insurance / guarantee expiration warning. In the performance bond configuration section, the system provides two operations: save and clear. The save operation allows the user to set the performance bond ratio (bond_ratio) and save the setting; the clear operation restores the performance bond to the default value (ratio of 1). In the bond / insurance expiration warning configuration section, the system allows the user to enable or disable the warning function. When enabled, the user can set the number of warning days (warning_days), and the system will issue a warning and alarm when the bond or insurance is about to expire. The disable operation clears the warning parameters and related instructions.

[0143] 3: Full parameter fine-tuning FFT

[0144] After data preparation is completed, high-quality data sets are used to perform FFT fine-tuning of all parameters on the general large model to transform the model into an expert model in the field of power operation and maintenance.

[0145] 4: Deploy and apply the PoD-RAG method

[0146] Through the PoD-RAG two-stage retrieval enhancement mechanism, the fine-tuning model can obtain and update the latest information in the power operation and maintenance knowledge base in real time, thereby providing more specialized answers that meet actual needs.

[0147] 5: Generate and output answers and formulate relevant maintenance strategies

[0148] After completing the search enhancement, the system uses the contextualized knowledge base content as a reference for the generation model, generating answers that meet the professional standards of power operation and maintenance. Based on these answers, corresponding maintenance strategies can be formulated, such as answers and maintenance strategies for equipment failure warnings:

[0149] When the answer is: "The system detects that the oil temperature of a certain transformer has risen abnormally, and there may be a risk of overheating."

[0150] The maintenance strategy can be: take appropriate measures according to the cause of the fault, such as cleaning the radiator, replacing the cooling oil, etc. Or strengthen the monitoring of the transformer to ensure that the oil temperature returns to normal and remains stable. Or, if it is confirmed that it is an overload problem, use infrared thermal imagers and other equipment to conduct a comprehensive inspection of the transformer, confirm the fault point, and adjust the load distribution in time to ensure that the transformer operates within the rated load range.

[0151] In summary, the embodiment of the present invention utilizes a two-stage retrieval mechanism in the process of training an expert large model for power operation and maintenance data, that is, constructing a new retrieval architecture of the Mamba architecture-Kolmogorov-Arnold network (KAN), and by freezing some model parameters, it achieves efficient fine-tuning of the model, reduces computational overhead, and optimizes the efficiency of large model retrieval; and introduces a gated expert integration strategy in the architecture, combined with multiple different retrieval architectures, which can comprehensively deal with various problems in the field of power operation and maintenance and improve the reliability of model output. The large operation and maintenance model constructed by the present invention can be targeted based on the generated operation and maintenance knowledge, so that relevant personnel can formulate accurate maintenance strategies.

[0152] Another embodiment of the present invention provides a power system operation and maintenance system based on retrieval enhancement technology. For details, see Figure 5 , Figure 5 The structure diagram of the power system operation and maintenance system based on the retrieval enhancement technology in one embodiment of the present invention is shown, including:

[0153] The full-parameter fine-tuning large model training module M1 is used to perform weight analysis on each layer of the preset large model during the training of the preset large model based on the acquired power operation and maintenance training sample data, and dynamically freeze the layers in the preset large model that meet the preset conditions according to the weight analysis results to obtain a trained full-parameter fine-tuning large model;

[0154] The first-level search optimization module M2 is used to construct a base searcher body based on the state space model of the Mamba architecture, and introduce the Kolmogorov-Arnold network to optimize the parameters of each level in the base searcher body to obtain the first searcher body;

[0155] A secondary search optimization module M3, used for performing integrated optimization on the first search engine body through a gated expert integrated network to obtain a second search engine body;

[0156] A retriever training module M4 is used to construct an enhanced data set based on the output of the large model with all-parameter fine-tuning during the retriever training process, and train the second retriever body according to the enhanced data set to obtain a target retriever;

[0157] The model integration module M5 is used to integrate the target retriever and the full parameter fine-tuning large model using a collaborative filtering algorithm to obtain a target operation and maintenance large model;

[0158] The operation and maintenance strategy generation module M6 is used to execute a matching operation and maintenance strategy based on the operation and maintenance results output by the target operation and maintenance large model during the power operation and maintenance process of the target power grid.

[0159] The technical features and technical effects of the power system operation and maintenance system based on retrieval enhancement technology proposed in the embodiment of the present invention are the same as the technical features and technical effects of the power system operation and maintenance method based on retrieval enhancement technology proposed in the embodiment of the present invention, and will not be repeated here.

[0160] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A method for operation and maintenance of a power system based on retrieval enhancement technology, characterized in that: include: In the process of training the preset large model based on the acquired power operation and maintenance training sample data, weight analysis is performed on each layer of the preset large model, and the layers in the preset large model that meet the preset conditions are dynamically frozen according to the weight analysis results to obtain a trained full-parameter fine-tuning large model; The base retriever body is constructed based on the state space model of the Mamba architecture, and the Kolmogorov-Arnold network is introduced to optimize the parameters of each level in the base retriever body to obtain the first retriever body; Performing integrated optimization on the first retriever body through a gated expert integration network to obtain a second retriever body; During the retriever training process, an enhanced data set is constructed based on the output of the large model with all-parameter fine-tuning, and the second retriever body is trained according to the enhanced data set to obtain a target retriever; The target retriever and the full-parameter fine-tuning large model are integrated by using a collaborative filtering algorithm to obtain a target operation and maintenance large model; During the power operation and maintenance process of the target power grid, a matching operation and maintenance strategy is executed based on the operation and maintenance results output by the target operation and maintenance large model.

2. The power system operation and maintenance method based on retrieval enhancement technology according to claim 1, characterized in that: The weight analysis of each layer of the preset large model includes: According to the weights of each layer of the preset large model, the principal component analysis technology is used to decompose the eigenvalues ​​corresponding to each layer and generate the corresponding principal component matrix; Based on the principal component matrix, projecting the weights of each layer by a sparse sub-projection technique; In the projected space, the feature importance score of each layer is calculated based on the feature value.

3. The power system operation and maintenance method based on retrieval enhancement technology according to claim 2, characterized in that: According to the weight analysis result, the layers in the preset large model that meet the preset conditions are dynamically frozen to obtain a trained full-parameter fine-tuning large model, including: Analyze the importance of the features of each layer to determine the priority order, so as to sort the layers, freeze the layers that meet the preset conditions according to the sorting results and perform iterative updates; During each iteration, the frozen layers are re-determined based on the weighted average norm of all layers in the preset large model currently calculated, so as to train and obtain the full-parameter fine-tuning large model.

4. The power system operation and maintenance method based on retrieval enhancement technology according to claim 1, characterized in that: The state space model based on the Mamba architecture constructs a base retriever body, and introduces a Kolmogorov-Arnold network to optimize the parameters of each level in the base retriever body to obtain a first retriever body, including: Under the Mamba architecture, the weights of each base layer and the input power operation and maintenance training sample data are recursively applied layer by layer to form the base retriever body; the architecture of the base retriever body is represented by the following formula: in, is the weight matrix of the Mamba architecture layer; For the The input of the layer; It is a recursive calculation function layer by layer; The basic structure of the base search engine. The architecture of the first retriever body is represented by the following formula: in, is the weight of the Kolmogorov-Arnold network layer; An expression layer for nonlinearly reconstructing features of the Mamba architecture layer; This is the architecture of the first retriever body.

5. The power system operation and maintenance method based on retrieval enhancement technology according to claim 1, characterized in that: The process of acquiring the enhanced data set includes: Inputting the pre-built power operation and maintenance query sample into the full-parameter fine-tuning large model, and performing pre-defined query operations through the full-parameter fine-tuning large model to generate initial enhanced data; The initial enhanced data is integrated and classified and annotated to obtain the enhanced data set.

6. The power system operation and maintenance method based on retrieval enhancement technology according to claim 1, characterized in that: The step of training the second retriever body according to the enhanced data set to obtain a target retriever includes: Defining positive and negative sample pairs in the enhanced data set according to the matching characteristics between the query text and the related text; Under the preset loss conditions of the positive and negative samples, the second retriever body is trained by an optimizer to obtain the target retriever.

7. The power system operation and maintenance method based on retrieval enhancement technology according to claim 1, characterized in that: In the process of power operation and maintenance of the target power grid, the operation and maintenance results output by the target operation and maintenance large model are used to execute a matching operation and maintenance strategy, including: In response to external power operation and maintenance question and answer instructions, control the target operation and maintenance large model to output the operation and maintenance results composed of power operation and maintenance question and answer knowledge; Execute an operation and maintenance strategy that matches the operation and maintenance result.

8. The power system operation and maintenance method based on retrieval enhancement technology according to claim 7, characterized in that: The step of responding to the external power operation and maintenance question and answer instruction and controlling the target operation and maintenance big model to output the operation and maintenance result composed of power operation and maintenance question and answer knowledge includes: Inputting the power operation and maintenance question and answer instruction into the full parameter fine-tuning large model to generate a corresponding query vector; In the target retriever, the query vector is used to retrieve knowledge items from the power operation and maintenance standard knowledge base to obtain several recommended knowledge sets; The several recommended knowledge sets are fed back into the full-parameter fine-tuning large model to generate the operation and maintenance results that meet the conditions.

9. The power system operation and maintenance method based on retrieval enhancement technology according to claim 1, characterized in that: After the first retriever body is integrated and optimized by the gated expert integration network to obtain the second retriever body, the method further includes: The weights of the base retriever body, the first retriever body and the second retriever body are respectively initialized randomly with uniform distribution.

10. A power system operation and maintenance system based on retrieval enhancement technology, characterized in that: include: A full-parameter fine-tuning large model training module is used to perform weight analysis on each layer of the preset large model during the training of the preset large model based on the acquired power operation and maintenance training sample data, and dynamically freeze the layers in the preset large model that meet the preset conditions according to the weight analysis results to obtain a trained full-parameter fine-tuning large model; The first-level search optimization module is used to construct a base searcher body based on the state space model of the Mamba architecture, and introduce the Kolmogorov-Arnold network to optimize the parameters of each level in the base searcher body to obtain the first searcher body; A secondary search optimization module, used for integrating and optimizing the first search engine body through a gated expert integration network to obtain a second search engine body; A retriever training module, used to construct an enhanced data set based on the output of the large model with all-parameter fine-tuning during the retriever training process, and train the second retriever body according to the enhanced data set to obtain a target retriever; A model integration module, used to integrate the target retriever and the full-parameter fine-tuning large model using a collaborative filtering algorithm to obtain a target operation and maintenance large model; The operation and maintenance strategy generation module is used to execute a matching operation and maintenance strategy based on the operation and maintenance results output by the target operation and maintenance large model during the power operation and maintenance process of the target power grid.

Images