Multisource heterogeneous data power system fault diagnosis method and system based on reinforcement learning fine tuning

Abstract

The invention discloses a multi-source heterogeneous data power system fault diagnosis method and system based on reinforcement learning fine tuning, and the method comprises the steps: firstly carrying out the preprocessing of an original traveling wave signal, and completing the preliminary recognition of a fault type through a one-dimensional convolutional neural network; the recognition result is fused with multi-source context information such as fault positioning, real-time meteorological environment and line inspection state, and the information limitation of single traveling wave signal diagnosis is broken through; and finally, inputting a fusion result into a fine-tuned large language model or a multi-modal fusion neural network model to realize deep semantic analysis and association of multi-source information, and generating a structured fault analysis report. Wherein the pre-trained large language model adopts a LoRA technology to carry out lightweight parameter fine tuning, a reinforcement learning RLHF mechanism based on human feedback is combined to optimize LoRA adapter parameters, and report quality is evaluated through a lightweight reward model, so that the quality of a fault analysis report is improved, and a basis with high reference value is provided for rapid fault disposal and decision making of a power system.

Description

Technical Field

[0001] This invention belongs to the field of circuit fault analysis technology, and in particular relates to a method and system for fault diagnosis of multi-source heterogeneous data power systems based on reinforcement learning fine-tuning. Background Technology

[0002] In the field of power system operation and maintenance, rapid diagnosis and accurate handling of transmission line faults are crucial for ensuring the reliable operation of the power grid. Traditional fault diagnosis methods mainly rely on relay protection action information, human experience judgment, and fault identification and location techniques based on signal processing or machine learning (such as wavelet transform, support vector machine, convolutional neural network, etc.). Although these methods have achieved certain results in fault type identification and location estimation, they still have significant limitations: First, most methods can only achieve fault identification and fault location, lacking in-depth analysis and explanation of the fault causes; second, traditional expert systems or rule engines rely on manually constructed knowledge bases, resulting in poor generalization ability, high maintenance costs, and difficulty in adapting to complex and ever-changing field environments; third, existing methods often focus on a single data source (such as electrical measurement data), failing to fully integrate multi-source heterogeneous information such as meteorological and inspection data, leading to one-sided diagnostic results and insufficient decision support capabilities.

[0003] With the development of artificial intelligence technology, large language models have demonstrated powerful semantic understanding and generation capabilities in natural language processing tasks. In recent years, some studies have attempted to apply pre-trained language models such as BERT to power industry text data analysis, or have begun to explore the application of large models in scenarios such as power dispatch instruction generation and equipment defect description. However, these studies are still in their early stages, and problems exist such as models not being deeply adapted to the power industry context, weak multi-source information fusion capabilities, and limited operability of generated content. Furthermore, fine-tuning all parameters of large models requires extremely high computational resources and has high deployment costs, limiting their practical application in power industry scenarios with high real-time requirements and limited data scale. Therefore, there is an urgent need for an intelligent fault diagnosis method that can efficiently fuse multi-source monitoring data, possess deep domain knowledge understanding and generation capabilities, and simultaneously balance computational efficiency and deployment feasibility. Summary of the Invention

[0004] To achieve the aforementioned objectives, this invention provides a method and system for fault diagnosis of multi-source heterogeneous data in power systems based on reinforcement learning fine-tuning. The core of this method lies in constructing an end-to-end closed-loop system of signal perception, intelligent diagnosis, and decision generation. This system integrates the powerful sensing capabilities of convolutional neural networks (CNNs) for traveling wave signals with the deep semantic understanding and generation capabilities of large language models (LLMs), and employs efficient parameter fine-tuning techniques to ultimately output a fault analysis report that is both interpretable and operable.

[0005] Specifically, the present invention provides a method for fault diagnosis of multi-source heterogeneous data power systems based on reinforcement learning fine-tuning, the method comprising: Step 1: After preprocessing the acquired raw traveling wave signal, a one-dimensional convolutional neural network is used for preliminary fault type identification; Step 2: The preliminary identification results are fused with multi-source contextual information, which includes fault location information, real-time meteorological and environmental data, and line inspection status information. Step 3: Input the fusion results into the fine-tuned large language model or multimodal fusion neural network model to perform deep semantic analysis and association on the fused multi-source information, and generate the final structured report; The fine-tuned large language model uses LoRA technology to fine-tune the parameters of the selected pre-trained large language model, and introduces a reinforcement learning (RLHF) mechanism based on human feedback to optimize the trainable parameters in the LoRA adapter. At the same time, a lightweight reward model is constructed to score the quality of the fault analysis report generated by the model.

[0006] This invention achieves preliminary and accurate identification of power system fault types through a one-dimensional convolutional neural network. By fusing multi-source contextual information such as fault location, real-time meteorological environment, and line inspection status, it overcomes the information limitations of single traveling wave signal diagnosis, improving the information dimension and comprehensiveness of fault diagnosis. Simultaneously, it utilizes LoRA technology to achieve lightweight parameter fine-tuning of a pre-trained large language model, and combines a human feedback-based reinforcement learning (RLHF) mechanism with a lightweight reward model to optimize the trainable parameters of the adapter. This allows the model to perform deep semantic analysis and association on the fused multi-source heterogeneous data, significantly enhancing the semantic understanding and association capabilities of fault information, improving the structure and quality of fault analysis reports, and ultimately achieving intelligent, accurate, and efficient power system fault diagnosis, providing a more valuable reference for rapid handling and decision-making regarding power system faults.

[0007] Step 1 uses a one-dimensional convolutional neural network for preliminary fault type identification, including: The one-dimensional convolutional neural network consists of an input layer, multiple convolutional blocks, and an output layer. Each convolutional block includes a one-dimensional convolutional layer, a batch normalization layer, a ReLU activation function layer, and a max pooling layer. The convolutional layer automatically extracts local abrupt change features from the traveling wave signal through its sliding filter, and the pooling layer downsamples the feature map to enhance the robustness of the model and reduce computational complexity. The fully connected layer maps high-level abstract features to the classification space of fault categories, and the Softmax function outputs the probability distribution of fault types, thus completing the automatic identification and classification of typical faults such as single-phase grounding, two-phase short circuit, two-phase grounding, three-phase short circuit, and open circuit.

[0008] The fine-tuned large language model uses LoRA technology to fine-tune the parameters of the selected pre-trained large language model, including: For non-fault scenarios, layer sensitivity analysis is used to calculate the contribution weight of each layer to fault root cause extraction and maintenance suggestion generation. Key layers of the pre-trained model are dynamically selected, and the original weight matrices of the top 3 key layers with the highest contribution weights are frozen. The remaining layers remain in a fine-tunable state; Introducing low-rank matrices and ; When substituting the multi-source input x of the power system fault into the calculation, the output is obtained by superimposing the original weight result and the low-rank matrix increment result: ; in, Initialize from a standard normal distribution. Initialize from zero, hyperparameters It represents the low-rank dimension and controls the fine-tuning capacity and computational cost of the model.

[0009] The introduction of a reinforcement learning (RLHF) mechanism based on human feedback to optimize the trainable parameters in the LoRA adapter includes: constructing a power operation and maintenance-specific reward model, including: Based on the BERT model, the original classification output layer of BERT is removed and replaced with a linear scalar projection head. A fault level adaptation layer and a multi-dimensional scoring fusion layer are added. A compliance constraint term is introduced on the basis of the Bradley-Terry loss. A three-level feedback dataset is constructed for training the reward model. The three-level feedback dataset includes an expert feedback set, a procedure constraint feedback set, and a historical case feedback set. The optimization of trainable parameters in the LoRA adapter includes: optimizing the PPO algorithm using a proximal strategy to optimize the LoRA adapter parameters. The update will be performed as follows: For the fault prompt, the policy model Generate a fault analysis report y based on the current parameters, input (x,y) into the reward model to obtain the basic reward score; calculate the KL divergence between the output distributions of the strategy model and the reference model to obtain the penalty term. ; This is the penalty coefficient; Calculate the final reward: ; Using the PPO algorithm to evaluate LoRA adapter parameters Gradient updates are performed with the goal of maximizing the expected reward of all prompts within a batch. ; Introduce a comprehensive reward function as the learning objective of the reward model, or directly use it as the reward signal for the REINFORCE algorithm in the absence of a reward model; ; in arrive These are hyperparameters used to balance the weights of each item; .

[0010] The A trained probabilistic root cause classifier is introduced; the root cause system is reconstructed for the probabilistic root cause classifier, and a multi-dimensional label system of core root causes, sub-root causes, and related factors is established. The BERT joint attention mechanism is used to optimize the architecture of the probabilistic root cause classifier; the inferred cause category is determined by taking the generated text as input.

[0011] A dynamic thesaurus linking industry regulations and real-time cases is constructed and categorized by risk level and operational scenario. The BERT-NLI model is used to distinguish between citations of illegal statements and actual illegal statements. Dependency parsing and safety operation graphs are used to determine whether the logical order between steps is compliant.

[0012] Use regular expressions or a parser to check if the generated text contains clear action verbs, if it contains explicit location information, and if the suggestions are presented in list form.

[0013] Calculate the KL divergence between the output distribution of the text generated by the current reinforcement learning policy model and the output distribution of the reference model; use this divergence as a penalty term. .

[0014] The multimodal fusion neural network model further includes: using a convolutional neural network (CNN) branch to extract the spatiotemporal features of the traveling wave signal, while using a fully connected network branch to process the numerical meteorological data; using a recurrent neural network (RNN) or Transformer branch to encode the text inspection report; and finally, a classifier outputs the probability distribution of the root cause of the fault, and the most likely root cause is input into a pre-written expert rule engine to generate the final structured report.

[0015] Based on the same inventive concept, this invention also provides a multi-source heterogeneous data power system fault diagnosis system based on reinforcement learning fine-tuning, the system comprising: The preprocessing module is used to preprocess the acquired raw traveling wave signal; The first processing unit uses a one-dimensional convolutional neural network to perform preliminary fault type identification; The fusion unit integrates the preliminary identification results with multi-source contextual information; The second processing unit inputs the fusion result into a large language model or multimodal fusion neural network model that has been fine-tuned to perform deep semantic analysis and association on the fused multi-source information and generate a final structured report.

[0016] The second processing unit includes: The model optimization module uses LoRA technology to fine-tune the parameters of the selected pre-trained large language model, and introduces the reinforcement learning RLHF mechanism based on human feedback to optimize the trainable parameters in the LoRA adapter. The quality assessment module uses a lightweight reward model to score the quality of the fault analysis reports generated by the model.

[0017] Based on the same inventive concept, the present invention also provides a power grid fault diagnosis device, the device including a memory and a processor, the memory for storing a computer program, and the processor for running the computer program to enable the electronic device to perform the multi-source heterogeneous data power system fault diagnosis method based on reinforcement learning fine-tuning as described in the first aspect.

[0018] Compared with the prior art, the present invention has the following technical effects: The method proposed in this invention transforms the problem of integrating multi-source heterogeneous data into a natural language understanding problem, which is best suited for large language models. This greatly simplifies the complexity of data preprocessing and feature engineering, naturally establishing contextual semantic relationships between various types of information within the model. This enables the large language model to perform fault reasoning, much like an electrical expert, laying a solid foundation for subsequent in-depth analysis.

[0019] This invention further designs a reinforcement learning algorithm guided by a customized reward function, combining the PPO algorithm and LoRA technology. During fine-tuning, the model is explicitly trained to prioritize generating decision suggestions that simultaneously possess high accuracy, high security, and high operability. This achieves a leap from generating fluent text to making optimal decisions, ensuring that the model's output is no longer an empty language imitation, but a truly practical power operation and maintenance solution. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of a multi-source heterogeneous data power system fault diagnosis method based on reinforcement learning fine-tuning in this embodiment.

[0021] Figure 2 This is a diagram of a multi-source heterogeneous data power system fault diagnosis system based on reinforcement learning fine-tuning in this embodiment. Detailed Implementation

[0022] The following description is intended to disclose the invention and enable those skilled in the art to implement it. The preferred embodiments described below are merely examples, and other obvious variations will occur to those skilled in the art. Example 1

[0023] like Figure 1 As shown, the present invention provides a method for fault diagnosis of multi-source heterogeneous data power systems based on reinforcement learning fine-tuning, the method comprising: Step 1: After preprocessing the acquired raw traveling wave signal, a one-dimensional convolutional neural network is used for preliminary fault type identification.

[0024] In this embodiment, the preprocessing of the acquired raw traveling wave signal specifically includes: processing the raw traveling wave signal using a wavelet threshold denoising algorithm, decomposing the signal to 5 levels using the db4 wavelet basis function, and processing the high-frequency coefficients using a heuristic threshold function to suppress interference noise such as circuit breaker operation and load fluctuations, while retaining the high-frequency abrupt change components unique to fault traveling waves; subsequently, Z-score standardization is used to normalize the signal to an interval with a mean of 0 and a standard deviation of 1, eliminating data distribution differences caused by different line parameters and sampling frequencies, and providing a standardized data foundation for subsequent neural network input. This preprocessing process retains the high-frequency singularity features of the traveling wave signal while improving the model's anti-interference capability through mid-to-low frequency stabilization. In this embodiment, the one-dimensional convolutional neural network adopts a lightweight architecture consisting of an input layer, three concatenated convolutional blocks, a fully connected layer, and an output layer, specifically: Input layer: Receives preprocessed one-dimensional traveling wave signal time series data with input dimensions of (N, L), where N is the number of samples and L is the length of a single sample signal, ensuring complete coverage of the transient process of the fault traveling wave.

[0025] Convolutional blocks: One-dimensional convolutional layers: A dynamic multi-scale convolutional kernel group design is adopted. The first convolutional block uses 32 kernels of both 3×1 and 5×1 sizes; the second convolutional block uses 64 kernels of both 5×1 and 7×1 sizes; and the third convolutional block uses 128 kernels of both 7×1 and 9×1 sizes. A sliding filter scans the signal point-by-point. Small-sized convolutional kernels focus on capturing the steep wavefront abrupt changes in the early stages of a fault, while large-sized convolutional kernels focus on extracting the continuous oscillation features during the fault development process, achieving comprehensive coverage of local features of the traveling wave signal at different time scales. The convolution stride is uniformly set to 1, and the same padding method is used to ensure that the feature map size is consistent with the input, avoiding the loss of edge features.

[0026] Batch Normalization Layer: Following the convolutional layers, this layer normalizes the convolutional output feature maps by calculating the mean and variance of features within each batch and performing a linear transformation to normalize the feature values ​​to a stable range. This design effectively alleviates the internal covariate shift problem, keeps the input distribution of each network layer stable, allows for higher learning rates, and improves training convergence speed.

[0027] The ReLU activation function layer activates the batch-normalized feature map through a nonlinear transformation of f(x) = max(0,x). Compared to the traditional Sigmoid function, ReLU effectively avoids the vanishing gradient problem and introduces sparsity representation, allowing the network to focus more on key abrupt changes in the traveling wave signal and suppressing redundant information interference.

[0028] Max-pooling layer: An adaptive max-pooling strategy is adopted, and the pooling window size is dynamically calculated based on the length of the input feature map. The output feature map size of the first and second convolutional blocks is compressed to 1 / 2, and the size of the third convolutional block is compressed to 1 / 4. By selecting the maximum value within the window for downsampling, the computational complexity is significantly reduced while preserving the key peak information of the fault features, and the robustness of the model to small signal shifts is enhanced.

[0029] Output layer: First, a 256-dimensional fully connected layer maps the high-level abstract features output by the third convolutional block to the fault classification space, and a Dropout layer is introduced to prevent overfitting; then, a 5-dimensional fully connected layer is connected to the Softmax activation function to output the probability distribution of five typical faults: single-phase grounding, two-phase short circuit, two-phase grounding, three-phase short circuit, and open circuit. The category corresponding to the maximum probability is the preliminary identification result.

[0030] Step 2: The preliminary identification results are fused with multi-source contextual information, which includes fault location information, real-time meteorological and environmental data, and line inspection status information. It should be noted that it is difficult to directly integrate four types of heterogeneous data: preliminary fault identification results (probability vector), fault location information (spatial coordinates + topological relationship), real-time meteorological and environmental data (numerical time series), and line inspection status information (text + structured labels).

[0031] In this embodiment, targeted preprocessing is prioritized for different types of data. The preliminary fault identification results are transformed into dense feature vectors through a 128-dimensional fully connected embedding layer; the fault location information, combined with the power grid topology, is mapped to 64-dimensional topological features containing the correlation of adjacent lines using a graph embedding algorithm; real-time meteorological data, such as temperature, humidity, wind speed, thunderstorm level, and other eight indicators, are extracted for time trend features through one-dimensional temporal convolution (1×3 convolution kernel), outputting a 64-dimensional temporal vector; line inspection status information (such as aging of insulators on tower #123, and 5mm line icing thickness) is transformed into 128-dimensional text embedding vectors using a lightweight text encoder (DistilBERT), and structured labels (such as defect level and equipment type) are supplemented into 32-dimensional features through an embedding lookup table method, and the two are concatenated into 160-dimensional inspection features.

[0032] Furthermore, an Adaptive Batch Normalization (AdaBN) module is introduced to calibrate the distribution of embedded features for each modality. By calculating the mean and variance of different modal features on the training set, the distribution parameters of the current batch features are dynamically adjusted to ensure that cross-modal features remain consistent within the range of mean (±0.1) and variance (±0.05), thus solving the feature distribution shift problem caused by differences in data types.

[0033] Furthermore, a shared attention pooling layer is designed to uniformly map the aligned features from each modality (128-dimensional for fault identification, 64-dimensional for location, 64-dimensional for meteorology, and 160-dimensional for inspection) onto a 256-dimensional fusion feature space. A self-attention mechanism is used to calculate the correlation weights between features from different modalities, highlighting the strong correlations between fault types, location topology, and meteorological impacts, while suppressing redundant information. The final output is a 256-dimensional cross-modal fusion feature vector with unified dimensions and semantic relevance, thus solving the problem of heterogeneous data fusion that cross-modal embedding mechanisms cannot address.

[0034] Step 3: Input the fusion results into the fine-tuned large language model or multimodal fusion neural network model to perform deep semantic analysis and association on the fused multi-source information, and generate the final structured report.

[0035] In this embodiment, the fine-tuned large language model uses LoRA technology to fine-tune the parameters of the selected pre-trained large language model, and introduces a reinforcement learning RLHF mechanism based on human feedback to optimize the trainable parameters in the LoRA adapter. At the same time, a lightweight reward model is constructed to score the quality of the fault analysis report generated by the model.

[0036] Furthermore, the fine-tuned large language model is used to fine-tune the parameters of the selected pre-trained large language model using LoRA technology, including: For non-fault scenarios, layer sensitivity analysis is used to calculate the contribution weight of each layer to fault root cause extraction and maintenance suggestion generation. Key layers of the pre-trained model are dynamically selected, and the original weight matrices of the top 3 key layers with the highest contribution weights are frozen. The remaining layers remain in a fine-tunable state; Preferably, layer sensitivity analysis and dynamic selection of key layers specifically include: Evaluation dimensions are defined as follows: ① Contribution to root cause extraction (R layer): The ability of the layer to capture the correlation between the fault phenomenon and the root cause is measured by calculating the mutual information (MI) value between the output features of this layer and the root cause label. ② Contribution to operation and maintenance suggestion generation (O layer): The BLEU-4 and ROUGE-L indicators are used to evaluate the quality decay rate of the operation and maintenance suggestions generated by the model after removing this layer. The higher the decay rate, the greater the contribution.

[0037] Sensitivity calculation method: For the 32 Transformer layers of the pre-trained large language model (using Llama-2-7B), the scores of the R layer and O layer are calculated separately. The comprehensive sensitivity score of each layer is obtained by weighted summation (weight α=0.6, β=0.4, highlighting the priority of root cause extraction) S layer = α × R layer + β × O layer.

[0038] Key layer freezing strategy: For the top 3 key layers with the highest comprehensive sensitivity scores (experiments typically show layers 12, 18, and 24, corresponding to the high-level semantic feature extraction modules in the model), freeze their original weight matrix W0, and only connect the LoRA adapter to the remaining 29 layers. This design preserves the model's core semantic extraction capabilities while reserving parameter space for injecting fault-specific knowledge.

[0039] Furthermore, efficient parameter fine-tuning is achieved through low-rank decomposition, which reduces computational overhead while ensuring the model's expressive power.

[0040] Design a low-rank matrix pair for each fine-tunable layer, i.e., introduce a low-rank matrix. and Where d is the hidden layer dimension of the model (d=4096 in Llama-2-7B), k is the output dimension, and r is the low-rank dimension (hyperparameter, ranging from 8 to 64). Matrix A is initialized with a standard normal distribution N(0, 0.02) to ensure the stability of the initial incremental signal; matrix B is initialized with zero to make the model output in the early stage of fine-tuning close to the original pre-trained model, avoiding training instability caused by parameter mutations.

[0041] When substituting the multi-source input x of the power system fault into the calculation, the output is obtained by superimposing the original weight result and the low-rank matrix increment result: ; Where ΔW=BA is a low-rank increment matrix with a parameter quantity of r×(d+k).

[0042] It should be noted that the optimal r value is determined by grid search. When r=32, the model achieves the best balance between fault diagnosis accuracy and computational cost. When r<16, the model's expressive power is insufficient, and the root cause extraction accuracy drops to below 88%. When r>64, the number of parameters increases by 2%, but the performance improvement is less than 1%, resulting in diminishing marginal benefits.

[0043] The introduction of a reinforcement learning (RLHF) mechanism based on human feedback to optimize the trainable parameters in the LoRA adapter includes: constructing a power operation and maintenance-specific reward model, specifically including: Model structure modification: Based on the BERT model, the original classification output layer of BERT is removed and replaced with a linear scalar projection head. A fault level adaptation layer and a multi-dimensional scoring fusion layer are added. A compliance constraint term is introduced on the basis of Bradley-Terry loss. A three-level feedback dataset is constructed for training the reward model. The three-level feedback dataset includes an expert feedback set, a procedure constraint feedback set, and a historical case feedback set.

[0044] The preferred approach is to use expert feedback sets, which focus on the subjective professional judgment of experts. Fifteen senior power experts were selected, and 15,000 reports were generated based on 5,000 full-scenario fault prompts. 12,000 pairs of preference annotations were completed to learn the experts' preferences for report quality.

[0045] The preferred set of regulatory constraint feedback sets strengthens compliance scoring by reviewing 200 core clauses from 10 power regulations, collecting 800 violation reports and 1000 compliance reports, and having 3 regulatory experts complete a compliance score of 0-10, resulting in a total of 1800 data points.

[0046] The preferred approach is to use a historical case feedback set that aligns with actual application results. This set includes 3,000 complete fault handling cases from a provincial power grid between 2019 and 2024, and uses a validity score of 0-10 based on "ROUGE-L similarity (70%) + maintenance personnel evaluation (30%)".

[0047] Preferably, to address the differences in the severity of power faults (general, moderate, severe), fault level embedding features are introduced to enhance the model's adaptability to different fault scenarios. Specifically, this is implemented as follows: Power system faults are classified into three levels: Common faults: such as momentary single-phase grounding; More serious faults: such as permanent two-phase short circuits; Serious faults: such as three-phase short circuit tripping.

[0048] Each level is assigned a 64-dimensional trainable embedding vector E, and the embedding matrix is ​​initialized using a random normal distribution N.

[0049] Feature fusion method: The 256-dimensional intermediate features output by the linear scalar projection head are concatenated with the current fault level embedding vector e to obtain 320-dimensional fused features.

[0050] Adaptation layer mapping: A 128-dimensional fully connected layer is used to perform nonlinear mapping on the 320-dimensional fused features, which further compresses the feature dimensions and strengthens the correlation between fault level and report semantics.

[0051] An attention mechanism is used to adaptively fuse multi-dimensional scores and output the final reward score. Specific implementation details are as follows: Multi-dimensional scoring branches: Three parallel fully connected branches are designed to map the 128-dimensional features output by the fault level adaptation layer to obtain preliminary scores for each dimension: an accuracy branch (outputting an accuracy score of 0-1), a safety branch (outputting a safety score of 0-1), and an operability branch (outputting an operability score of 0-1). Attention weight calculation: A self-attention mechanism is introduced to calculate the importance weights of each dimension's score. First, the preliminary scores of the three dimensions are concatenated into a 3-dimensional vector, and the attention weight α is calculated using a single-layer perceptron. Final score output: The scores of each dimension are weighted and summed with the attention weights, then mapped to a reward score of 0-10.

[0052] Loss function optimization: The optimization of trainable parameters in the LoRA adapter includes: optimizing the PPO algorithm using a proximal strategy to optimize the LoRA adapter parameters. The update will be performed as follows: For the fault prompt, the policy model Generate a fault analysis report y based on the current parameters, input (x,y) into the reward model to obtain the basic reward score; calculate the KL divergence between the output distributions of the strategy model and the reference model to obtain the penalty term. ; This is the penalty coefficient; Calculate the final reward: ; Using the PPO algorithm to evaluate LoRA adapter parameters Gradient updates are performed with the goal of maximizing the expected reward of all prompts within a batch. ; Introduce a comprehensive reward function as the learning objective of the reward model, or directly use it as the reward signal for the REINFORCE algorithm in the absence of a reward model; ; in arrive These are hyperparameters used to balance the weights of each item; .

[0053] The A trained probabilistic root cause classifier is introduced; the root cause system is reconstructed for the probabilistic root cause classifier, and a multi-dimensional label system of core root causes, sub-root causes, and related factors is established. The BERT joint attention mechanism is used to optimize the architecture of the probabilistic root cause classifier; the inferred cause category is determined by taking the generated text as input. A dynamic thesaurus linking industry regulations and real-time cases is constructed and categorized by risk level and operational scenario; the BERT-NLI model is used to distinguish between quotations of illegal expressions and actual illegal expressions, and dependency parsing and safety operation graphs are used to determine whether the logical order between steps is compliant; Use regular expressions or a parser to check if the generated text contains clear action verbs, if it contains explicit location information, and if the suggestions are presented in list form. Calculate the KL divergence between the output distribution of the text generated by the current reinforcement learning policy model and the output distribution of the reference model; use this divergence as a penalty term. ; The multimodal fusion neural network model further includes: using a convolutional neural network (CNN) branch to extract the spatiotemporal features of the traveling wave signal, while using a fully connected network branch to process the numerical meteorological data; using a recurrent neural network (RNN) or Transformer branch to encode the text inspection report; and finally, a classifier outputs the probability distribution of the root cause of the fault, and the most likely root cause is input into a pre-written expert rule engine to generate the final structured report.

[0054] Example 2, as Figure 2 As shown, the present invention also provides a multi-source heterogeneous data power system fault diagnosis system based on reinforcement learning fine-tuning, the system comprising: The preprocessing module is used to preprocess the acquired raw traveling wave signal; The first processing unit uses a one-dimensional convolutional neural network to perform preliminary fault type identification; The fusion unit integrates the preliminary identification results with multi-source contextual information; The second processing unit inputs the fusion result into a large language model or multimodal fusion neural network model that has been fine-tuned to perform deep semantic analysis and association on the fused multi-source information and generate a final structured report.

[0055] Includes: the second processing unit, including: The model optimization module uses LoRA technology to fine-tune the parameters of the selected pre-trained large language model, and introduces the reinforcement learning RLHF mechanism based on human feedback to optimize the trainable parameters in the LoRA adapter. The quality assessment module uses a lightweight reward model to score the quality of the fault analysis reports generated by the model.

[0056] In embodiment three, the present invention also provides a power grid fault diagnosis device, the device including a memory and a processor, the memory for storing a computer program, and the processor for running the computer program to enable the electronic device to perform the multi-source heterogeneous data power system fault diagnosis method based on reinforcement learning fine-tuning as described in the first aspect.

[0057] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention. The scope of protection claimed by the appended claims and their equivalents is defined.

Claims

1. A multi-source heterogeneous data power system fault diagnosis method based on reinforcement learning fine-tuning, characterized in that, The method comprises: Step 1: After preprocessing the obtained original traveling wave signals, a one-dimensional convolutional neural network is used for preliminary identification of fault types; Step 2: The preliminary identification result is fused with multi-source context information, and the multi-source context information includes fault location information, real-time meteorological environment data and line inspection state information; Step 3: The fusion result is input into a large language model that has been fine-tuned or a multi-modal fusion neural network model to perform deep semantic analysis and correlation on the fused multi-source information, and generate a final structured report; The large language model that has been fine-tuned uses LoRA technology to fine-tune the selected pre-trained large language model, and introduces a reinforcement learning RLHF mechanism based on human feedback to optimize the trainable parameters in the LoRA adapter, while constructing a lightweight reward model to score the quality of the fault analysis report generated by the model.

2. The method of claim 1, wherein, The step 1 uses a one-dimensional convolutional neural network to preliminarily identify fault types, which comprises: The one-dimensional convolutional neural network is composed of an input layer, multiple convolutional blocks and an output layer; each convolutional block comprises a one-dimensional convolutional layer, a batch normalization layer, a ReLU activation function layer and a max pooling layer in sequence; wherein the convolutional layer automatically extracts local mutation features in the traveling wave signal through its sliding filter, and the pooling layer down-samples the feature map to enhance the robustness of the model and reduce the computational complexity; the high-level abstract features are mapped to the classification space of fault categories through the fully connected layer, and the probability distribution of fault types is output by the Softmax function, completing the automatic identification and classification of typical faults such as single-phase grounding, two-phase short circuit, two-phase grounding, three-phase short circuit and broken line.

3. The method of claim 2, wherein, The large language model that has been fine-tuned uses LoRA technology to fine-tune the selected pre-trained large language model, which comprises: For the non-fault scenario, the contribution weights of each layer to fault root cause extraction and operation and maintenance suggestion generation are calculated through layer sensitivity analysis, and the key layers of the pre-trained model are dynamically selected, and the original weight matrix of the key layers with the top 3 contribution weights is frozen , and the remaining layers remain in a fine-tunable state Introducing low-rank matrices and ; When the multi-source input x of the power system fault is substituted into the calculation, the output is obtained by superimposing the original weight result and the low-rank matrix increment result: ； where, initialized from the standard normal distribution, initialized from zero, hyperparameter denotes the low-rank dimension, controlling the fine-tuning capacity and computational overhead of the model.

4. The method of claim 3, wherein, The RLHF mechanism based on human feedback is introduced to optimize the trainable parameters in the LoRA adapter, which comprises: A power operation and maintenance exclusive reward model is constructed, which comprises:

5. The method of claim 4, wherein, The trainable parameters in the LoRA adapter are optimized, including: adopting a proximal policy optimization PPO algorithm to optimize the LoRA adapter parameters updating, specifically: For the failure Prompt x, the policy model According to the current parameters, a failure analysis report y is generated, and (x, y) is input into the reward model to obtain a basic reward score; the output distribution KL divergence of the policy model and the reference model is calculated to obtain a penalty term: ; is a penalty coefficient; Compute final reward: ; Adapting LoRA adapter parameters with PPO algorithm Gradient update with the goal of maximizing the expected reward for all prompts in the batch ; Based on the Bert model, the original classification output layer of Bert is removed and replaced with a linear scalar projection head, and a fault level adaptation layer and a multi-dimensional score fusion layer are added; a compliance constraint term is introduced based on the Bradley-Terry loss; a three-level feedback dataset is constructed for training the reward model; the three-level feedback dataset includes an expert feedback set, a regulation constraint feedback set and a historical case feedback set. ； wherein to is a hyperparameter balancing the weights of the terms; .

6. The method of claim 5, wherein, A comprehensive reward function is introduced as the learning goal of the reward model, or directly as the reward signal of the REINFORCE algorithm in the case of no reward model; The : introducing a trained probabilistic root cause classifier; reconstructing the root cause system for the probabilistic root cause classifier, establishing a multi-dimensional label system of core root causes, sub-root causes and associated factors, and optimizing the architecture of the probabilistic root cause classifier by using BERT joint attention mechanism; inputting the generated text y to determine the inferred reason category; A dynamic vocabulary is constructed by linking industry regulations and real-time cases, classified by risk level and operation scenario; the BERT-NLI model is used to distinguish between quoted violations and actual violations, and the dependency syntax analysis and safety operation map are used to determine whether the logical sequence between steps is compliant; : Check if the generated text contains clear action verbs using regular expressions or parsers, check if it contains explicit location information, check if the suggestions are presented in a list format; : compute the KL divergence between the output distribution of the current reinforcement learning policy model generating text and the reference model output distribution; use this as a penalty term: .

7. The method of claim 6, wherein, Further comprising The multi-modal fusion neural network model further comprises: The spatial and temporal features of the traveling wave signals are extracted using a convolutional neural network (CNN) branch, while the numerical weather data is processed using a fully connected network branch; a recurrent neural network or a Transformer branch is used to encode the text inspection reports; finally, a classifier outputs the probability distribution of the fault root cause, and the most likely root cause identified is input into a pre-written expert rule engine to generate a final structured report.

8. The system for power system fault diagnosis based on reinforcement learning fine-tuning of multi-source heterogeneous data according to any one of claims 1-7, wherein, The system comprises: a preprocessing module for preprocessing the acquired original traveling wave signals; a first processing unit for preliminary identification of fault types using a one-dimensional convolutional neural network; a fusion unit for fusing the preliminary identification results with multi-source context information; a second processing unit for inputting the fusion results into a large language model or a multi-modal fusion neural network model that has been fine-tuned to perform deep semantic analysis and correlation on the fused multi-source information and generate a final structured report.

9. The system of claim 8, wherein, The second processing unit comprises: a model optimization module for fine-tuning selected pre-trained large language models using LoRA technology and introducing a reinforcement learning based on human feedback (RLHF) mechanism to optimize the trainable parameters in the LoRA adapter; a quality evaluation module for scoring the quality of the fault analysis report generated by the model using a lightweight reward model. The device comprises a memory for storing a computer program and a processor for running the computer program to enable the electronic device to perform the method of claim 1-7.

10. A power system fault diagnostic device characterized by comprising: The device comprises a memory for storing a computer program and a processor for running the computer program to enable the electronic device to perform the method of claim 1-7.

Images