Railway power equipment fault prediction method based on deep reinforcement learning

Abstract

The invention discloses a railway power equipment fault prediction method based on deep reinforcement learning, and the method comprises the following steps: S1, collecting the operation data of railway power equipment, and carrying out the preprocessing of the operation data; s2, performing feature extraction by adopting an improved noise reduction sparse automatic encoder and a graph neural network; s3, constructing a deep reinforcement learning algorithm based on a diffusion model; s4, generating a strategy sample by adopting a diffusion process of a deep reinforcement learning algorithm based on a diffusion model, and training the deep reinforcement learning algorithm based on the diffusion model; s5, performing fault prediction by using the trained deep reinforcement learning algorithm based on the diffusion model; and S6, generating a predictive maintenance suggestion in combination with the prediction result, the fault maintenance record and the equipment life curve. According to the method, the deep reinforcement learning algorithm based on the diffusion model, the noise reduction sparse automatic encoder and the graph neural network are combined, railway power equipment fault prediction is realized, and the method has the advantages of high precision, low false alarm rate and adaptive optimization capability.

Description

Technical Field

[0001] The present invention relates to the field of railway power equipment fault prediction and intelligent maintenance, and in particular to a railway power equipment fault prediction method based on deep reinforcement learning. Background Art

[0002] Railway power equipment is an important part of the rail transit system. The stability of its operation directly affects the safety and reliability of railway transportation. With the improvement of the electrification level of railways, the complexity of power facilities such as traction power supply systems, signal power supply systems, and substation equipment continues to increase. The operating status of the equipment is affected by factors such as the external environment, load fluctuations, and equipment aging, and electrical failures are prone to occur. The current maintenance method of railway power equipment mainly adopts a model of regular inspections and manual experience judgment, relying on the professional knowledge of operation and maintenance personnel to perform troubleshooting and equipment replacement. However, this maintenance method has a large lag and cannot predict the health status of the equipment in advance, resulting in sudden failures, which in turn affect the safe operation of trains. In addition, the manual inspection method consumes a lot of human resources and is difficult to meet the intelligent operation and maintenance needs of large-scale railway power equipment.

[0003] In recent years, the development of intelligent monitoring technology has provided new ideas for the status assessment of railway power equipment. SCADA systems and PMUs and other equipment can collect key operating parameters such as voltage, current, power, temperature, frequency, vibration signals, etc., forming massive time series data. Based on these data, researchers have proposed a variety of data-driven fault prediction methods, such as statistical analysis-based methods, traditional machine learning-based methods, and deep learning-based methods. Statistical analysis methods rely on the historical distribution characteristics of data, such as using mean, variance, trend analysis, etc. to calculate outliers, but such methods are insensitive to nonlinear fault characteristics and cannot effectively identify complex fault modes. Traditional machine learning methods such as support vector machines and random forests can learn classification boundaries from data, but are susceptible to feature redundancy and feature interaction in high-dimensional nonlinear data, and the model generalization ability is weak. In addition, these methods usually require a lot of manual feature engineering and are difficult to adapt to the dynamically changing railway power equipment environment.

[0004] With the development of deep learning, researchers have used methods such as deep neural networks, convolutional neural networks, and long short-term memory networks to extract features and identify faults in the time series data of railway power equipment. Long short-term memory networks have strong time series modeling capabilities, can learn long-term dependencies, and improve the accuracy of fault detection. However, traditional deep learning methods mainly rely on static supervised learning, that is, fault detection is performed after training the model based on historical data. They lack online adaptive capabilities and cannot effectively respond to complex environmental changes and fault mode evolution. In addition, deep neural networks require a large amount of labeled data for training, and the fault data of railway power equipment is often unbalanced, resulting in data bias in the model training process, affecting the prediction effect.

[0005] In recent years, reinforcement learning has received widespread attention in the field of intelligent operation and maintenance. Reinforcement learning learns optimal strategies by interacting with the environment, and is suitable for dynamically adjusting prediction models to improve the adaptability of equipment fault prediction. Deep reinforcement learning combines the representation ability of deep learning with the decision-making ability of reinforcement learning, enabling the model to optimize prediction strategies under different operating conditions and improve the accuracy and stability of fault prediction. However, traditional deep reinforcement learning algorithms, such as deep Q networks and proximal strategy optimization, still have problems such as slow convergence, easy to fall into local optimality, and sensitivity to environmental changes, which limit their application in railway power equipment fault prediction.

[0006] In recent years, researchers have proposed reinforcement learning algorithms based on diffusion models to address the stability and generalization issues of reinforcement learning models. Diffusion models model the state space through random diffusion and denoising processes, which can generate high-quality policy samples and improve the policy optimization capabilities of reinforcement learning. Reinforcement learning based on diffusion models can effectively enhance the stability of fault prediction and enable the model to better adapt to changes in the operating status of railway power equipment. However, current research mainly focuses on decision optimization of reinforcement learning, while there are still many challenges in high-dimensional time series data feature extraction and topological modeling of railway power equipment.

[0007] Existing fault prediction methods generally have the following problems: First, traditional methods based on statistical analysis and machine learning have difficulty in effectively handling complex nonlinear fault modes, resulting in low fault prediction accuracy and an inability to meet the intelligent operation and maintenance needs of railway power equipment; Second, although deep learning-based methods can automatically extract features, their training methods mainly rely on static data and are difficult to adapt to the dynamic changes in equipment operating status; Third, although reinforcement learning methods can optimize fault prediction strategies, the strategy learning efficiency of traditional reinforcement learning algorithms is low and they are easily affected by environmental changes, making it difficult to generalize prediction strategies; In addition, existing methods often lack the ability to model the topological structure of railway power equipment, making it difficult to capture the correlation characteristics between devices, affecting the stability and reliability of the prediction. Summary of the Invention

[0008] One purpose of the present invention is to propose a railway power equipment fault prediction method based on deep reinforcement learning. The present invention integrates an improved denoising sparse autoencoder, a graph neural network and a reinforcement learning algorithm based on a diffusion model, uses an improved denoising sparse autoencoder for feature extraction, and combines the graph neural network to construct the topological relationship of railway power equipment, realizes the fusion of timing features and equipment topology information, improves the accuracy of fault prediction, uses a deep reinforcement learning algorithm based on a diffusion model to optimize the fault prediction strategy, generates strategy samples through a diffusion process, and dynamically adjusts the strategy parameters of the prediction model in combination with the reinforcement learning training process to improve the generalization ability of the model.

[0009] A railway power equipment fault prediction method based on deep reinforcement learning according to an embodiment of the present invention includes the following steps:

[0010] S1. Using the SCADA system, PMU device and intelligent sensors to collect operating data of railway power equipment, and pre-processing the operating data to generate standardized time series data;

[0011] S2. We use an improved denoising sparse autoencoder combined with a graph neural network to extract features from standardized time series data. We also analyze the topological relationships of railway power equipment to generate a comprehensive feature representation for fault prediction. Based on this comprehensive feature representation, we construct the state space for a deep reinforcement learning algorithm.

[0012] S3. Define the action space and construct the reward function. Combine the state space, action space, and reward function to build a deep reinforcement learning algorithm based on the diffusion model.

[0013] S4. Generate policy samples using the diffusion process of the deep reinforcement learning algorithm based on the diffusion model, and train the deep reinforcement learning algorithm based on the diffusion model based on the policy samples;

[0014] S5. Combined with real-time monitoring data, the trained diffusion model-based deep reinforcement learning algorithm is used to predict faults and derive the failure probability and health score of railway power equipment.

[0015] S6. Generate predictive maintenance recommendations based on the health score of railway power equipment, combined with fault repair records and equipment life curves.

[0016] Optionally, the operating data includes voltage, current, power, temperature, frequency and vibration signals.

[0017] Optionally, the preprocessing includes denoising, normalization, interpolation and completion, and time synchronization.

[0018] Optionally, the S2 specifically includes:

[0019] S21. Perform window processing on the standardized time series data and set the time window length to T w , construct the time series matrix:

[0020]

[0021] Among them, X represents the time series matrix, X t represents the standardized time series data at time t, X t+1 Represents the standardized time series data at time t+1, Represents time t+T w Normalized time series data at time T w Indicates the length of the time window;

[0022] S22. Perform feature extraction on the time series matrix X using an improved denoising sparse autoencoder, wherein the improved denoising sparse autoencoder is formed by stacking denoising sparse autoencoders, and the denoising sparse autoencoder is composed of an encoding layer, a hidden layer, and a decoding layer;

[0023] S23. Use the nonlinear transformation principal component analysis method to reduce the dimension of the feature representation extracted by the improved denoising sparse autoencoder and calculate the covariance matrix:

[0024]

[0025] Among them, C represents the covariance matrix, describing the nonlinear principal component, ReLU represents the rectified linear unit activation function, m represents the total number of samples, h i represents the hidden layer feature representation of the i-th sample, μ represents the feature mean vector, and T represents the transposition operation;

[0026] S24. Perform eigenvalue decomposition on the covariance matrix C to obtain the feature representation after dimensionality reduction:

[0027] h'=V T h+ξ·ReLU(W c h);

[0028] Among them, h' represents the feature representation after dimensionality reduction, V represents the principal component matrix after eigenvalue decomposition, and W c represents the adjustment matrix in the dimensionality reduction process, ξ represents the dynamic adjustment coefficient;

[0029] S25. Use graph neural network to model the topological relationship of the feature representation h' after dimensionality reduction, and construct the topological graph G = (V, E) of railway power equipment, where V represents the set of equipment nodes and E represents the connection relationship between equipment. For each node v in the topological graph, update the feature representation

[0030]

[0031] in, represents the feature representation of node v in layer l, σ represents the nonlinear activation function, W g represents the weight matrix of the graph neural network, b g Represents the bias term of the graph neural network, N(v) represents the set of neighbor nodes of node v, and A vu represents the adjacency matrix weight between nodes v and u, represents the feature representation of node v at the l-1 layer, |N(v)| represents the number of neighbors of node v, describing the degree of node v;

[0032] S26. Dimensionally match the feature representation output by the graph neural network with the feature representation output by the improved denoising sparse autoencoder, and perform feature fusion to construct a comprehensive feature representation for fault prediction:

[0033] Z=αh'+(1-α)F GNN +ζ·Softmax(W f X);

[0034] Where Z represents the comprehensive feature representation of fault prediction, α represents the weight factor, h' represents the feature representation after dimensionality reduction output by the improved denoising sparse autoencoder, and F GNN represents the feature representation of the graph neural network output, ζ represents the fusion enhancement coefficient, Softmax represents normalization, and W f represents the fusion adjustment matrix, X represents the time series matrix;

[0035] S27. Construct a state space based on the comprehensive feature representation.

[0036] Optionally, the S22 specifically includes:

[0037] S221. Add Gaussian noise to the time series matrix X and use the adaptive dynamic mask noise reduction method to generate a noise data matrix:

[0038]

[0039] in, represents the noise data matrix, X represents the time series matrix, ⊙ represents the element-by-element multiplication operation, N represents the Gaussian distribution, σ 2 represents the variance of the noise, and M represents the dynamic mask matrix, which is derived from the attention weight:

[0040] M t =1-exp(-βA t );

[0041] Among them, M t represents the dynamic mask matrix, exp represents the natural exponential function, β represents the dynamic adjustment factor, A t represents the feature importance weight at time step t;

[0042] S222, the noise data Mapped to the hidden layer through the encoder, calculate the hidden layer feature representation h:

[0043]

[0044] Among them, h represents the hidden layer feature representation, σ represents the nonlinear activation function, and W e represents the encoder weight matrix, b e represents the encoder bias term, λ represents the adaptive adjustment factor, tanh represents the hyperbolic tangent function, W f represents the additional weight matrix of the fully connected layer, introducing residual information, and X represents the time series matrix;

[0045] S223. Apply dynamic sparsity constraints to the hidden layer and calculate the regularization term:

[0046]

[0047] in, represents the sparsity regularization loss, ρ represents the target sparsity, represents the average activation of the jth neuron in the hidden layer, γ represents the L1 constraint coefficient, which is used to enhance sparsity, h represents the hidden layer feature representation, and ||h||1 represents the L1 norm of the hidden layer feature representation;

[0048] S224: pass the hidden layer feature representation h to the decoder, reconstruct the input data, compare the obtained reconstructed data X' with the original input X, calculate the reconstruction loss, optimize the parameters of the improved denoising sparse autoencoder, and then optimize the hidden layer feature representation. The reconstruction loss adopts the cross entropy loss:

[0049] X'=σ(W d h+b d +η·W r X);

[0050] Among them, X' represents the reconstructed data, describing the decoded data matrix, σ represents the nonlinear activation function, ReLU represents the nonlinear activation function, W d and b d Denote the decoder weight matrix and bias term respectively, η represents the training coefficient, which is used to introduce the residual information of the original input to enhance the reconstruction ability, and W r represents the additional recovery matrix.

[0051] Optionally, the actions in the action space include updating fault prediction strategies, adjusting alarm thresholds, triggering alarm mechanisms, optimizing equipment operating parameters, optimizing inspection plans, and maintaining current status, and the reward function is based on prediction accuracy, false alarm rate, and early warning capability.

[0052] Optionally, the S4 specifically includes:

[0053] S41, using random diffusion model for state transfer, constructing state vector Z t The diffusion process:

[0054]

[0055] Among them, Z t+1 Represents the state vector at time t+1, Z t represents the state vector at time t, δ(Z t ,t) represents the drift term of the state vector, describing the changing trend of the fault characteristics, Δt represents the time step, which is used to control the scale of state change, σ(Z t ,t) represents the diffusion coefficient, describing the uncertainty of the state, ∈ represents the standard normal distribution noise;

[0056] S42, the state vector Z after diffusion t+1 Perform reverse denoising and use the time reversal method to obtain the optimized state vector:

[0057]

[0058] Among them, Z' t represents the optimized state vector, Z t+1 Represents the state vector at time t+1;

[0059] S43, combined with the optimized state vector Z' t , calculate the state probability density:

[0060]

[0061] Among them, p(Z' t ) represents the state probability density, d represents the dimension of the state vector, Σ represents the covariance matrix of the state vector, |Σ| represents the determinant value of the covariance matrix of the state vector, exp represents the natural exponential function, Z' t represents the state vector after denoising, (Z' t -μ) T Σ -1 (Z' t -μ) represents the Mahalanobis distance, μ represents the mean vector of the state vector, and -1 represents the "inverse" operation of the matrix;

[0062] S44, based on the state probability density p(Z' t ), perform weighted correction on the optimal action:

[0063]

[0064] in, represents the optimized action, τ represents the balance parameter, and controls the influence of the state probability density on the optimal action. t ,A t ) represents the state vector Z' t Next, perform action A t The corresponding Q value, argmax represents the variable value when the function value is maximum;

[0065] S45, based on the optimization strategy sample Train the deep reinforcement learning algorithm, calculate the target Q value, construct the loss function, and update the parameters through gradient descent:

[0066]

[0067]

[0068] Among them, Q target (Z t ,A t ) represents the target Q value, which is used to train the deep reinforcement learning algorithm, R t Represents the immediate reward, which measures the action A taken at the current time step t t The benefit of ,γ represents the discount factor, which controls the impact of future rewards, Represents the state vector Z' after optimization t Execute the optimization strategy sample Q value, Represents the loss function, describing the mean square error loss of the Q value, Q θ (Z t ,A t ) represents the Q-value function with parameter θ;

[0069] S46. Train the policy network of the deep reinforcement learning algorithm and use the policy gradient optimization method to update the parameters of the policy network:

[0070]

[0071] in, represents the gradient of the policy objective function, J(θ) represents the policy objective function, describing the maximization of long-term rewards, E represents the mathematical expectation, π θ (A t |Z t ) represents the policy function.

[0072] Optionally, the failure probability and health score are:

[0073] P f (Z t )=sigmoid(W f Z t +b f );

[0074] H t =1-P f (Z t );

[0075] Among them, P f (Z t ) represents the railway power equipment in the state vector Z t The failure probability under , sigmoid represents the nonlinear activation function, W f represents the fault prediction weight matrix, b f represents the corresponding bias term, H t Indicates the device health status score.

[0076] The beneficial effects of the present invention are:

[0077] First, by introducing an improved denoising sparse autoencoder and graph neural network for feature extraction and topological modeling, the robustness of fault features and the expressiveness of topological information are enhanced, which can more accurately characterize the operating status of railway power equipment and effectively identify complex fault modes. At the same time, a deep reinforcement learning algorithm optimized by a diffusion model is used to construct an adaptive fault prediction strategy. High-quality strategy samples are generated through the diffusion process, and the strategy parameters are dynamically adjusted in combination with the reinforcement learning training process, thereby improving the adaptability of the model in different operating environments.

[0078] Secondly, the present invention collects multimodal time series data and performs denoising, normalization, interpolation and time synchronization processing to ensure the integrity and consistency of the data, thereby improving the quality of the input data and making fault prediction more accurate. The time series features extracted by the improved denoising sparse autoencoder and the graph neural network are combined for topological structure modeling, which further enhances the feature fusion capability. It not only focuses on the operating status of a single device, but also considers the topological association between devices, thereby more effectively identifying potential fault propagation paths and improving the accuracy and interpretability of fault prediction. In addition, a deep reinforcement learning algorithm based on the diffusion model is adopted, and the diffusion process is used to simulate the evolution of the health status of the equipment. The inversion denoising method is combined to optimize the state estimation, ensuring that the reinforcement learning model converges faster and has higher stability during the training process, thereby optimizing the fault prediction strategy.

[0079] Finally, the present invention not only improves the accuracy of fault prediction, but also reduces the false alarm rate and delayed warning problems. It can identify potential equipment failures in advance and give warnings or maintenance suggestions at the optimal time point, thereby effectively reducing the impact of sudden equipment failures on railway operation safety. At the same time, the method of the present invention has good scalability and can be applied to different types of railway power equipment. It can also continuously optimize the prediction model as the data grows, thereby improving its adaptability in actual application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0081] Figure 1 This is a flowchart of a railway power equipment fault prediction method based on deep reinforcement learning proposed by the present invention;

[0082] Figure 2 This is a schematic diagram of a railway power equipment fault prediction method based on deep reinforcement learning proposed by the present invention, which generates strategy samples based on a diffusion model and optimizes the fault prediction strategy. DETAILED DESCRIPTION

[0083] The present invention will now be described in further detail with reference to the accompanying drawings, which are simplified schematic diagrams that illustrate the basic structure of the present invention in a schematic manner.

[0084] refer to Figure 1 and Figure 2 , a railway power equipment fault prediction method based on deep reinforcement learning, comprising the following steps:

[0085] S1. Using the SCADA system, PMU device and intelligent sensors to collect operating data of railway power equipment, and pre-processing the operating data to generate standardized time series data;

[0086] S2. We use an improved denoising sparse autoencoder combined with a graph neural network to extract features from standardized time series data. We also analyze the topological relationships of railway power equipment to generate a comprehensive feature representation for fault prediction. Based on this comprehensive feature representation, we construct the state space for a deep reinforcement learning algorithm.

[0087] S3. Define the action space and construct the reward function. Combine the state space, action space, and reward function to build a deep reinforcement learning algorithm based on the diffusion model.

[0088] S4. Generate policy samples using the diffusion process of the deep reinforcement learning algorithm based on the diffusion model, and train the deep reinforcement learning algorithm based on the diffusion model based on the policy samples;

[0089] S5. Combined with real-time monitoring data, the trained diffusion model-based deep reinforcement learning algorithm is used to predict faults and derive the failure probability and health score of railway power equipment.

[0090] S6. Generate predictive maintenance recommendations based on the health score of railway power equipment, combined with fault repair records and equipment life curves.

[0091] In this embodiment, the operating data includes voltage, current, power, temperature, frequency and vibration signals.

[0092] In this embodiment, the preprocessing includes denoising, normalization, interpolation and completion, and time synchronization.

[0093] In this embodiment, S2 specifically includes:

[0094] S21. Perform window processing on the standardized time series data and set the time window length to T w , construct the time series matrix:

[0095]

[0096] Among them, X represents the time series matrix, X t represents the standardized time series data at time t, X t+1 Represents the standardized time series data at time t+1, Represents time t+T w Normalized time series data at time T w Indicates the length of the time window;

[0097] S22. Perform feature extraction on the time series matrix X using an improved denoising sparse autoencoder, wherein the improved denoising sparse autoencoder is formed by stacking denoising sparse autoencoders, and the denoising sparse autoencoder is composed of an encoding layer, a hidden layer, and a decoding layer;

[0098] S23. Use the nonlinear transformation principal component analysis method to reduce the dimension of the feature representation extracted by the improved denoising sparse autoencoder and calculate the covariance matrix:

[0099]

[0100] Among them, C represents the covariance matrix, describing the nonlinear principal component, ReLU represents the rectified linear unit activation function, m represents the total number of samples, h irepresents the hidden layer feature representation of the i-th sample, μ represents the feature mean vector, and T represents the transposition operation;

[0101] S24. Perform eigenvalue decomposition on the covariance matrix C to obtain the feature representation after dimensionality reduction:

[0102] h'=V T h+ξ·ReLU(W c h);

[0103] Among them, h' represents the feature representation after dimensionality reduction, V represents the principal component matrix after eigenvalue decomposition, and W c represents the adjustment matrix in the dimensionality reduction process, ξ represents the dynamic adjustment coefficient;

[0104] S25. Use graph neural network to model the topological relationship of the feature representation h' after dimensionality reduction, and construct the topological graph G = (V, E) of railway power equipment, where V represents the set of equipment nodes and E represents the connection relationship between equipment. For each node v in the topological graph, update the feature representation

[0105]

[0106] in, represents the feature representation of node v in layer l, σ represents the nonlinear activation function, W g represents the weight matrix of the graph neural network, b g Represents the bias term of the graph neural network, N(v) represents the set of neighbor nodes of node v, and A vu represents the adjacency matrix weight between nodes v and u, represents the feature representation of node v at the l-1 layer, |N(v)| represents the number of neighbors of node v, describing the degree of node v;

[0107] S26. Dimensionally match the feature representation output by the graph neural network with the feature representation output by the improved denoising sparse autoencoder, and perform feature fusion to construct a comprehensive feature representation for fault prediction:

[0108] Z=αh'+(1-α)F GNN +ζ·Softmax(W f X);

[0109] Where Z represents the comprehensive feature representation of fault prediction, α represents the weight factor, h' represents the feature representation after dimensionality reduction output by the improved denoising sparse autoencoder, and F GNN represents the feature representation of the graph neural network output, ζ represents the fusion enhancement coefficient, Softmax represents normalization, and W f represents the fusion adjustment matrix, X represents the time series matrix;

[0110] S27. Construct a state space based on the comprehensive feature representation.

[0111] In this embodiment, the S22 specifically includes:

[0112] S221. Add Gaussian noise to the time series matrix X and use the adaptive dynamic mask noise reduction method to generate a noise data matrix:

[0113]

[0114] in, represents the noise data matrix, X represents the time series matrix, ⊙ represents the element-by-element multiplication operation, N represents the Gaussian distribution, σ 2 represents the variance of the noise, and M represents the dynamic mask matrix, which is derived from the attention weight:

[0115] M t =1-exp(-βA t );

[0116] Among them, M t represents the dynamic mask matrix, exp represents the natural exponential function, β represents the dynamic adjustment factor, A t represents the feature importance weight at time step t;

[0117] S222, the noise data Mapped to the hidden layer through the encoder, calculate the hidden layer feature representation h:

[0118]

[0119] Among them, h represents the hidden layer feature representation, σ represents the nonlinear activation function, and W e represents the encoder weight matrix, b e represents the encoder bias term, λ represents the adaptive adjustment factor, tanh represents the hyperbolic tangent function, W f represents the additional weight matrix of the fully connected layer, introducing residual information, and X represents the time series matrix;

[0120] S223. Apply dynamic sparsity constraints to the hidden layer and calculate the regularization term:

[0121]

[0122] in, represents the sparsity regularization loss, ρ represents the target sparsity, represents the average activation of the jth neuron in the hidden layer, γ represents the L1 constraint coefficient, which is used to enhance sparsity, h represents the hidden layer feature representation, and ||h||1 represents the L1 norm of the hidden layer feature representation;

[0123] S224: pass the hidden layer feature representation h to the decoder, reconstruct the input data, compare the obtained reconstructed data X' with the original input X, calculate the reconstruction loss, optimize the parameters of the improved denoising sparse autoencoder, and then optimize the hidden layer feature representation. The reconstruction loss adopts the cross entropy loss:

[0124] X'=σ(W d h+b d +η·W r X);

[0125] Among them, X' represents the reconstructed data, describing the decoded data matrix, σ represents the nonlinear activation function, ReLU represents the nonlinear activation function, W d and b d Denote the decoder weight matrix and bias term respectively, η represents the training coefficient, which is used to introduce the residual information of the original input to enhance the reconstruction ability, and W r represents the additional recovery matrix.

[0126] In this embodiment, the actions in the action space include updating the fault prediction strategy, adjusting the alarm threshold, triggering the alarm mechanism, optimizing the equipment operating parameters, optimizing the inspection plan, and maintaining the current state. The reward function is based on the prediction accuracy, false alarm rate, and early warning capability.

[0127] In this embodiment, the S4 specifically includes:

[0128] S41, using random diffusion model for state transfer, constructing state vector Z t The diffusion process:

[0129]

[0130] Among them, Z t+1 Represents the state vector at time t+1, Z t represents the state vector at time t, δ(Z t ,t) represents the drift term of the state vector, describing the changing trend of the fault characteristics, Δt represents the time step, which is used to control the scale of state change, σ(Z t ,t) represents the diffusion coefficient, describing the uncertainty of the state, ∈ represents the standard normal distribution noise;

[0131] S42, the state vector Z after diffusion t+1 Perform reverse denoising and use the time reversal method to obtain the optimized state vector:

[0132]

[0133] Among them, Z' trepresents the optimized state vector, Z t+1 Represents the state vector at time t+1;

[0134] S43, combined with the optimized state vector Z' t , calculate the state probability density:

[0135]

[0136] Among them, p(Z' t ) represents the state probability density, d represents the dimension of the state vector, Σ represents the covariance matrix of the state vector, |Σ| represents the determinant value of the covariance matrix of the state vector, exp represents the natural exponential function, Z' t represents the state vector after denoising, (Z' t -μ) T Σ -1 (Z' t -μ) represents the Mahalanobis distance, μ represents the mean vector of the state vector, and -1 represents the "inverse" operation of the matrix;

[0137] S44, based on the state probability density p(Z' t ), perform weighted correction on the optimal action:

[0138]

[0139] in, represents the optimized action, τ represents the balance parameter, and controls the influence of the state probability density on the optimal action. t ,A t ) represents the state vector Z' t Next, perform action A t The corresponding Q value, argmax represents the variable value when the function value is maximum;

[0140] S45, based on the optimization strategy sample Train the deep reinforcement learning algorithm, calculate the target Q value, construct the loss function, and update the parameters through gradient descent:

[0141]

[0142]

[0143] Among them, Q target (Z t ,A t ) represents the target Q value, which is used to train the deep reinforcement learning algorithm, R t Represents the immediate reward, which measures the action A taken at the current time step t tThe benefit of ,γ represents the discount factor, which controls the impact of future rewards, Represents the state vector Z' after optimization t Execute the optimization strategy sample below Q value, Represents the loss function, describing the mean square error loss of the Q value, Q θ (Z t ,A t ) represents the Q-value function with parameter θ;

[0144] S46. Train the policy network of the deep reinforcement learning algorithm and use the policy gradient optimization method to update the parameters of the policy network:

[0145]

[0146] in, represents the gradient of the policy objective function, J(θ) represents the policy objective function, describing the maximization of long-term rewards, E represents the mathematical expectation, π θ (A t |Z t ) represents the policy function.

[0147] In this embodiment, the failure probability and health score are:

[0148] P f (Z t )=sigmoid(W f Z t +b f );

[0149] H t =1-P f (Z t );

[0150] Among them, P f (Z t ) represents the railway power equipment in the state vector Z t The failure probability under , sigmoid represents the nonlinear activation function, W f represents the fault prediction weight matrix, b f represents the corresponding bias term, H t Indicates the device health status score.

[0151] Example 1:

[0152] In order to verify the feasibility of the present invention in implementation, the present invention is applied to the railway power equipment fault prediction scenario of a high-speed railway traction power supply system, and the performance in a complex operating environment is studied. The experiment is carried out in a power supply section on a high-speed railway line in China. The section contains multiple traction substations, catenary power supply systems and corresponding signal power supply equipment. The data monitoring of this power supply system relies on the SCADA system, PMU devices and multiple intelligent sensors to collect data such as voltage, current, power, temperature, frequency and vibration signals in real time. However, for a long time, the maintenance of this power supply system has mainly relied on manual inspections and fixed periodic maintenance, resulting in a lag in equipment status monitoring. Sudden failures still occur frequently, affecting the safe operation of trains.

[0153] In this experiment, operating data from railway power equipment was collected between January 2023 and January 2024, totaling over 10TB of time-series data. To improve the accuracy of fault prediction, the collected data was first subjected to denoising, normalization, interpolation, and time synchronization to ensure data integrity and remove abnormal noise signals. Subsequently, an improved denoising sparse autoencoder was used to extract fault features. This was combined with a graph neural network to construct the device topology and form a multidimensional fault feature matrix. Based on this, a deep reinforcement learning algorithm based on a diffusion model was used to predict faults, optimize prediction strategies, and assess the health of the equipment.

[0154] During the experiment, the system conducted real-time monitoring of the power equipment in each traction substation and catenary power supply point every day, and calculated the fault probability and health score. To verify the effectiveness of the present invention, we compared the performance of the present invention with traditional machine learning methods (SVM, random forest), deep learning methods (LSTM, CNN), and reinforcement learning methods without the introduction of diffusion models (DQN, PPO) on the same dataset. The main evaluation indicators included fault prediction accuracy, false alarm rate, early warning capability, and model training convergence speed.

[0155] Table 1 Experimental comparison data table

[0156]

[0157]

[0158] It can be seen from the experimental data comparison table that the present invention adopts a deep reinforcement learning method based on the diffusion model, which shows significant advantages in the prediction of railway power equipment faults.

[0159] First, in terms of prediction accuracy, the present invention achieved 97.6%, which is 15% higher than traditional machine learning methods, 7.8% higher than deep learning methods, and 5.2% higher than ordinary reinforcement learning methods. This shows that the present invention is more robust in identifying failure modes of complex power equipment and can effectively reduce false detections and missed detections.

[0160] In terms of false alarm rate, the present invention is controlled at 2.4%, which is significantly lower than traditional machine learning (9.2%) and deep learning (5.8%), and is also better than the reinforcement learning method without diffusion model optimization (4.2%). This result shows that the high-quality strategy samples generated by the diffusion model can effectively reduce false alarms and improve the stability of predictions, allowing the system to perform fault alarms more accurately and reduce unnecessary operation and maintenance interventions.

[0161] In terms of early warning capability, the present invention can provide accurate warnings in 92.3% of cases before a fault occurs, while the early warning capabilities of traditional machine learning methods are only 68.5%, deep learning methods are 80.1%, and ordinary reinforcement learning methods are 86.2%. The reason why the present invention can provide accurate warnings earlier than a fault occurs is mainly attributed to the diffusion model's ability to dynamically model the health status of equipment, enabling the reinforcement learning model to more keenly capture subtle changes in the health status of equipment, thereby optimizing the early warning strategy, ensuring that the equipment is promptly repaired before a fault occurs, and reducing the impact of sudden failures.

[0162] In terms of training convergence time, the present invention can complete training within 24 hours, while traditional reinforcement learning methods require 37 hours and deep learning methods require 22 hours. Although traditional machine learning methods have a shorter training time (15 hours), the prediction effect is poor. The present invention shortens the training time while maintaining high prediction performance. This is mainly due to the fact that the strategy samples generated by the diffusion model improve data utilization efficiency during the reinforcement learning training process, enabling the model to learn the optimal fault prediction strategy more quickly.

[0163] In terms of fault identification success rate, the present invention achieved 95.2%, which is much higher than the 78.4% of traditional machine learning methods and 86.7% of deep learning methods. It is also better than the 89.1% of ordinary reinforcement learning methods. This shows that the reinforcement learning method optimized by diffusion model is more accurate in processing high-dimensional nonlinear fault modes of railway power equipment, can effectively distinguish between normal and fault states, and improve the reliability of fault diagnosis.

[0164] In terms of computing resource usage, the present invention consumes approximately 120 GPU hours, which is lower than ordinary reinforcement learning methods (150 GPU hours), but higher than traditional machine learning (50 GPU hours) and deep learning methods (85 GPU hours). Although the present invention requires more computing resources than traditional methods, due to its high prediction accuracy, low false alarm rate, and strong early warning capabilities, the investment in computing resources is worthwhile. In addition, through efficient optimization strategies, the present invention still has significant advantages in computing efficiency compared to ordinary reinforcement learning methods.

[0165] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with the technical field, within the technical scope disclosed by the present invention, who makes equivalent replacements or changes based on the technical solution and inventive concept of the present invention, should be covered by the scope of protection of the present invention.

Claims

1. A railway power equipment fault prediction method based on deep reinforcement learning, characterized in that: The steps include: S1. Using the SCADA system, PMU device and intelligent sensors to collect operating data of railway power equipment, and pre-processing the operating data to generate standardized time series data; S2. We use an improved denoising sparse autoencoder combined with a graph neural network to extract features from standardized time series data. We also analyze the topological relationships of railway power equipment to generate a comprehensive feature representation for fault prediction. Based on this comprehensive feature representation, we construct the state space for a deep reinforcement learning algorithm. S3. Define the action space and construct the reward function. Combine the state space, action space, and reward function to build a deep reinforcement learning algorithm based on the diffusion model. S4. Generate policy samples using the diffusion process of the deep reinforcement learning algorithm based on the diffusion model, and train the deep reinforcement learning algorithm based on the diffusion model based on the policy samples; S5. Combined with real-time monitoring data, the trained diffusion model-based deep reinforcement learning algorithm is used to predict faults and derive the failure probability and health score of railway power equipment. S6. Generate predictive maintenance recommendations based on the health score of railway power equipment, combined with fault repair records and equipment life curves.

2. A railway power equipment fault prediction method based on deep reinforcement learning according to claim 1, characterized in that: The operating data includes voltage, current, power, temperature, frequency and vibration signals.

3. The railway power equipment fault prediction method based on deep reinforcement learning according to claim 1 is characterized in that: The preprocessing includes denoising, normalization, interpolation and completion, and time synchronization.

4. The railway power equipment fault prediction method based on deep reinforcement learning according to claim 1 is characterized in that: The S2 specifically includes: S21. Perform window processing on the standardized time series data and set the time window length to T w , construct the time series matrix: Among them, X represents the time series matrix, X t represents the standardized time series data at time t, X t+1 Represents the standardized time series data at time t+1, Represents time t+T w Normalized time series data at time T w Indicates the length of the time window; S22. Perform feature extraction on the time series matrix X using an improved denoising sparse autoencoder, wherein the improved denoising sparse autoencoder is formed by stacking denoising sparse autoencoders, and the denoising sparse autoencoder is composed of an encoding layer, a hidden layer, and a decoding layer; S23. Use the nonlinear transformation principal component analysis method to reduce the dimension of the feature representation extracted by the improved denoising sparse autoencoder and calculate the covariance matrix: Among them, C represents the covariance matrix, describing the nonlinear principal component, ReLU represents the rectified linear unit activation function, m represents the total number of samples, h i represents the hidden layer feature representation of the i-th sample, μ represents the feature mean vector, and T represents the transposition operation; S24. Perform eigenvalue decomposition on the covariance matrix C to obtain the feature representation after dimensionality reduction: h'=V T h+ξ ReLU(W c h); Among them, h ' represents the feature representation after dimensionality reduction, V represents the principal component matrix after eigenvalue decomposition, and W c represents the adjustment matrix in the dimensionality reduction process, ξ represents the dynamic adjustment coefficient; S25. Use graph neural network to represent the features after dimensionality reduction h ' Perform topological relationship modeling and construct a topological graph G = (V, E) of railway power equipment, where V represents the set of equipment nodes and E represents the connection relationship between devices. For each node v in the topological graph, update the feature representation in, represents the feature representation of node v in layer l, σ represents the nonlinear activation function, W g represents the weight matrix of the graph neural network, b g Represents the bias term of the graph neural network, N(v) represents the set of neighbor nodes of node v, and A vu represents the adjacency matrix weight between nodes v and u, represents the feature representation of node v at the l-1 layer, |N(v)| represents the number of neighbors of node v, describing the degree of node v; S26. Dimensionally match the feature representation output by the graph neural network with the feature representation output by the improved denoising sparse autoencoder, and perform feature fusion to construct a comprehensive feature representation for fault prediction: Z=αh′+(1-α)F GNN +ζ·Softmax(W f X); Among them, Z represents the comprehensive feature representation of fault prediction, α represents the weight factor, and h ' represents the feature representation after dimensionality reduction output by the improved denoising sparse autoencoder, F GNN represents the feature representation of the output of the graph neural network, ζ represents the fusion enhancement coefficient, Softmax represents normalization, and W f represents the fusion adjustment matrix, X represents the time series matrix; S27. Construct a state space based on the comprehensive feature representation.

5. A railway power equipment fault prediction method based on deep reinforcement learning according to claim 4, characterized in that: The S22 specifically includes: S221. Add Gaussian noise to the time series matrix X and use the adaptive dynamic mask noise reduction method to generate a noise data matrix: in, represents the noise data matrix, X represents the time series matrix, ⊙ represents the element-by-element multiplication operation, N represents the Gaussian distribution, σ 2 represents the variance of the noise, and M represents the dynamic mask matrix, which is derived from the attention weight: M t =1-exp(-βA t ); Among them, M t represents the dynamic mask matrix, exp represents the natural exponential function, β represents the dynamic adjustment factor, A t represents the feature importance weight at time step t; S222, the noise data Mapped to the hidden layer through the encoder, calculate the hidden layer feature representation h: Among them, h represents the hidden layer feature representation, σ represents the nonlinear activation function, and W e represents the encoder weight matrix, b e represents the encoder bias term, λ represents the adaptive adjustment factor, tanh represents the hyperbolic tangent function, W f represents the additional weight matrix of the fully connected layer, introducing residual information, and X represents the time series matrix; S223. Apply dynamic sparsity constraints to the hidden layer and calculate the regularization term: in, represents the sparsity regularization loss, ρ represents the target sparsity, represents the average activation of the jth neuron in the hidden layer, γ represents the L1 constraint coefficient, which is used to enhance sparsity, h represents the hidden layer feature representation, and ||h||1 represents the L1 norm of the hidden layer feature representation; S224, pass the hidden layer feature representation h to the decoder, reconstruct the input data, and obtain the reconstructed data X ' Compare with the original input X, calculate the reconstruction loss, optimize the parameters of the improved denoising sparse autoencoder, and then optimize the hidden layer feature representation. The reconstruction loss adopts the cross entropy loss: X′=σ(W d h+b d +η·W r X); Among them, X ' Represents the reconstructed data, describes the decoded data matrix, σ represents the nonlinear activation function, ReLU represents the nonlinear activation function, W d and b d Denote the decoder weight matrix and bias term respectively, η represents the training coefficient, which is used to introduce the residual information of the original input to enhance the reconstruction ability, and W r represents the additional recovery matrix.

6. A railway power equipment fault prediction method based on deep reinforcement learning according to claim 1, characterized in that: The actions in the action space include updating fault prediction strategies, adjusting alarm thresholds, triggering alarm mechanisms, optimizing equipment operating parameters, optimizing inspection plans, and maintaining current status. The reward function is based on prediction accuracy, false alarm rate, and early warning capability.

7. The railway power equipment fault prediction method based on deep reinforcement learning according to claim 1 is characterized in that: The S4 specifically includes: S41, using random diffusion model for state transfer, constructing state vector Z t The diffusion process: Among them, Z t+1 Represents the state vector at time t+1, Z t represents the state vector at time t, δ(Z t ,t) represents the drift term of the state vector, describing the changing trend of the fault characteristics, Δt represents the time step, which is used to control the scale of state change, σ(Z t ,t) represents the diffusion coefficient, describing the uncertainty of the state, ∈ represents the standard normal distribution noise; S42, the state vector Z after diffusion t+1 Perform reverse denoising and use the time reversal method to obtain the optimized state vector: Among them, Z' t represents the optimized state vector, Z t+1 Represents the state vector at time t+1; S43, combined with the optimized state vector Z' t , calculate the state probability density: Among them, p(Z' t ) represents the state probability density, d represents the dimension of the state vector, Σ represents the covariance matrix of the state vector, |Σ| represents the determinant value of the covariance matrix of the state vector, exp represents the natural exponential function, Z' t represents the state vector after denoising, (Z' t -μ) T Σ -1 (Z' t -μ) represents the Mahalanobis distance, μ represents the mean vector of the state vector, and -1 represents the "inverse" operation of the matrix; S44, based on the state probability density p(Z' t ), perform weighted correction on the optimal action: in, represents the optimized action, τ represents the balance parameter, and controls the influence of the state probability density on the optimal action. t ,A t ) represents the state vector Z' t Next, perform action A t The corresponding Q value, argmax represents the variable value when the function value is maximum; S45, based on the optimization strategy sample Train the deep reinforcement learning algorithm, calculate the target Q value, construct the loss function, and update the parameters through gradient descent: Among them, Q target (Z t ,A t ) represents the target Q value, which is used to train the deep reinforcement learning algorithm, R t Represents the immediate reward, which measures the action A taken at the current time step t t The benefit of ,γ represents the discount factor, which controls the impact of future rewards, Represents the state vector Z' after optimization t Execute the optimization strategy sample Q value, Represents the loss function, describing the mean square error loss of the Q value, Q θ (Z t ,A t ) represents the Q-value function with parameter θ; S46. Train the policy network of the deep reinforcement learning algorithm and use the policy gradient optimization method to update the parameters of the policy network: in, represents the gradient of the policy objective function, J(θ) represents the policy objective function, describing the maximization of long-term rewards, E represents the mathematical expectation, π θ (A t |Z t ) represents the policy function.

8. The railway power equipment fault prediction method based on deep reinforcement learning according to claim 1 is characterized in that: The failure probability and health score are: P f (Z t )=sigmoid(W f Z t +b f ); H t =1-P f (Z t ); Among them, P f (Z t ) represents the railway power equipment in the state vector Z t The failure probability under , sigmoid represents the nonlinear activation function, W f represents the fault prediction weight matrix, b f represents the corresponding bias term, H t Indicates the device health status score.

Images