New energy multi-mode fault diagnosis method based on dynamic graph convolution and transfer learning

Abstract

The invention discloses a new energy multi-mode fault diagnosis method based on dynamic graph convolution and transfer learning. According to the method, multiple types of sensors are deployed to collect operation, environment and historical data of new energy equipment, and a multi-modal data set is generated through preprocessing; the method comprises the following steps: defining an equipment component as a graph node, constructing a directed graph containing a time-varying edge weight by utilizing dynamic time warping and an attention mechanism, and updating a graph structure on line; a dual-branch dynamic graph convolutional network is adopted to extract multi-modal data features, an adversarial transfer learning algorithm and a meta-learning algorithm are combined, distribution of a source domain and a target domain is aligned, and a special diagnosis model is generated by using small samples. And fault classification is carried out, a fault component is positioned in combination with a graph node attention weight, and a fault evolution index is constructed by fusing a fault prediction probability and an attention weight change rate. The method effectively fuses multi-modal data, adapts to complex working conditions, improves small sample diagnosis precision, realizes fault accurate positioning and evolution prediction, and has important significance for guaranteeing stable operation of new energy equipment.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of new energy equipment fault diagnosis, in particular to a new energy multi-modal fault diagnosis method based on dynamic graph convolution and transfer learning. BACKGROUND

[0002] With the growing demand for clean energy worldwide, the new energy industry has developed rapidly. Wind power, photovoltaic power and other forms of new energy are increasingly occupying a larger share of the energy structure. For example, wind power, the global installed capacity of wind power continues to grow rapidly, and the single-machine capacity of large-scale wind turbines is constantly improving. The scale of wind farms is also getting larger. In terms of photovoltaic power generation, photovoltaic power stations have been built all over the world, from large-scale centralized photovoltaic power stations to distributed photovoltaic power systems, and have made remarkable progress.

[0003] However, new energy equipment faces many challenges during operation. New energy equipment usually works in complex and variable environments, such as wind turbines that need to cope with different wind speeds, wind directions and weather conditions, and photovoltaic equipment that needs to adapt to different light intensities, temperatures and humidity, etc. These complex environmental factors increase the probability of equipment failure, posing a huge challenge to the stable operation of new energy equipment.

[0004] Traditional new energy equipment fault diagnosis methods are mainly based on the analysis of single sensor data, such as vibration sensors, temperature sensors, etc. These methods have the following obvious limitations:

[0005] 1. The operating state of new energy equipment can be reflected by multiple types of data, such as vibration signals, current waveforms, infrared images, sound signals, etc. Traditional methods only focus on a single type of data and cannot fully exploit the internal relationship between multi-modal data, resulting in incomplete and inaccurate diagnosis of equipment failure.

[0006] 2. The operating conditions of new energy equipment are dynamically changing, for example, the operating state of wind turbines under different wind speeds is very different. Traditional fault diagnosis models are usually trained based on data under fixed operating conditions, and when the operating conditions of the equipment change, the diagnostic performance of the model will decrease significantly, and the fault cannot be detected in time and accurately.

[0007] 3. The fault data of new energy equipment is usually scarce, especially early fault data. Traditional machine learning and deep learning methods require a large amount of labeled data to train the model, and in the case of small samples, the model is prone to overfitting, resulting in reduced diagnostic accuracy. SUMMARY

[0008] The technical problem to be solved by the present application is to provide a new energy multi-modal fault diagnosis method based on dynamic graph convolution and transfer learning to solve the deficiencies of the prior art described in the background.

[0009] To solve the above technical problems, the embodiments of the present application provide the following technical solutions: a new energy multi-modal fault diagnosis method based on dynamic graph convolution and transfer learning, comprising:

[0010] By deploying multiple types of sensors to synchronously collect the operation data, environmental data and historical data of new energy equipment, the collected data is denoised, normalized, image preprocessed and missing value filled to generate a multi-modal data set;

[0011] The equipment components are defined as graph nodes, the node features are generated by fusing the multi-modal data, the time-varying edge weight is calculated based on dynamic time warping and attention mechanism, a directed graph reflecting the dynamic correlation of the components is constructed, and the graph structure is dynamically updated through an online learning mechanism;

[0012] A double-branch dynamic graph convolution network (DGCN) is used to extract features from the multi-modal data, and the spatial and temporal features are fused through a cross-modal fusion layer;

[0013] The source domain and the target domain distribution are aligned, and the fault evolution pattern is learned by combining the adversarial transfer learning and the meta-learning algorithm, and a special diagnostic model is quickly generated using the small sample data of the target domain;

[0014] The fused features are input into the full connection layer of the special diagnostic model for fault classification, a fault evolution index is constructed by combining the change rate of the graph node attention weight, and the fault component is located through the graph node attention weight.

[0015] Further, the directed graph reflecting the dynamic correlation of the components is constructed, specifically:

[0016] The components of the new energy equipment are defined as graph nodes V={v1, v2, …v n}, and the node features f v The multi-modal data of the components are fused, the time-varying edge weight matrix E(t) is calculated based on dynamic time warping (DTW) and attention mechanism, a directed graph G(t)=(V, E(t)) containing time-varying edge weight is generated, and the dynamic correlation of the equipment components is reflected;

[0017] The element E ij (t) in the dynamic edge weight matrix E(t) is calculated according to the following formula:

[0018] E ij (t)=α·DTW(X i (t),X j (t))+(1-α)·CosineSim(f v(i),f v (j))

[0019] wherein, a is a modal fusion coefficient, and its optimal value is determined by a gradient descent optimization algorithm; DTW(f i (i), f j (j)) is the similarity between the time series data X i (i) and X j (j) corresponding to nodes i and j respectively, CosineSim(f v (i), f v (j)) is the cosine similarity of the node features f v (i) and f v (j) corresponding to nodes i and j respectively.

[0020] Further, the dual-branch dynamic graph convolution network DGCN includes a spatial branch and a temporal branch, wherein: the spatial branch adopts a graph attention convolution GAT to capture the spatial correlation of node neighborhoods, the temporal branch introduces a time gating mechanism to capture the evolution law of time series features, and the outputs of the spatial and temporal branches are fused through a cross-modal fusion layer to generate a multi-dimensional feature vector H = {h1, h2, …, h T}.

[0021] Further, in the graph attention convolution GAT, the feature representation of node v at the l+1th layer is The calculation formula is:

[0022]

[0023] wherein, h is the feature representation of node u at the lth layer, N(v) is the neighbor node set of node v, a vu is the attention weight of node v at neighbor node u, W (l) is the convolution kernel of the lth layer, and σ is an activation function.

[0024] Further, in the time gating mechanism, the gating output g t at time t is calculated according to the following formula:

[0025] g t = σ(W g ·h v (t)+U g ·g t-1 )

[0026] wherein, h v (t) is the feature representation of node v at time t, W g and U g are learnable weight matrices, and g t-1 is the gating output at the previous time.

[0027] Further, in the training process of the adversarial transfer learning, the parameters of the feature extractor f and the domain discriminator d are adjusted by minimizing the following loss function:

[0028]

[0029] where e denotes expectation, x s is a data sample of the source domain, x t is a data sample of the target domain, D s is the historical fault data of the same type of device as the source domain, and D t is the small sample data of the current device to be diagnosed as the target domain.

[0030] Further, the fused features are input into the full connection layer of the special diagnostic model for fault classification. The fused feature vector is input into the full connection layer, and the probability distribution of the fault type is output through the Softmax function. The Softmax function maps the feature vector to the probability space, and the probability of each class represents the possibility of the sample belonging to the fault type. The class with the maximum probability is selected as the predicted fault type to realize fault classification.

[0031] Further, the cross-modal fusion layer fuses the spatial and temporal features by using the splicing or weighted summation method to fuse the outputs of the spatial and temporal branches.

[0032] Further, the online learning mechanism dynamically updates the graph structure by minimizing the feedback loss function to update the edge weight matrix:

[0033] L g = λ1·CE(y pred ,y true ) + λ2·||ΔE(t)||2

[0034] L g is the graph structure feedback loss function, λ1 is the balance coefficient of the cross-entropy loss term, CE(y pred ,y true ) is the cross-entropy loss between the predicted fault class y pred and the real fault class y true , λ2 is the balance coefficient of the edge weight change norm term, ΔE(t) is the change amount of the edge weight matrix E(t) at the current time, and ||ΔE(t)||2 is the L2 norm of the edge weight change ΔE(t).

[0035] Further, the learning of the fault evolution pattern is performed by fusing the fault prediction probability and the attention weight change rate to construct a fault evolution index:

[0036] FEI = β P(fault) + (1 - β) Rate(A v )

[0037] Wherein, FEI is a fault evolution index, beta is a fusion coefficient, P(fault) is the probability of fault occurrence, 1-beta is a weight coefficient related to the fusion coefficient, and Rate(A v ) is the change rate of the graph node attention weight.

[0038] The beneficial effects of the above technical solutions of the application are as follows:

[0039] 1. The application effectively fuses multi-modal data such as vibration, electrical signal and image by constructing a dynamic graph model, mines the spatio-temporal correlation features between multi-modal data, and improves the comprehensiveness and accuracy of fault diagnosis. And through the online learning mechanism, the graph structure is dynamically updated, and when the working condition of the new energy equipment changes, the loss function can adjust the edge weight, so that the model quickly adapts to the change of the correlation between components.

[0040] 2. The application introduces dynamic edge weight and time gating mechanism, so that the model can adapt to the dynamic change of the running condition of the new energy equipment, accurately capture the dynamic coupling relationship between the equipment components, and improve the fault diagnosis performance under complex working conditions.

[0041] 3. The application adopts a transfer learning strategy, aligns the data distribution of the source domain and the target domain through an adversarial transfer learning method, uses the knowledge of the source domain to help the model training of the target domain, reduces the dependence on large-scale labeled data of the target domain, and improves the fault diagnosis precision under small sample conditions. At the same time, through a small amount of fault evolution samples, the time sequence evolution mode of the fault features can be quickly learned, so that the model can also carry out effective analysis under the condition of limited data.

[0042] 4. The application identifies the key components corresponding to abnormal features through the graph node attention weight, realizes accurate positioning of the fault occurrence position, provides clear troubleshooting direction for the operation and maintenance personnel, and improves the operation and maintenance efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0043] Figure 1 The flow chart of the new energy multi-modal fault diagnosis method of the dynamic graph convolution and transfer learning of the application. DETAILED DESCRIPTION

[0044] In order to make the technical problems, technical solutions and advantages of the application more clear, the following will be described in detail in combination with the drawings and specific embodiments.

[0045] Embodiment 1

[0046] As Figure 1As shown, the present application proposes a new energy multi-modal fault diagnosis method based on dynamic graph convolution and transfer learning, which includes:

[0047] S1, by deploying multiple types of sensors to synchronously collect the operation data, environmental data and historical data of new energy equipment, the collected data is denoised, normalized, image preprocessed and missing value filled to generate a multi-modal data set;

[0048] S2, the device components are defined as graph nodes, the node features are generated by fusing multi-modal data, the time-varying edge weights are calculated based on dynamic time warping and attention mechanism, the directed graph reflecting the dynamic association of components is constructed, and the graph structure is dynamically updated through online learning mechanism;

[0049] S3, a double-branch dynamic graph convolution network DGCN is used to extract features from multi-modal data, and spatial and temporal features are fused through a cross-modal fusion layer;

[0050] S4, combining the adversarial transfer learning and meta-learning algorithm, aligning the source domain and target domain distribution, learning the fault evolution mode, and using the target domain small sample data to quickly generate a special diagnostic model;

[0051] S5, the fused features are input into the full connection layer of the special diagnostic model for fault classification, the fault evolution index is constructed by combining the change rate of graph node attention weight, and the fault component is located by the graph node attention weight.

[0052] Step S1 in order to comprehensively obtain the running state information of new energy equipment, it is necessary to deploy multiple types of sensors, such as:

[0053] Vibration sensor: used to collect the vibration signals of the equipment, such as the vibration of the gear box, bearing and other components of the wind turbine. Vibration sensors usually have a high sampling rate to accurately capture the high frequency components of the vibration signal.

[0054] Current / voltage sensor: real-time monitoring of the current and voltage changes of the equipment, such as the input and output current and voltage of the photovoltaic inverter. These electrical signals can reflect the electrical performance and running state of the equipment.

[0055] Infrared thermal imager: by detecting the infrared radiation of the equipment surface, generating infrared images, used for detecting the heating condition of the equipment, such as the overheating of the electrical equipment joint, the heating of the motor winding, etc.

[0056] IMU inertial measurement unit: measures the acceleration, angular velocity and other motion parameters of the equipment, which can be used to monitor the attitude and motion state of the equipment, such as the vibration and swing of the wind turbine blade.

[0057] Environmental sensors: Collect relevant data about the operating environment of the equipment, such as temperature, humidity, wind speed, and light intensity. Environmental factors have an important impact on the operation of new energy equipment, so the collection of environmental data helps to more accurately diagnose faults.

[0058] Historical data records: Collect historical fault records, maintenance logs, and other information of the equipment, which can provide important references for fault diagnosis.

[0059] Then, the collected raw data often has problems such as noise and missing values, which need to be preprocessed to improve data quality. The specific preprocessing steps are as follows:

[0060] (1) Noise reduction: For vibration signals, use wavelet packet noise reduction method to remove noise interference. Wavelet packet transform can decompose the signal into different frequency subbands, and then reconstruct the signal after threshold processing of each subband coefficients. For electrical signals, similar methods can also be used for noise reduction.

[0061] (2) Normalization: Normalize the collected data to map it to the range of [0, 1] or [-1, 1] to eliminate the dimensional differences between different data and improve the training effect of the model. Common normalization methods include min-max normalization and Z-score normalization.

[0062] Image preprocessing: For infrared images, perform grayscale equalization to enhance the contrast of the image and make the details in the image clearer. At the same time, the image can be cropped, scaled, etc. to meet the requirements of model input.

[0063] Missing value processing: Use multiple imputation method to fill in missing values in the data. Multiple imputation method generates multiple complete data sets through multiple sampling and estimation, and then analyzes these data sets comprehensively to improve the accuracy of missing value filling.

[0064] After preprocessing, a multi-modal time series data set D = {X1, X2, … X m} is generated, where X i contains multi-dimensional features such as time domain, frequency domain, and image.

[0065] In step S2, a directed graph reflecting the dynamic association of components is constructed, specifically:

[0066] First, define the components of new energy equipment as graph nodes V = {v1, v2, … v n}, for example, for a wind turbine, nodes can include gearboxes, bearings, generators, blades, etc. Components; for photovoltaic inverters, nodes can include power modules, capacitors, inductors, etc. Components. The feature f vThe multi-modal data of the component, such as the root mean square value of vibration, temperature, historical failure probability, etc., are fused. By fusing the multi-modal data into the node features, the running state of the device component can be more comprehensively reflected.

[0067] In order to reflect the dynamic coupling relationship between the device components, the dynamic edge weight needs to be calculated. The application adopts a method combining dynamic time warping (DTW) and attention mechanism to calculate the edge weight. The specific steps are as follows:

[0068] Dynamic time warping (DTW): DTW is a method for calculating the similarity between two time series, which can handle the problem of inconsistent length and stretching on the time axis of time series. For the time series data X i (t) and X j (t) corresponding to nodes i and j respectively, the similarity DTW(X i (t), X j (t)) between them is calculated by DTW.

[0069] Cosine similarity calculation: At the same time, the cosine similarity CosineSim(f v (i), f v (j)) between the node features f v (i) and f v (j) is calculated, which can reflect the directional similarity between two node features.

[0070] Then, the multi-modal data of the component is fused through the node feature f v , and the time-varying edge weight matrix E(t) is calculated based on dynamic time warping (DTW) and attention mechanism. The element E ij (t) in the dynamic edge weight matrix E(t) is calculated according to the following formula:

[0071] E ij (t) = a · DTW(X i (t), X j (t)) + (1-a) · CosineSim(f v (i), f v (j))

[0072] Wherein, a is a modal fusion coefficient, and its optimal value is determined by a gradient descent optimization algorithm; DTW(X i (t), X j (t)) is the similarity between the time series data X i (t) and X j (t) corresponding to nodes i and j respectively, and CosineSim(f v (i), f v (j)) is the node feature fv (i) and f v (j) cosine similarity.

[0073] Finally, according to the node definition and the dynamic edge weight calculation result, a directed graph G(t) = (V, E(t)) with time-varying edge weights is generated, which can accurately reflect the dynamic association relationship between device components. With the passage of time, the edge weights will be dynamically adjusted according to the changes in the device running state, so as to better capture the coupling relationship between device components.

[0074] In this embodiment, the double-branch dynamic graph convolution network DGCN in step S3 includes a spatial branch and a temporal branch, wherein: the spatial branch adopts a graph attention convolution GAT to capture the spatial association of node neighborhood, and the temporal branch introduces a time gating mechanism to capture the evolution law of time series features. The outputs of the spatial and temporal branches are fused through a cross-modal fusion layer to generate a multi-dimensional feature vector H = {h1, h2, …, h T}.

[0075] A graph attention convolution (GAT) is used to capture the spatial association of node neighborhood. The graph attention convolution assigns different attention weights to the neighbor nodes of each node through a multi-head attention mechanism, so as to more effectively aggregate the information of neighbor nodes. The feature representation of node v in the (l+1) th layer in the graph attention convolution GAT is The calculation formula is:

[0076]

[0077] Wherein is the feature representation of node u in the l th layer, N(v) is the neighbor node set of node v, α vu is the attention weight of node v in neighbor node u, W (l) is the convolution kernel of the l th layer, and σ is the activation function.

[0078] A time gating mechanism (TemporalGate) is introduced to capture the evolution law of time series features. The time gating mechanism controls the fusion degree of historical features and current features through a gating unit. The gating output g t of the time gating mechanism at t time is

[0079] g t = σ(W g ·h v (t) + U g ·g t-1 )

[0080] Wherein h v (t) is the feature representation of node v at t time, W g and Ug is a learnable weight matrix, g t-1 is the gating output of the last time step.

[0081] Finally, the outputs of the spatial branch and the temporal branch are fused by a cross-modal fusion layer to generate a multi-dimensional feature vector H. The cross-modal fusion layer can fuse the spatial and temporal features in a manner such as concatenation or weighted summation to make full use of the spatio-temporal information of the multi-modal data.

[0082] In the training process of the step S4 of the adversarial transfer learning, the parameters of the feature extractor f and the domain discriminator d are adjusted by minimizing the following loss function:

[0083]

[0084] where e represents expectation, x s is a data sample of the source domain, x t is a data sample of the target domain, D s The historical fault data of the same type of equipment is taken as the source domain, D t The small sample data of the current equipment to be diagnosed is taken as the target domain. An adversarial transfer learning method is used to reduce the data distribution difference between the source domain and the target domain. The adversarial transfer learning is realized by introducing a domain discriminator. The role of the domain discriminator is to distinguish whether the input data is from the source domain or the target domain. In the training process, the feature extractor and the domain discriminator are trained in an adversarial manner, and the parameters are constantly adjusted until the domain discriminator cannot accurately distinguish the data of the source domain and the target domain. At this time, the distribution of the data of the source domain and the target domain in the feature space is well aligned. After the data distribution alignment of the source domain and the target domain is completed, the model is fine-tuned using a small amount of labeled data (such as 5-10 fault samples) of the target domain. The process of fine-tuning mainly adjusts the parameters of the classification layer of the model, so that the model can better adapt to the fault diagnosis task of the target domain.

[0085] Step S5 inputs the fused feature vector H into the full connection layer of the special diagnostic model for fault classification. The fused feature vector is input into the full connection layer, and the probability distribution of the fault type is output through the Softmax function. The Softmax function maps the feature vector to the probability space, and the probability of each category represents the possibility that the sample belongs to the fault type. According to the probability distribution, the category with the maximum probability is selected as the predicted fault type, and fault classification is realized. For example, the fault types can include bearing fault, inverter overheating, battery internal resistance anomaly, etc. At the same time, based on the graph node attention weight, the key components corresponding to the abnormal features are identified, so as to locate the position where the fault occurs. In the process of graph attention convolution, the neighbor nodes of each node are assigned different attention weights, and the attention weight reflects the importance of the neighbor nodes to the current node. When the attention weight of a certain node abnormally increases, it indicates that the association between the node and other nodes has changed, and there may be a fault. By analyzing the distribution of the graph node attention weight, the node with an abnormal attention value is found, and the device component corresponding to the node is taken as the position where the fault occurs. For example, in the fault diagnosis of a wind turbine, if the attention weight of the gearbox node is significantly higher than those of other nodes, and the related vibration and temperature data also appear abnormal, it can be judged that the gearbox may have a fault.

[0086] In this embodiment, after the existing dynamic graph model is constructed, the update of the edge weight mainly depends on the historical data and fixed rules, and it is difficult to adapt to the dynamic association changes under the conditions of device aging, component performance degradation or extreme working conditions in real time. For example, after long-term operation of the device, the coupling relationship between components may change nonlinearly due to wear and aging, and the traditional dynamic graph model cannot timely capture this gradual change process, resulting in a decrease in diagnostic accuracy with the use time of the device. Step S2 introduces an online learning mechanism, dynamically adjusts the modal fusion coefficient α and node feature f v in the calculation of the edge weight based on the real-time collected multi-modal data through the gradient descent algorithm

[0087] L g = λ1·CE(y pred ,y true )+ λ2·||ΔE(t)||2

[0088] L g is the graph structure feedback loss function, which is used to guide the update of the dynamic graph structure to optimize the representation ability of the fault diagnosis model for the dynamic association of the device components;

[0089] λ1 is a balance coefficient of the cross-entropy loss term, which is used to adjust the weight of the fault classification accuracy in the loss function and reflects the importance of the classification result.

[0090] CE(y pred ,y true ) is the cross-entropy loss between the predicted fault class y pred and the true fault class y true , which measures the accuracy of the model in classifying faults, and the smaller the value, the more accurate the classification is;

[0091] λ2 is the balance coefficient of the edge weight change amount norm term, which is used to adjust the influence of the edge weight change range in the loss function and control the stability of the graph structure update;

[0092] ΔE(t) is the change amount of the edge weight matrix E(t) at the current time, which reflects the update range of the dynamic association between device components;

[0093] ||ΔE(t)||2 is the L2 norm of the edge weight change amount ΔE(t), which is used to constrain the update range of the edge weight to avoid excessive adjustment leading to unstable graph structure.

[0094] By minimizing the loss function, the model parameters can be adjusted according to the cross-entropy loss between the predicted fault class y pred and the true fault class y true , so that the model can make the prediction result more close to the true situation when classifying faults, improve the accuracy of fault classification, and avoid misjudgment. The self-calibration mechanism is introduced in the graph attention convolution (GAT), which automatically adjusts the weight distribution strategy of multi-head attention when detecting that the device enters a new working condition (such as sudden change of wind speed of wind turbine or sudden increase of load of photovoltaic inverter). For example, in high wind speed working condition, the attention weight of blade vibration and gear box node is increased to suppress the interference of irrelevant components and enhance the feature extraction ability of key path. Through online learning mechanism, the changes of component association caused by device aging or working condition mutation can be captured in real time.

[0095] In this embodiment, step S4 combines the adversarial transfer learning and meta-learning algorithm to align the source domain and target domain distribution and learn the fault evolution pattern, wherein the fault evolution pattern is learned by fusing the fault prediction probability and the attention weight change rate to construct a fault evolution index FEI:

[0096] FEI=β·P(fault)+(1-β)·Rate(A v )

[0097] Wherein, FEI is the fault evolution index, which is used to comprehensively evaluate the evolution trend of device faults and judge whether the device is developing towards the fault direction and the degree of development. This index combines the probability of fault occurrence and the change rate of graph node attention weight, which can more comprehensively reflect the development trend of faults.

[0098] β is a fusion coefficient, which is a parameter between 0 and 1, and is calibrated by historical failure data. It is used to balance the weight of the failure occurrence probability P(fault) and the rate of attention weight change Rate(A v ) in the calculation of the failure evolution index, to determine the relative importance of the two factors in the judgment of failure evolution.

[0099] P(fault) is the probability of failure occurrence, which is output by the failure evolution prediction model based on meta-learning, representing the possibility of device failure at a certain time in the future. The probability value is obtained based on the analysis of the multi-modal data of the device and the learning of the time evolution pattern of the failure characteristics.

[0100] 1-β is a weight coefficient related to the fusion coefficient, used to determine the weight of the rate of attention weight change in the failure evolution index, and works together with β to achieve a trade-off between the two factors;

[0101] Rate(A v ) is the rate of change of the attention weight of the graph node, for example, the weekly growth rate of the attention weight of a certain component. It reflects the relative change of the attention weight of the graph node within a certain period of time, and embodies the dynamic change of the importance of the device components in fault diagnosis over time, which can be used to assist in judging whether the fault is developing or has a potential development trend.

[0102] The existing transfer learning only focuses on the fault diagnosis at the current time, and lacks the prediction ability of the failure evolution process. The early failure of new energy equipment is usually characterized by gradual change of features (such as slow rise of vibration amplitude in the early stage of bearing wear), and traditional methods are difficult to distinguish between normal fluctuations and failure precursors, resulting in a high rate of missed diagnosis.

[0103] The present application first defines the meta-learning task, abstracts the fault diagnosis task as a "few-shot time series prediction task" in meta-learning, and defines the task Wherein contains multi-modal time series data of the device in the failure evolution process (such as time series of vibration signals, temperature, and current), is the data to be predicted at the future time.

[0104] Then a failure evolution prediction model is constructed, a meta-learner (Meta-Learner) is embedded in the transfer learning module, and a model-independent meta-learning (MAML) algorithm is used to quickly learn the time evolution pattern of failure characteristics through a small number of failure evolution samples (such as 3-5 failure development period data). The specific steps are as follows:

[0105] Meta-training phase: use the failure evolution data of multiple devices in the source domain to train the meta-learner to generate initial model parameters, so that the model can quickly adapt to the failure evolution law of new devices;

[0106] Meta-testing phase: For target domain devices, only a small sample of 1-2 fault development cycles is needed to obtain a dedicated fault evolution prediction model through gradient updates, outputting the fault probability curve at future time T.

[0107] Finally, the fault prediction results are combined with the rate of change of the attention weight of the graph node (such as the weekly growth rate of the attention weight of a certain component) to construct a fault evolution index. The fault evolution pattern can effectively cope with the data scarcity dilemma, breaking through the limitations of traditional methods that rely on large amounts of labeled data, allowing the model to carry out effective analysis even in the case of limited data.

[0108] Example 2

[0109] (1) The following gives a wind turbine gearbox fault diagnosis case based on the method of the present application. First, a variety of sensors are deployed on a 2MW wind turbine gearbox. Specifically, they include:

[0110] Acceleration sensor: X / Y / Z three-axis acceleration sensors are installed on the input shaft, intermediate shaft and output shaft of the gearbox, with a sampling rate of 50kHz, used to collect vibration signals of each shaft of the gearbox.

[0111] Temperature sensor: Temperature sensors are installed at key parts of the gearbox, such as bearings, gear meshing, etc., with an accuracy of ±0.5℃, to monitor the temperature changes of the gearbox in real time.

[0112] Speed sensor: Installed on the generator shaft, used to measure the speed of the generator, thereby indirectly reflecting the running state of the gearbox.

[0113] Environmental sensor: An anemometer and a wind vane are installed on the wind farm site to collect wind speed and direction data, as well as temperature, humidity and other environmental data.

[0114] At the same time, the historical fault records and maintenance logs of the wind turbine are collected, including past failures such as gear wear, bearing pitting, etc.

[0115] Then, each component of the gearbox is defined as a graph node, including the input shaft bearing, gear pair, output shaft bearing, etc. Based on the collected vibration signals, the dynamic time warping (DTW) method is used to calculate the similarity between nodes, and the cosine similarity of node features is combined to calculate the dynamic edge weight. For example, for the input shaft bearing and gear pair nodes, the DTW similarity of their vibration signals and the cosine similarity of the node features (such as vibration root mean square value, temperature, etc.) are calculated to obtain the edge weight between them. Over time, the edge weight will be dynamically adjusted according to the changes in the running state of the device, thereby constructing a directed graph reflecting the dynamic association of the gearbox components.

[0116] Next, a double-branch dynamic graph convolutional network (DGCN) is used for feature extraction. The spatial branch uses a graph attention convolution (GAT) to capture the spatial correlations between the gearbox components. For example, by using the graph attention convolution, the correlation between the vibration signals of the input shaft bearing and the gear pair is analyzed to identify the key components that have a greater impact on the operation of the gearbox. The temporal branch introduces a temporal gating mechanism to analyze abnormal fluctuations in temperature over time. For example, when the temperature of a certain part of the gearbox rises sharply in a short period of time, the temporal gating mechanism can capture this change in time and extract it as a fault feature. Finally, the spatial and temporal features are fused through a cross-modal fusion layer to generate a feature vector that contains spatio-temporal correlations.

[0117] The historical fault data of a wind turbine that has been running for 3 years with the same model is used as the source domain to train the newly commissioned wind turbine (target domain). Through the adversarial transfer learning method, the distribution of the data in the feature space of the source domain and the target domain is well aligned. Then, the model is fine-tuned using 5 early bearing fault samples from the target domain to adjust the classification layer parameters of the model, so that the model can better adapt to the fault diagnosis task of the target domain.

[0118] Then, the fused feature vector is input into a fully connected layer, and a Softmax function is used for fault classification. When the attention weight of the input shaft bearing node suddenly increases and the high-frequency component of the vibration signal is abnormal, the model outputs the "bearing outer ring wear" fault. Through the analysis of the graph node attention weight, the location of the fault is accurately located, with an accuracy of 92%. Based on the fault diagnosis result, the operation and maintenance personnel timely replaced and repaired the input shaft bearing, avoiding further expansion of the fault and ensuring the normal operation of the wind turbine.

[0119] (2) The following gives a photovoltaic inverter fault diagnosis case based on the method of the present application

[0120] First, current / voltage sensors, temperature sensors, and infrared thermographs are deployed on the photovoltaic inverter. The current / voltage sensors monitor the input and output current and voltage of the inverter in real time, the temperature sensors measure the temperature of the key components inside the inverter, and the infrared thermograph periodically acquires infrared images of the inverter. At the same time, the running time of the inverter, environmental data such as light intensity, and historical fault records are recorded.

[0121] Then, the power modules, capacitors, inductors, and other components of the photovoltaic inverter are defined as graph nodes. Based on the similarity of the time-domain waveforms of the current / voltage signals and the correlation of the node features, the dynamic edge weights are calculated to construct a directed graph reflecting the dynamic correlations between the components of the inverter. For example, when the current waveform of the power module changes abnormally, the edge weights between it and other components will be adjusted accordingly to reflect the dynamic coupling relationship between the components.

[0122] The double-branch dynamic graph convolution network (DGCN) is used for feature extraction. The spatial branch analyzes the electrical connection relationship between components through the graph attention convolution analysis component, and the time branch captures the trend of temperature and current changes over time through the time gating mechanism. For example, when the temperature of a certain power module continues to rise and the current fluctuates abnormally, the time gating mechanism will extract these information as important features. The spatial and temporal features are fused through the cross-modal fusion layer to obtain a feature vector containing spatio-temporal information.

[0123] Then, the historical fault data of other inverters in the same type of photovoltaic power station is selected as the source domain to perform transfer learning on the current inverter (target domain). Through the adversarial transfer learning method, the data distribution of the source domain and the target domain is aligned. Then, the model is fine-tuned using 8 fault samples of the target domain to improve the diagnostic performance of the model in the target domain.

[0124] Finally, the fused feature vector is input into the classifier for fault classification. When the attention weight of a certain power module node is abnormally increased, and the infrared image shows that the temperature of the module is too high, the model judges that the power module has a fault. Through the visualization analysis of the graph node attention weight, the fault power module is accurately located, providing a basis for timely maintenance.

[0125] The above is the preferred embodiment of the present application. It should be noted that for those skilled in the art, without departing from the principles of the present application, several improvements and refinements can be made, which should also be considered within the scope of protection of the present application.

Claims

1. A method for multimodal fault diagnosis of new energy sources based on dynamic graph convolution and transfer learning, comprising: By deploying multiple types of sensors to synchronously collect operational data, environmental data, and historical data of new energy equipment, the collected data is denoised, normalized, image preprocessed, and missing value filled to generate a multimodal dataset. Device components are defined as graph nodes. Node features are generated by fusing multimodal data. Time-varying edge weights are calculated based on dynamic time warping and attention mechanisms to construct a directed graph that reflects the dynamic relationships between components. The graph structure is then dynamically updated through an online learning mechanism. A dual-branch dynamic graph convolutional network (DGCN) is used to extract features from multimodal data, and spatial and temporal features are fused through a cross-modal fusion layer. By combining adversarial transfer learning and meta-learning algorithms, the distributions of the source and target domains are aligned, and fault evolution patterns are learned. A dedicated diagnostic model can be quickly generated using small sample data from the target domain. The fused features are input into the fully connected layer of the dedicated diagnostic model for fault classification. The fault evolution index is constructed by combining the graph node attention weight change rate, and the fault components are located by the graph node attention weight.

2. The method for multimodal fault diagnosis of new energy sources based on dynamic graph convolution and transfer learning according to claim 1, characterized in that, The construction of the directed graph reflecting the dynamic relationships between components is specifically as follows: The components of new energy equipment are defined as graph nodes V = {v1, v2, ... v} n }, node features f v By integrating the multimodal data of the components, the time-varying edge weight matrix E(t) is calculated based on dynamic time warping (DTW) and attention mechanism, and a directed graph G(t) = (V, E(t)) containing time-varying edge weights is generated, which reflects the dynamic relationship of the device components. The elements E in the dynamic edge weight matrix E(t) ij The formula for calculating (t) is: E ij (t)=α·DTW(X i (t),X j (t))+(1-α)·CosineSim(f v (i),f v (j)) Where α is the modality fusion coefficient, and its optimal value is determined by the gradient descent optimization algorithm; DTW(X) i (t),X j (t) represents the time series data X corresponding to nodes i and j, respectively. i (t) and X j The similarity between (t), CosineSim(f) v (i),f v (j) represents the node features f corresponding to nodes i and j, respectively. v (i) and f v Cosine similarity of (j).

3. The method for multimodal fault diagnosis of new energy sources using dynamic graph convolution and transfer learning according to claim 1, characterized in that, The dual-branch dynamic graph convolutional network (DGCN) includes a spatial branch and a temporal branch. The spatial branch uses graph attention convolution (GAT) to capture the spatial relationships within node neighborhoods, while the temporal branch introduces a temporal gating mechanism to capture the evolution of time-series features. The outputs of the spatial and temporal branches are fused through a cross-modal fusion layer to generate a multi-dimensional feature vector H = {h1, h2, ..., h...}. T } 4. The method for multimodal fault diagnosis of new energy sources using dynamic graph convolution and transfer learning according to claim 3, characterized in that, The feature representation of node v in the graph attention convolutional GAT at layer l+1 The calculation formula is: in Let α be the feature representation of node u at layer l, and N(v) be the set of neighboring nodes of node v. vu W is the attention weight of node v to its neighbor node u. (l) σ is the convolutional kernel of the l-th layer, and σ is the activation function.

5. The method for multimodal fault diagnosis of new energy sources using dynamic graph convolution and transfer learning according to claim 3, characterized in that, The gated output g at time t in the time-gating mechanism t The calculation formula is: g t =σ(W g ·h v (t)+U g ·g t-1 ) Where h v (t) is the feature representation of node v at time t, W g and U g It is a learnable weight matrix, g t-1 It is the gating output from the previous moment.

6. The method for multimodal fault diagnosis of new energy sources using dynamic graph convolution and transfer learning according to claim 1, characterized in that, During the training process of adversarial transfer learning, the parameters of the feature extractor f and the domain discriminator d are adjusted by minimizing the following loss function: Where e represents expectation, x s It is a data sample from the source domain, x t It is a data sample of the target domain, D s Using historical fault data of similar equipment as the source domain, D t The target domain is a small sample of data from the device to be diagnosed.

7. The method for multimodal fault diagnosis of new energy sources using dynamic graph convolution and transfer learning according to claim 1, characterized in that, The fused features are input into the fully connected layer of the dedicated diagnostic model for fault classification. The fused feature vector is input into the fully connected layer, and the probability distribution of the fault type is output through the Softmax function. The Softmax function maps the feature vector to the probability space. The probability of each category represents the possibility that the sample belongs to that fault type. The category with the highest probability is selected as the predicted fault type to achieve fault classification.

8. The method for multimodal fault diagnosis of new energy sources using dynamic graph convolution and transfer learning according to claim 1, characterized in that, The cross-modal fusion layer fuses spatial and temporal features by splicing or weighted summation to merge the outputs of the spatial and temporal branches.

9. The method for multimodal fault diagnosis of new energy sources using dynamic graph convolution and transfer learning according to claim 1, characterized in that, The online learning mechanism dynamically updates the graph structure by minimizing the feedback loss function and updating the edge weight matrix. L g =λ1·CE(y pred ,y true )+λ2·||ΔE(t)||2 L g The graph structure feedback loss function is given by λ1, where λ1 is the balance coefficient of the cross-entropy loss term, and CE(y) is the graph structure feedback loss function. pred ,y true Predicting fault category y pred Compared with the actual fault category y true The cross-entropy loss between them, λ2 is the balance coefficient of the norm term of the edge weight change, ΔE(t) is the change of the edge weight matrix E(t) at the current time, and ||ΔE(t)||2 is the L2 norm of the edge weight change ΔE(t).

10. The method for multimodal fault diagnosis of new energy sources using dynamic graph convolution and transfer learning according to claim 1, characterized in that, The learned fault evolution model constructs a fault evolution index by fusing fault prediction probability and attention weight change rate: FEI=β·P(fault)+(1-β)·Rate(A v ) Where FEI is the fault evolution index, β is the fusion coefficient, P(fault) is the probability of fault occurrence, 1-β is the weighting coefficient related to the fusion coefficient, and Rate(A) is the probability of fault occurrence. v ) represents the rate of change of attention weights for graph nodes.

Images