Offshore wind turbine fault diagnosis method based on multi-modal fusion and physical information guidance

Abstract

This invention discloses a fault diagnosis method and system based on a fault diagnosis model for key components of offshore wind turbines. The invention includes: constructing a fault diagnosis model for key components of offshore wind turbines, the model comprising a multi-source data coupling layer, a feature extraction layer, and a fault diagnosis layer; acquiring multi-source operation and maintenance data such as meteorological data, turbine SCADA parameters, and inspection images; utilizing the multi-source data coupling layer to perform spatiotemporal alignment and adaptive weighted fusion using dynamic time warping and entropy weighting to construct a three-dimensional fusion dataset; utilizing the feature extraction layer to extract deep features through 3D convolutional neural networks, graph neural networks, and Transformer cross-modal attention mechanisms, and obtaining a physical-data dual-driven feature vector through feature distillation compression and generative adversarial network small-sample enhancement; utilizing the fault diagnosis layer to embed failure partial differential equations as regularization terms through a physical information neural network, combining adaptive sliding windows and gated memory units to capture temporal features, and performing multi-task classification using a lightweight gradient booster to output fault diagnosis results. This invention improves the diagnostic accuracy of early, subtle faults through physical information-guided multimodal fusion.

Description

Technical Field

[0001] This invention relates to the field of wind power equipment condition monitoring and fault diagnosis technology, specifically to a fault diagnosis method and system based on a fault diagnosis model of key components of offshore wind turbines. Background Technology

[0002] With the continuous increase in offshore wind power installed capacity, intelligent fault diagnosis technology has become a key support in the construction of new power systems and the zero-carbon energy strategy. Traditional wind turbine fault diagnosis models are mostly based on linear relationships between single SCADA data and shallow statistical features, passively responding to explicit faults through threshold triggering or simple classifiers (such as SVM and random forests). Few methods integrate information from maintenance and inspection images, multi-source electromechanical sensors, and marine weather forecasts to construct cross-modal correlations for early latent fault prediction. In reality, progressive faults such as offshore wind turbine blade corrosion and gearbox bearing wear have weak early signs and are strongly coupled with the marine environment (e.g., salt spray corrosion accelerates cracking, and typhoon turbulence induces abnormal vibrations). Existing methods struggle to capture the physical correlations between multi-source data, leading to high false alarm rates and short warning windows.

[0003] Therefore, there is an urgent need in this field to construct a fault evolution characterization system that integrates meteorology, generator set, and imagery. Based on multimodal coupling features, this system should make reasonable use of physical mechanism constraints and the generalization ability of deep learning to effectively improve the accuracy of early fault diagnosis under complex operating conditions. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a fault diagnosis method and system based on a fault diagnosis model for key components of offshore wind turbines.

[0005] In a first aspect, the present invention provides a fault diagnosis method based on a fault diagnosis model of key components of offshore wind turbines, comprising the following steps:

[0006] S1: Construct a fault diagnosis model for key components of offshore wind turbines, the model including:

[0007] The multi-source data coupling layer is configured to perform spatiotemporal alignment and adaptive weighted fusion of multi-source operation and maintenance data using dynamic time warping and entropy weighting.

[0008] The feature extraction layer is configured to perform deep feature extraction on the fused data using a cross-modal collaborative learning framework, and to perform feature distillation compression and generative adversarial network few-shot enhancement. The cross-modal collaborative learning framework includes a 3D convolutional neural network, a graph neural network, and a Transformer cross-modal attention mechanism.

[0009] The fault diagnosis layer is configured to use a physical information neural network to embed the component failure partial differential equation as a physical regularization term, employ an adaptive spatiotemporal sliding window and gated memory unit to capture temporal features, and perform multi-task classification through a lightweight gradient booster.

[0010] S2: Acquire multi-source operation and maintenance data of offshore wind turbines, including meteorological data, turbine SCADA parameters, and inspection images;

[0011] S3: Utilize the multi-source data coupling layer to perform spatiotemporal alignment and adaptive weighted fusion of the multi-source operation and maintenance data to construct a three-dimensional fusion dataset of meteorology, generator set, and image.

[0012] S4: The feature extraction layer is used to perform deep feature extraction, feature distillation compression, and small sample enhancement on the three-dimensional fusion dataset to obtain a physical-data dual-driven feature vector.

[0013] S5: Utilize the fault diagnosis layer to perform physical constraint-enhanced feature extraction, dynamic segmentation, and multi-task classification on the physical-data dual-driven feature vector, and output the fault diagnosis result.

[0014] Secondly, the present invention provides a fault diagnosis system for key components of offshore wind turbines, comprising:

[0015] The multi-source data coupling module is configured to acquire multi-source operation and maintenance data of offshore wind turbines, including meteorological data, turbine SCADA parameters, and inspection images. The multi-source data coupling module is also configured to perform spatiotemporal alignment and adaptive weighted fusion of the multi-source operation and maintenance data using dynamic time warping and entropy weighting to construct a three-dimensional fusion dataset of meteorology, turbine, and images.

[0016] The feature extraction module, connected to the multi-source data coupling module, is configured to perform deep feature extraction on the 3D fusion dataset using a cross-modal collaborative learning framework, which includes a 3D convolutional neural network, a graph neural network, and a Transformer cross-modal attention mechanism. The feature extraction module is also configured to perform feature distillation compression on the extracted features and use a generative adversarial network for few-sample enhancement to obtain a physical-data dual-driven feature vector.

[0017] The fault diagnosis module, connected to the feature extraction module, is configured to process the physical-data dual-driven feature vector using a physical information neural network, embedding the component failure partial differential equation into the loss function of the physical information neural network as a physical regularization term to obtain physical constraint enhancement features; the fault diagnosis module is also configured to dynamically segment the physical constraint enhancement features using an adaptive spatiotemporal sliding window, and capture long-term temporal dependencies through a gated memory unit to obtain fault temporal features; and input the fault temporal features into a lightweight gradient booster, perform composite fault classification through multi-task collaborative training, and output fault diagnosis results.

[0018] Beneficial effects:

[0019] This invention constructs a fault diagnosis model for key components of offshore wind turbines, comprising a multi-source data coupling layer, a feature extraction layer, and a fault diagnosis layer. It introduces a collaborative paradigm of three-dimensional fusion of meteorology, turbine, and image data, along with physical mechanism embedding, significantly improving the accuracy of composite fault diagnosis and early warning capabilities. An adaptive fusion method combining dynamic time warping and entropy weighting solves the feature shift problem caused by spatiotemporal scale mismatch in traditional single-source data analysis. A cross-modal collaborative learning algorithm is used for in-depth mining of heterogeneous features, and 3D-CNN spatiotemporal feature extraction and GNN topological relationship modeling address the poor ability of traditional single models to capture the correlation of multi-source data. A physical embedding diagnostic architecture combining PINN and LightGBM is employed, using PDE physical constraint embedding and adaptive spatiotemporal sliding windows to reduce false alarm rates caused by operating condition fluctuations. A gated memory unit is used to capture early, subtle signs of faults, enhancing the early warning capability for latent faults. Attached Figure Description

[0020] Figure 1 This is a flowchart of the fault diagnosis method based on a fault diagnosis model of key components of offshore wind turbines, as described in one embodiment of the present invention.

[0021] Figure 2 This is a schematic diagram of the structure of the fault diagnosis system for key components of the offshore wind turbine, as shown in another embodiment of the present invention.

[0022] Figure 3 The flowchart below shows a method for fault diagnosis model of key components of offshore wind turbine based on multimodal fusion and physical information guidance, as described in another embodiment of the present invention. Detailed Implementation

[0023] The following is in conjunction with the appendix Figures 1 to 3 The embodiments and examples will be described in further detail below. The following examples are for illustrative purposes only and should not be construed as limiting the scope of the invention.

[0024] In the description of this invention, unless otherwise stated, "a plurality of" means two or more; the terms "upper," "lower," "left," "right," "inner," "outer," "front end," "rear end," "head," "tail," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, the terms "first," "second," "third," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0025] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "connected" and "linked" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0026] See Figure 1 In one embodiment, a fault diagnosis method based on a fault diagnosis model of key components of offshore wind turbines is provided, comprising the following steps:

[0027] S1: Construct a fault diagnosis model for key components of offshore wind turbines, the model including:

[0028] The multi-source data coupling layer is configured to perform spatiotemporal alignment and adaptive weighted fusion of multi-source operation and maintenance data using dynamic time warping and entropy weighting.

[0029] The feature extraction layer is configured to perform deep feature extraction on the fused data using a cross-modal collaborative learning framework, and to perform feature distillation compression and generative adversarial network few-shot enhancement. The cross-modal collaborative learning framework includes a 3D convolutional neural network, a graph neural network, and a Transformer cross-modal attention mechanism.

[0030] The fault diagnosis layer is configured to use a physical information neural network to embed the component failure partial differential equation as a physical regularization term, employ an adaptive spatiotemporal sliding window and gated memory unit to capture temporal features, and perform multi-task classification through a lightweight gradient booster.

[0031] S2: Acquire multi-source operation and maintenance data of offshore wind turbines, including meteorological data, turbine SCADA parameters, and inspection images;

[0032] S3: Utilize the multi-source data coupling layer to perform spatiotemporal alignment and adaptive weighted fusion of the multi-source operation and maintenance data to construct a three-dimensional fusion dataset of meteorology, generator set, and image.

[0033] S4: The feature extraction layer is used to perform deep feature extraction, feature distillation compression, and small sample enhancement on the three-dimensional fusion dataset to obtain a physical-data dual-driven feature vector.

[0034] S5: Utilize the fault diagnosis layer to perform physical constraint-enhanced feature extraction, dynamic segmentation, and multi-task classification on the physical-data dual-driven feature vector, and output the fault diagnosis result.

[0035] The core of this invention lies in constructing a physical information-guided multimodal fusion model. By embedding physical mechanisms into the data-driven model through a three-layer structure, it solves the problem that pure data-driven methods ignore the physical laws of fault evolution.

[0036] In another embodiment, the multi-source data coupling layer in step S1 is configured to: use SCADA parameters as the reference mode, and achieve alignment of meteorological data, SCADA parameters and inspection images in the time dimension through a physical constraint dynamic time warping algorithm, the cost function of which includes a power-wind speed monotonicity constraint term.

[0037] In this embodiment, the traditional DTW is improved by introducing physical constraints (power-wind speed monotonicity), so that the alignment result not only meets the timing matching, but also conforms to the aerodynamic characteristics of the wind turbine.

[0038] In another embodiment, the multi-source data coupling layer in step S1 is further configured to: perform feature standardization on the aligned multi-source data, calculate the physical information entropy of each feature dimension, allocate fusion weights based on the entropy weight method, and perform joint optimization through a physical constraint loss function containing aerodynamic load equations.

[0039] In this embodiment, the entropy weight method objectively determines the weights of each feature, avoiding subjective weighting bias; the physical constraint loss function guides the fusion process under the guidance of the aerodynamic load equation.

[0040] In another embodiment, in the cross-modal collaborative learning framework of the feature extraction layer in step S1: the 3D-CNN is configured to extract the spatiotemporal features of the inspection image sequence; the GNN is configured to model the topological relationship between the components of the wind turbine; and the Transformer cross-modal attention mechanism is configured to fuse the temporal features of meteorological data with SCADA parameters, wherein the attention weight matrix is ​​superimposed with a physical guidance mask.

[0041] In this embodiment, the physical topology between components is explicitly modeled using GNN, enabling the model to capture the cross-component propagation characteristics of offshore wind turbine faults; physical masks focus attention on operating conditions that conform to aerodynamic laws.

[0042] In another embodiment, when the feature extraction layer performs feature distillation compression in step S1, a knowledge distillation method is used, with the multimodal fusion network formed by the cross-modal collaborative learning framework as the teacher network and the lightweight feature extraction network as the student network; the generative adversarial network uses physical condition vectors as the input of the generator to generate synthetic fault features to expand the small sample fault categories.

[0043] In this embodiment, knowledge distillation transfers knowledge from complex models to lightweight models, making them easier to deploy at the edge; physical condition GAN generates fault samples of offshore wind turbines that conform to physical laws.

[0044] In another embodiment, in the physical information neural network of the fault diagnosis layer in step S1, the component failure partial differential equation includes the blade crack propagation equation, the bearing wear evolution equation, or the gearbox tooth surface fatigue equation; the loss function of the physical information neural network includes the observation data fitting term and the partial differential equation residual term.

[0045] In this embodiment, by embedding specific failure PDEs, the features learned by the neural network satisfy the physical evolution law of key components of offshore wind turbines.

[0046] In another embodiment, the window size of the adaptive spatiotemporal sliding window of the fault diagnosis layer in step S1 is dynamically adjusted according to the real-time speed fluctuation rate; the gated memory unit includes a reset gate and an update gate, the reset gate is configured to filter operating condition noise, and the update gate is configured to enhance early fault memory.

[0047] In this embodiment, the window adapts to fluctuations in operating conditions to avoid false alarms of offshore wind turbine component failures when the fixed window is used during drastic changes in operating conditions; the GMU filters out noise by resetting the gate and retains the memory of key faults of offshore wind turbine components by updating the gate.

[0048] In another embodiment, the objective function of the multi-task collaborative training of the lightweight gradient booster of the fault diagnosis layer in step S1 is the sum of the weighted and logarithmic losses of multiple fault diagnosis tasks, and includes an L1 regularization term; the composite fault includes at least two of blade cracks, bearing wear and gearbox failure.

[0049] In this embodiment, multi-task learning leverages inter-task correlations to enhance the generalization capability of the fault diagnosis model for key components of offshore wind turbines, while L1 regularization promotes sparse feature selection.

[0050] In another embodiment, a fault diagnosis system based on a fault diagnosis model of key components of offshore wind turbines is provided, comprising:

[0051] The multi-source data coupling module is configured to acquire multi-source operation and maintenance data of offshore wind turbines, including meteorological data, turbine SCADA parameters, and inspection images. The multi-source data coupling module is also configured to perform spatiotemporal alignment and adaptive weighted fusion of the multi-source operation and maintenance data using dynamic time warping and entropy weighting to construct a three-dimensional fusion dataset of meteorology, turbine, and images.

[0052] The feature extraction module, connected to the multi-source data coupling module, is configured to perform deep feature extraction on the 3D fusion dataset using a cross-modal collaborative learning framework, which includes a 3D convolutional neural network, a graph neural network, and a Transformer cross-modal attention mechanism. The feature extraction module is also configured to perform feature distillation compression on the extracted features and use a generative adversarial network for few-sample enhancement to obtain a physical-data dual-driven feature vector.

[0053] The fault diagnosis module, connected to the feature extraction module, is configured to process the physical-data dual-driven feature vector using a physical information neural network, embedding the component failure partial differential equation into the loss function of the physical information neural network as a physical regularization term to obtain physical constraint enhancement features; the fault diagnosis module is also configured to dynamically segment the physical constraint enhancement features using an adaptive spatiotemporal sliding window, and capture long-term temporal dependencies through a gated memory unit to obtain fault temporal features; and input the fault temporal features into a lightweight gradient booster, perform composite fault classification through multi-task collaborative training, and output fault diagnosis results.

[0054] In another embodiment, the feature extraction module includes a 3D-CNN unit, a GNN unit, and a Transformer unit. The outputs of each unit are fused by a gated memory unit to obtain a physical-data dual-driven feature vector.

[0055] In this embodiment, GMU dynamically weights and fuses multi-branch features, retaining their respective advantages and improving the feature discrimination of fault diagnosis for key components of offshore wind turbines.

[0056] In another embodiment, the present invention proposes a fault diagnosis model for key components of offshore wind turbines based on multimodal fusion and physical information guidance. The overall framework consists of the following parts:

[0057] Multi-source data coupling layer: Through spatiotemporal alignment and adaptive weighted fusion technology of multi-source data, dynamic time warping (DTW) and entropy weighting are used to fuse meteorological data, unit SCADA parameters and inspection images to construct a three-dimensional fusion dataset of meteorology-unit-image, effectively eliminating intermodal heterogeneity and enhancing data physical consistency, providing high-dimensional and low-noise input data for subsequent feature extraction layers.

[0058] Feature Extraction Layer: Through cross-modal collaborative learning framework technology, using 3D convolutional neural networks (3D-CNN), graph neural networks (GNN), and Transformer cross-modal attention mechanism, it extracts spatiotemporal features of inspection image sequences, models the topological relationships of unit components, and fuses meteorological time-series features with SCADA parameters. Furthermore, it combines feature distillation compression and generative adversarial network (GAN) few-shot enhancement technology to efficiently integrate multi-source heterogeneous features, improve the quality and discriminativeness of feature representation, and provide high-quality physical-data dual-driven feature vectors for subsequent fault diagnosis layers.

[0059] Fault Diagnosis Layer: Utilizing a collaborative paradigm of physical mechanism and data-driven approaches, this layer employs Physical Information Neural Network (PINN) and Lightweight Gradient Boosting Machine (LightGBM) to embed the physical constraints constructed from component failure mechanisms (partial differential equations, PDEs) into the network loss function (physical regularization term). Adaptive spatiotemporal sliding windows and gated memory units (GMUs) are used to process real-time fused data, capturing early, subtle signs of faults and resolving false alarms due to operating condition fluctuations. This improves the accuracy, robustness, and early warning capabilities of fault diagnosis. Finally, a multi-task collaborative training mechanism enables simultaneous diagnosis of complex faults in key components such as blade cracks, bearing wear, and gearbox failure, achieving accurate assessment and prediction of the health status of critical components in offshore wind turbines. This part inputs the fused physical-data feature vector into the model and utilizes the physical constraints of PINN to extract fault evolution characteristics. Finally, it combines this with the efficient discrimination capabilities of LightGBM for fault classification and diagnosis.

[0060] For example, the system can be integrated into a computing device. This computing device includes at least one processor (CPU), at least one graphics processing unit (GPU) or neural network processor (NPU), and memory. The memory stores computer program instructions that, when executed by the processor, cause the computing device to implement the functions of the multi-source data coupling module, the feature extraction module, and the fault diagnosis module. Specifically, algorithms such as dynamic time warping and entropy weighting in the multi-source data coupling module can run on the CPU; deep learning models such as 3D-CNN, GNN, and Transformer in the feature extraction module can be accelerated on the GPU or NPU; and the PINN and LightGBM models in the fault diagnosis module can be executed on the CPU or GPU.

[0061] Furthermore, the three modules can be deployed on different hardware nodes and connected via network communication. For example, the multi-source data coupling module is deployed on the wind farm edge gateway (equipped with a low-power CPU and memory) for data preprocessing and fusion; the feature extraction module is deployed on the wind farm local server (equipped with a high-performance GPU) for feature extraction and enhancement; and the fault diagnosis module is deployed on a remote cloud server (equipped with large-capacity storage and a high-performance computing cluster) for fault diagnosis and model updates. The nodes communicate with each other via industrial Ethernet or 5G.

[0062] Furthermore, for scenarios requiring real-time response, the lightweight student network in the feature extraction module and the LightGBM model in the fault diagnosis module can be deployed on embedded devices (such as NVIDIA Jetson series or Raspberry Pi with neural network accelerator sticks) to run directly in offshore substations or wind turbine nacelles, outputting diagnostic results. The multi-source data coupling module can use FPGA to implement hardware acceleration for DTW and Kalman filtering.

[0063] The following is a more detailed example:

[0064] In another embodiment, the main flowchart of the fault diagnosis method based on the fault diagnosis model of key components of offshore wind turbines is as follows: Figure 3As shown, firstly, a multi-source data spatiotemporal alignment and adaptive weighted fusion method is adopted. Dynamic Time Warping (DTW) and entropy weighting are used to process meteorological data, unit SCADA parameters, and inspection images using 3D fusion technology to eliminate modal heterogeneity, enhance physical consistency, and construct a meteorological-unit-image fusion dataset. Next, 3D-CNN, Graph Neural Network (GNN), and Transformer cross-modal attention mechanism are used to extract deep features from the fusion dataset. Feature distillation compression and GAN few-shot enhancement are used to achieve efficient integration and feature enhancement, resulting in high-quality, highly discriminative physical-data dual-driven feature vectors as input to the fault diagnosis model. Finally, Physical Information Neural Network (PINN) and Lightweight Gradient Boosting Machine (LightGBM) are used, combined with adaptive spatiotemporal sliding window, gated memory unit (GMU), and multi-task collaborative training mechanism, to complete the synchronous diagnosis and health status prediction of complex faults in key components such as blade cracks, bearing wear, and gearbox failure.

[0065] This invention employs Dynamic Time Warping (DTW) and entropy weighting-mutual information weighted fusion to perform spatiotemporal alignment and adaptive fusion processing on multi-source heterogeneous data. In the processing of multi-source heterogeneous wind power data, traditional feature fusion methods, due to neglecting spatiotemporal scale differences and dynamic changes in feature weights, struggle to eliminate intermodal heterogeneity, leading to insufficient physical consistency. Therefore, this invention proposes a three-dimensional dynamic weighted fusion method based on tensor decomposition. The specific implementation process and mathematical principles are as follows:

[0066] 1. Spatiotemporal alignment module:

[0067] To address the differences in sampling frequencies among meteorological data, SCADA parameters, and inspection images, cross-modal alignment is achieved using SCADA data as the baseline mode through an improved physical constraint DTW algorithm.

[0068] Define the normalized path cost function:

[0069]

[0070]

[0071] Image sequence alignment:

[0072]

[0073]

[0074] in,

[0075] It is a time warping function. To smooth the regularization term, Power-wind speed monotonicity constraint;

[0076] For 3D convolutional feature extraction networks, For image feature matrix, This is the weight matrix. For function composition operators;

[0077] Indicates time SCADA reference data, This indicates the characteristics of meteorological data or images after time normalization. The first-order difference represents the regularized path; P represents the output power of the wind turbine. The original image sequence; It is the optimal regularization function.

[0078] 2. Adaptive Weighted Fusion Module:

[0079] (1) Feature standardization:

[0080]

[0081] In the formula, Let i be the element in the i-th row and j-th column of matrix M. Let be all time series values ​​of the j-th dimension feature.

[0082] (2) Information entropy calculation:

[0083]

[0084] In the formula, Let be the physical information entropy of the j-th dimension feature.

[0085] (3) Entropy weight allocation:

[0086]

[0087] In the formula, Let be the fusion weight of the j-th dimension feature.

[0088] (4) Physical constraint loss function:

[0089]

[0090]

[0091] In the formula, This is the balance coefficient for the loss term. For aerodynamic load equations, For time difference operators, L1 norm; R is the radius of the wind turbine. The wind energy utilization coefficient, For the tip speed ratio, The pitch angle; This is a balancing coefficient used to adjust the influence of the image feature fusion term; , These represent the characteristics of meteorological data and SCADA data after spatiotemporal alignment, respectively. The features of the fused image; This is the data for the m-th mode; Modal weights; air density; This refers to wind speed.

[0092] 3. Dynamic noise reduction and output

[0093] (1) Kalman filter model:

[0094]

[0095]

[0096]

[0097] In the formula, Let A be the state vector at time k, A be the state transition matrix, and B be the control input matrix. For example, A can be set as the identity matrix plus the time step multiplied by the system dynamics approximation Jacobian matrix, and B can be obtained by identifying historical data.

[0098] (2) Dataset construction:

[0099]

[0100]

[0101] In the formula, Meteorological modal characteristics, For the unit's SCADA modal characteristics, For image modal features, The threshold is used for entropy weighted filtering.

[0102] This invention generates three-dimensional fused data through physical constraint time alignment and joint optimization of entropy weight-physical loss, providing high-fidelity input for fault diagnosis.

[0103] This invention employs 3D-CNN, Graph Neural Networks (GNN), and Transformer cross-modal attention mechanisms to extract features from fused data. In the multimodal feature learning process, traditional single-modal models, due to neglecting cross-modal correlations and physical topological constraints, struggle to capture the cross-component propagation characteristics of faults, resulting in weak fault features being overwhelmed by noise. Therefore, this invention proposes a physically guided cross-modal collaborative learning framework. Its implementation process and mathematical principles are as follows:

[0104] 1. Multi-branch feature extraction

[0105] (1) Spatiotemporal feature extraction of inspection images (3D-CNN):

[0106]

[0107] In the formula, 3D convolution kernel, Represents an image sequence. This is the activation function.

[0108] (2) Generator topology modeling (GNN):

[0109]

[0110] In the formula, v represents the component node. For adjacent nodes, This is a learnable weight matrix.

[0111] (3) Meteorological-SCADA cross-modal fusion (Transformer):

[0112]

[0113] In the formula, For meteorological data matrix, This is the SCADA data key matrix. For physical boot mask.

[0114] 2. Feature Enhancement Module

[0115] (1) Characteristic distillation compression (knowledge distillation):

[0116]

[0117] (2) GAN small sample enhancement:

[0118]

[0119] In the formula, This is the physical condition vector.

[0120] 3. Cross-modal collaborative output

[0121] (1) Dual-drive feature vector generation:

[0122]

[0123] (2) Gated Memory Unit (GMU) Fusion:

[0124]

[0125] In the formula, For the gated vector, For Hadamard products.

[0126] This invention generates a dual-drive feature vector with strong discriminative power through cross-modal fusion with physical constraints, feature distillation, and physical conditional GAN, providing high-precision input for the fault diagnosis layer.

[0127] Furthermore, this invention employs a Physical Information Neural Network (PINN) and a Lightweight Gradient Boosting Machine (LightGBM) for fault diagnosis of feature vectors. During fault diagnosis, traditional data-driven models, by neglecting physical mechanism constraints, struggle to capture fault evolution patterns, leading to early fault underreporting and false alarms due to operating condition fluctuations. Therefore, this invention proposes a physical-data dual-drive collaborative diagnostic paradigm. Its implementation process and mathematical principles are as follows:

[0128] 1. Physically Constrained Feature Extraction (PINN)

[0129] (1) Construction of physical information neural network:

[0130]

[0131] (2) Loss function with embedded failure mechanism:

[0132]

[0133] In the formula, For the observed state vector, For differential operators, This is the partial differential equation for component failure.

[0134] 2. Dynamic operating condition adaptation module

[0135] (1) Adaptive spatiotemporal sliding window:

[0136] Window size adjusts dynamically:

[0137]

[0138] In the formula, For real-time speed fluctuation rate, As the base window size, This represents the rotational speed gradient.

[0139] (2) Gated Memory Unit (GMU):

[0140]

[0141]

[0142]

[0143]

[0144] 3. Multi-task collaborative diagnosis (LightGBM)

[0145] (1) Physical-feature fusion input:

[0146]

[0147] (2) Multi-task objective function:

[0148]

[0149] (3) Fault decision rules:

[0150]

[0151] In the formula, This is the kurtosis coefficient. To measure the meshing frequency, The theoretical meshing frequency, For logic AND operators; The partial derivative of the physical information neural network output with respect to time t is used to characterize the fault evolution rate. , , These are the predicted probabilities of blade, bearing, and gear failures, respectively. This is an empirical threshold.

[0152] This invention constructs a physical regularization loss by embedding a physical information neural network (PINN) into the component failure partial differential equation (PDE), combines it with an adaptive spatiotemporal sliding window to dynamically segment feature sequences, utilizes a gated memory unit (GMU) to capture the fault evolution law, and finally uses a multi-task LightGBM to fuse physical constraint features and data features to achieve synchronous diagnosis of composite faults.

[0153] See further Figure 3 , Figure 3 The process shown includes three core processing layers: multi-source data processing layer, dynamic feature selection layer, and composite fault diagnosis layer. The execution order and data flow of the internal modules of each layer are shown in the figure.

[0154] It should be noted that, Figure 3 In the specific implementation of this invention, the “CNN+GNN” module corresponds to a parallel structure of 3D-CNN (extracting spatiotemporal features of images) and GNN (modeling the topological relationship of components). After the meteorological-SCADA features are fused by the Transformer cross-modal attention mechanism, they are further fused by the gated memory unit (GMU) to finally output the feature vector. Figure 3 The "Feature Distillation and Dimension Compression" and "GAN Enhanced Samples" modules are auxiliary techniques used to optimize the feature extraction layer; the "PINN Embedding Physical Constraint Equations" module is an exemplary implementation of a physical information neural network; the "Adaptive Window Capturing Temporal Features" and "GMU Handling Long-Term Dependencies" modules jointly realize dynamic temporal modeling of the fault diagnosis layer; and the "LightGBM Multi-Task Classification" module outputs the final diagnostic results.

[0155] Three core levels:

[0156] 1. Multi-Source Data Processing Layer: The input is raw multi-source operation and maintenance data, which sequentially passes through "Multi-Source Data Input" → "Dynamic Time Warping" → "Entropy Weighting" → "Mechatronics-Image-Atmosphere Multi-Source Data Fusion". The modules within this layer are interconnected. Dynamic Time Warping (DTW) achieves time alignment, entropy weighting calculates the weights of each feature, and Kalman filtering (implicit in "Multi-Source Data Fusion") performs dynamic noise reduction. The 3D fused dataset output from this layer serves as the input for the next layer.

[0157] 2. Dynamic Feature Selection Layer: This layer sequentially passes through "CNN+GNN" → "Transformer Fusion" → "Feature Distillation and Dimensional Compression" → "GAN Enhancement of Samples". To meet the requirement of sufficient disclosure, this invention specifies "CNN+GNN" as follows: 3D-CNN and GNN are connected in parallel to extract spatiotemporal features and component topological features of the image. Then, these two features are fused with the meteorological-SCADA cross-modal features output by the Transformer through a gated memory unit (GMU) to obtain the final feature vector. Therefore, the actual internal structure of this layer is: parallel branches (3D-CNN, GNN, Transformer) → GMU fusion → feature distillation → GAN enhancement. Figure 3 The terms "feature distillation compression dimension" and "GAN augmented sample" are simplified representations of the above process.

[0158] 3. Composite Fault Diagnosis Layer: This layer sequentially passes through "PINN embedding physical constraint equations" → "adaptive window capturing temporal features" → "GMU processing long-term dependencies" → "LightGBM multi-task classification." Internally, the process is sequential. PINN outputs enhanced physical constraint features, the adaptive sliding window performs dynamic segmentation, the GMU models temporal dependencies, and LightGBM outputs the classification results.

[0159] This invention employs a standard two-stage training strategy:

[0160] Phase 1: Sub-model Pre-training. 3D-CNN, GNN, Transformer, PINN, GAN, and LightGBM are pre-trained on their respective sub-tasks (e.g., 3D-CNN for image classification, PINN for PDE solving) to determine initial parameters. During this phase, each sub-model has its own hyperparameters (e.g., learning rate, number of epochs, optimizer), which are tuned independently. After pre-training, the parameters of each sub-model are used as the initial weights for the overall model.

[0161] Phase Two: End-to-End Joint Training. The pre-trained sub-models are connected according to the overall structure described above to construct a complete fault diagnosis model. Then, the end-to-end joint loss function is used:

[0162] in, This is the multi-task loss of LightGBM, but the LightGBM parameters can be fixed; For knowledge distillation loss; The physical constraint loss for PINN; hyperparameters of the overall model (such as global learning rate, batch size, optimizer, loss weights). , (etc.) The parameters are uniformly determined through grid search or Bayesian optimization on the validation set. Finally, a global model with determined parameters is obtained.

[0163] Through the two-stage strategy described above, the hyperparameters of the overall model are uniquely determined during end-to-end training, while pre-training only provides initial parameters.

[0164] The following details the specific structure of each AI model and the method for determining hyperparameters during the pre-training phase. After pre-training, these parameters will be further fine-tuned during end-to-end training:

[0165] 1. 3D Convolutional Neural Network (3D-CNN)

[0166] Structure: Input Image sequence (T=16, H=224, W=224, C=3), consisting of: 3D convolutional layer (kernel=3×3×3, stride=1, padding=1, out=32, ReLU), 3D max pooling (2×2×2, stride=2), 3D convolutional layer (out=64, ReLU), 3D pooling (2×2×2), 3D convolutional layer (out=128, ReLU), global average pooling → outputting a 128-dimensional feature vector. .

[0167] Pre-trained hyperparameters: Optimizer SGD (lr=0.001, momentum=0.9, weight_decay=1e-4), loss function cross-entropy, training for 100 epochs, batch size 32, early stopping (patience=10). Hyperparameters were determined on the image classification validation set via grid search.

[0168] 2. Graph Neural Networks (GNN)

[0169] Structure: A two-layer Graph Convolutional Network (GCN). The input node feature dimensions are determined based on component sensor data, e.g., 16 dimensions. The first GCN layer outputs 128 dimensions, the second layer outputs 64 dimensions, and finally, global average pooling is used to obtain graph-level features. .

[0170] Pre-training hyperparameters: Optimizer Adam (lr=0.001), loss function node classification cross-entropy, training for 50 epochs, batch size 16. Graph topology is fixed (Note: determined by wind turbine design drawings).

[0171] 3. Transformer cross-modal attention mechanism

[0172] Structure: Single-head attention (can be expanded to 4 heads in practice), 128 embedding dimensions, 256 FFN hidden layers, layer normalization + residual connections. Input sequence length 30 (time points).

[0173] Pre-trained hyperparameters: Pre-trained on the wind speed prediction task (mean squared error loss), optimizer Adam (lr=1e-4, betas=(0.9,0.98)), trained for 50 epochs.

[0174] 4. Gated Memory Unit (GMU)

[0175] Structure: The formula is as described above, with input feature dimension (the dimension after concatenating the outputs of the three branches) and hidden state dimension of 64.

[0176] Pre-training: The GMU is not pre-trained separately; its parameters are learned from scratch during end-to-end training.

[0177] 5. Generative Adversarial Networks (GANs)

[0178] Structure: Generator: Three fully connected layers (110→256→512→128), ReLU+Dropout(0.3) after each layer; Discriminator: Two fully connected layers (128→256→1), LeakyReLU(0.2)+Dropout(0.3), output Sigmoid.

[0179] Pre-training hyperparameters: WGAN-GP loss, optimizer Adam (lr=1e-4), gradient penalty coefficient 10, alternating training (5 steps for discriminator, 1 step for generator), 2000 training rounds.

[0180] 6. Physical Information Neural Network (PINN)

[0181] Structure: Input spatial coordinates x (2D) and time t (1D), 4 hidden layers × 100 neurons, activation function tanh, output physical field u (5D).

[0182] Pre-training hyperparameters: loss function (Data fitting term + PDE residual term, weight) The optimizer L-BFGS was trained for 500 epochs.

[0183] 7. Lightweight Gradient Booster (LightGBM)

[0184] Structure: Tree model, Parameters: n_estimators=100, max_depth=5, num_leaves=31, learning_rate=0.05.

[0185] Pre-trained hyperparameters: Bayesian optimization, objective function multi-task log loss, 5-fold cross-validation.

[0186] 8. Knowledge Distillation (Teacher Network, Student Network)

[0187] Teacher network: a pre-trained 3D-CNN+GNN+Transformer+GMU system (approximately 2e7 parameters).

[0188] Student network: two convolutional layers (32 / 64 channels, stride=2) + two fully connected layers (128→64), with approximately 4e6 parameters.

[0189] Distillation training: Fixed teacher network, temperature T=5, loss The optimizer Adam (lr=1e-4) was trained for 50 epochs.

[0190] After completing the pre-training of each sub-model, according to Figure 3The overall model is constructed using the connection methods described above. The hyperparameters (global) for end-to-end training are uniformly determined in the following way:

[0191] Search space: learning rate {1e-5, 5e-5, 1e-4, 5e-4}, batch size {16, 32, 64}, optimizer {Adam, SGD}, loss weights , .

[0192] Validation method: The average accuracy and F1 score of composite fault diagnosis are calculated on the independent validation set, and the optimal combination of hyperparameters is selected by Bayesian optimization (50 iterations).

[0193] Typical optimal values: learning rate 1e-4, batch size 32, optimizer Adam. , .

[0194] Training termination condition: If the validation set loss does not decrease for 10 consecutive rounds, training will stop early, with a maximum of 200 rounds.

[0195] After the end-to-end training described above, all trainable parameters of the overall model (including some layers of 3D-CNN, GNN, Transformer, GMU, and PINN) are finally determined, while the LightGBM parameters can remain unchanged or only the input dimension can be adjusted.

[0196] In summary, this invention constructs a fault diagnosis model for key components of offshore wind turbines. It performs spatiotemporal alignment and adaptive weighted fusion of meteorological data, turbine SCADA parameters, and inspection images. Deep features are extracted using 3D-CNN, GNN, and Transformer cross-modal attention mechanisms. PINN is embedded into the component failure partial differential equation as a physical regularization term. An adaptive sliding window and GMU are combined to capture early, subtle signs. Finally, LightGBM is employed to achieve simultaneous diagnosis of multi-task composite faults. This invention breaks away from the bias of traditional purely data-driven diagnostic methods that neglect physical mechanisms. Through a physical-data dual-driven collaborative paradigm, it significantly improves the early warning capability and diagnostic accuracy of early faults in complex marine environments, reduces the false alarm rate caused by operating condition fluctuations, and has significant engineering application value.

[0197] Although the preferred embodiments of the present invention have been described in detail above with reference to the accompanying drawings, the present invention is not limited to the specific embodiments described above. Those skilled in the art should understand that various modifications, equivalent substitutions, combinations, or variations can be made to the above embodiments based on the concept and essence of the technical solution of the present invention, and such modifications and variations still fall within the scope of protection claimed by the present invention, as long as they do not depart from the spirit and principles of the present invention.

Claims

1. A fault diagnosis method based on a fault diagnosis model of key components of offshore wind turbines, characterized in that, Includes the following steps: S1: Construct a fault diagnosis model for key components of offshore wind turbines, the model including: The multi-source data coupling layer is configured to perform spatiotemporal alignment and adaptive weighted fusion of multi-source operation and maintenance data using dynamic time warping and entropy weighting. The feature extraction layer is configured to perform deep feature extraction on the fused data using a cross-modal collaborative learning framework, and to perform feature distillation compression and generative adversarial network few-shot enhancement. The cross-modal collaborative learning framework includes a 3D convolutional neural network, a graph neural network, and a Transformer cross-modal attention mechanism. The fault diagnosis layer is configured to use a physical information neural network to embed the component failure partial differential equation as a physical regularization term, employ an adaptive spatiotemporal sliding window and gated memory unit to capture temporal features, and perform multi-task classification through a lightweight gradient booster. S2: Acquire multi-source operation and maintenance data of offshore wind turbines, including meteorological data, turbine SCADA parameters, and inspection images; S3: Utilize the multi-source data coupling layer to perform spatiotemporal alignment and adaptive weighted fusion of the multi-source operation and maintenance data to construct a three-dimensional fusion dataset of meteorology, generator set, and image. S4: The feature extraction layer is used to perform deep feature extraction, feature distillation compression, and small sample enhancement on the three-dimensional fusion dataset to obtain a physical-data dual-driven feature vector. S5: Utilize the fault diagnosis layer to perform physical constraint-enhanced feature extraction, dynamic segmentation, and multi-task classification on the physical-data dual-driven feature vector, and output the fault diagnosis result.

2. The method according to claim 1, characterized in that, Preferably, the multi-source data coupling layer in step S1 is configured to: use SCADA parameters as the reference mode, and achieve alignment of meteorological data, SCADA parameters and inspection images in the time dimension through a physical constraint dynamic time warping algorithm, wherein the cost function of the physical constraint dynamic time warping algorithm includes a power-wind speed monotonicity constraint term.

3. The method according to claim 1, characterized in that, The multi-source data coupling layer in step S1 is further configured to: perform feature standardization on the aligned multi-source data, calculate the physical information entropy of each feature dimension, allocate fusion weights based on the entropy weight method, and perform joint optimization through a physical constraint loss function containing aerodynamic load equations.

4. The method according to claim 1, characterized in that, In the cross-modal collaborative learning framework of the feature extraction layer in step S1: the 3D convolutional neural network is configured to extract the spatiotemporal features of the inspection image sequence; the graph neural network is configured to model the topological relationship between the components of the wind turbine, where nodes represent components and edges represent physical connections or energy transfer relationships; the Transformer cross-modal attention mechanism is configured to fuse the temporal features of meteorological data with SCADA parameters, wherein the attention weight matrix is ​​superimposed with a physical guidance mask.

5. The method according to claim 1, characterized in that, In step S1, the feature extraction layer performs feature distillation and compression using a knowledge distillation method. The multimodal fusion network constructed by the cross-modal collaborative learning framework is used as the teacher network, and the lightweight feature extraction network is used as the student network. Distillation training is performed using mean squared error loss and KL divergence loss. The multimodal fusion network is composed of a 3D convolutional neural network, a graph neural network, and a Transformer cross-modal attention mechanism connected in parallel and then fused by a gated memory unit. The lightweight feature extraction network is a network containing two convolutional layers and two fully connected layers, and its parameter count does not exceed 20% of the teacher network's parameter count. The generative adversarial network uses physical condition vectors as the generator input to generate synthetic fault features to expand the small sample fault categories.

6. The method according to claim 1, characterized in that, In the physical information neural network of the fault diagnosis layer in step S1, the component failure partial differential equation includes the blade crack propagation equation, the bearing wear evolution equation, or the gearbox tooth surface fatigue equation; the loss function of the physical information neural network includes the observation data fitting term and the partial differential equation residual term.

7. The method according to claim 1, characterized in that, In step S1, the window size of the adaptive spatiotemporal sliding window of the fault diagnosis layer is dynamically adjusted according to the real-time speed fluctuation rate; the gated memory unit includes a reset gate and an update gate, the reset gate is configured to filter operating condition noise, and the update gate is configured to enhance early fault memory.

8. The method according to claim 1, characterized in that, The objective function of the multi-task collaborative training of the lightweight gradient booster of the fault diagnosis layer in step S1 is the sum of the weighted and logarithmic losses of multiple fault diagnosis tasks, and includes an L1 regularization term; the composite fault includes at least two of blade cracks, bearing wear and gearbox failure.

9. A fault diagnosis system for key components of offshore wind turbines, characterized in that, include: A multi-source data coupling module is configured to acquire multi-source operation and maintenance data of offshore wind turbines, including meteorological data, turbine SCADA parameters, and inspection images. The multi-source data coupling module is also configured to perform spatiotemporal alignment and adaptive weighted fusion of the multi-source operation and maintenance data using dynamic time warping and entropy weighting to construct a three-dimensional fusion dataset of meteorology-unit-image. The feature extraction module, connected to the multi-source data coupling module, is configured to perform deep feature extraction on the 3D fusion dataset using a cross-modal collaborative learning framework, which includes a 3D convolutional neural network, a graph neural network, and a Transformer cross-modal attention mechanism. The feature extraction module is also configured to perform feature distillation compression on the extracted features and use a generative adversarial network for few-sample enhancement to obtain a physical-data dual-driven feature vector. The fault diagnosis module, which is connected to the feature extraction module, is configured to process the physical-data dual-drive feature vector using a physical information neural network, embedding the component failure partial differential equation into the loss function of the physical information neural network as a physical regularization term to obtain physical constraint enhancement features. The fault diagnosis module is further configured to dynamically segment the physical constraint enhancement features using an adaptive spatiotemporal sliding window, capture long-term temporal dependencies through a gated memory unit to obtain fault temporal features, and input the fault temporal features into a lightweight gradient booster, perform composite fault classification through multi-task collaborative training, and output fault diagnosis results.

10. The system according to claim 9, characterized in that, The feature extraction module includes a 3D convolutional neural network unit, a graph neural network unit, and a Transformer cross-modal attention unit. The outputs of each unit are fused by a gated memory unit to obtain the physical-data dual-driven feature vector.

Images