Wind power plant diagnosis method and system based on multi-modal feature fusion and fault propagation analysis

Abstract

The invention relates to a wind power plant diagnosis method and system based on multi-modal feature fusion and fault propagation analysis, and belongs to the technical field of wind power equipment fault diagnosis. Visible light, infrared and LiDAR multi-mode data of wind power plant equipment are cooperatively collected through an unmanned aerial vehicle group, local features extracted by YOLOv5, thermodynamic features extracted by Transform and spatial structure features extracted by PointNet + + are fused by adopting a dynamic gating mechanism, and a weighted fusion feature vector is generated. And the feature vectors are fused to identify the fault type, position and confidence through a multi-task neural network, and a wind power equipment material failure model is embedded to optimize a diagnosis result. And a power grid topology model and a fault tracing algorithm are combined to position a fault source and an influence range. And finally generating a maintenance decision according to a fault propagation evaluation result.

Description

Technical Field

[0001] The present application relates to the technical field of wind power equipment fault diagnosis, and more specifically, to a wind farm diagnosis method and system based on multimodal feature fusion and fault propagation analysis. Background Art

[0002] Wind farm equipment fault diagnosis is a key step in ensuring the stable operation of wind power systems. Existing technologies have the following limitations:

[0003] The problem of a single data source: Traditional methods rely on single-modality data (such as visible light or infrared images) and cannot fully capture the characteristics of equipment anomalies. For example, visible light images have difficulty detecting internal thermodynamic defects, while infrared data is insensitive to spatial structure, resulting in low diagnostic accuracy and high false positive rates.

[0004] Lack of fault propagation analysis: Existing technologies lack quantitative analysis of fault propagation paths within the power grid topology. Fault location is limited to the device level, failing to assess its impact on upstream and downstream nodes, leading to uninformed maintenance decisions.

[0005] Inspection efficiency bottleneck: UAV inspection systems have problems such as rigid path planning, weak obstacle avoidance capabilities, and poor communication reliability, resulting in delayed data collection and insufficient real-time diagnostic capabilities.

[0006] Insufficient dynamic adaptability: The feature fusion mechanism is static and cannot dynamically adjust weights based on data quality; the fault analysis model does not embed domain knowledge, and its ability to suppress false alarms is limited. Summary of the Invention

[0007] To solve the above technical problems, the present invention proposes a wind farm diagnosis method and system based on multimodal feature fusion and fault propagation analysis.

[0008] The technical solutions of the present invention are as follows:

[0009] The present invention proposes a wind farm diagnostic system for multimodal feature fusion and fault propagation analysis, comprising:

[0010] The drone swarm collaborative inspection subsystem collects visible light, infrared, and LiDAR multimodal data from wind farm equipment through drone swarms;

[0011] Fault diagnosis platform, including:

[0012] Feature extraction unit: YOLOv5, Transformer, and PointNet++ networks are used to extract local features of visible light images, global thermodynamic features of infrared images, and spatial structural features of LiDAR. A dynamic gating mechanism is used for weighted fusion to output a multimodal fusion feature vector.

[0013] Fault analysis unit: Based on the multimodal fusion feature vector, the multi-task neural network outputs the fault type, fault location and fault confidence score;

[0014] Fault location unit: locates the root cause of the fault and assesses the impact scope based on the fault type, fault location, fault confidence score, grid topology model, and fault tracing algorithm;

[0015] The dynamic decision engine generates maintenance decision instructions based on the evaluation results of the fault location unit and the wind turbine operation data.

[0016] Preferably, the drone swarm collaborative inspection subsystem is deployed at a wind farm booster station base station, and includes:

[0017] A drone nest with built-in drone charging station and weather monitoring module;

[0018] The task scheduling center allocates inspection tasks based on wind turbine coordinates.

[0019] Preferably, the drone swarm collaborative inspection subsystem further includes:

[0020] Path planning submodule: The drone performs 3D geographic model path planning based on the map software API interface;

[0021] The intelligent obstacle avoidance module integrates LiDAR point cloud and millimeter-wave radar feedback data, and uses obstacle avoidance algorithms to avoid moving obstacles.

[0022] The dual-link communication submodule adopts 5G and LoRa dual-link communication and supports master-slave distributed machine nest networking.

[0023] Preferably, the dynamic gating mechanism dynamically allocates the fusion weight of the multimodal fusion feature vector through the visible light image clarity, the infrared image temperature anomaly intensity and the LiDAR point cloud density.

[0024] Preferably, the multi-task neural network of the fault analysis unit includes: a fault type branch, a spatial location branch and a confidence branch, wherein:

[0025] Fault type branch: A fully connected layer with Softmax activation is used to output the fault probability distribution. A wind power equipment material failure model is embedded in the output layer to apply fatigue curve constraints to the crack probability of wind turbine blades.

[0026] Spatial location branch: A deconvolutional network is used to perform bounding box regression and output the coordinates of the fault location. The grid topology space constraint is introduced into the loss function.

[0027] Confidence branch: uses the Gaussian distribution parameter estimation layer to output the confidence evaluation result of the fault.

[0028] Preferably, the fault tracing algorithm specifically performs:

[0029] Construct a node impedance matrix based on electrical connection relationships;

[0030] Calculate the voltage change at the fault point;

[0031] The fault location is determined by minimizing the fault impact function, which is:

[0032]

[0033] Where: FaultLoc is the value that minimizes the fault impact function; ΔV i is the voltage change at node i; Z ij is the impedance from node i to node j.

[0034] Preferably, the fault diagnosis platform further includes a reverse optimization mechanism:

[0035] When the fault location unit detects a positioning deviation, it is fed back to the feature extraction unit to enhance local feature extraction;

[0036] When the false alarm rate of the fault analysis unit exceeds the threshold, the gating weight retraining of the feature extraction unit is triggered.

[0037] On the other hand, the present invention also provides a wind farm diagnosis method based on multimodal feature fusion and fault propagation analysis, comprising the following steps:

[0038] Collect visible light, infrared and LiDAR multimodal data through drone swarms;

[0039] A dynamic gating fusion model is used to extract cross-modal feature vectors;

[0040] Identify fault type, fault location, and fault confidence score based on multi-task neural network;

[0041] Calculate the fault propagation path based on the grid topology impedance matrix;

[0042] Generate maintenance decisions based on fault propagation layer assessment results and wind turbine operating data.

[0043] On the other hand, the present invention further provides an electronic device having a computer program stored thereon, wherein when the computer program is executed by a processor, the wind farm diagnosis method for multimodal feature fusion and fault propagation analysis as described in any embodiment of the present invention is implemented.

[0044] On the other hand, the present invention also provides a computer-readable medium for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the wind farm diagnosis method of multimodal feature fusion and fault propagation analysis as described in any embodiment of the present invention.

[0045] The present invention has the following beneficial effects:

[0046] Based on the adaptive weighted features of visible light clarity, infrared temperature difference intensity, and LiDAR point cloud density, the system integrates YOLOv5 local features, Transformer thermodynamic features, and PointNet++ spatial features to improve anomaly detection sensitivity. For example, when a blade crack is accompanied by local overheating, the infrared weight β increases with increasing ΔT, ensuring that the thermodynamic features dominate the fusion vector. YOLOv5 is used to extract local visible light features, Transformer is used to capture global infrared thermodynamic features, and PointNet++ is used to analyze LiDAR spatial structure features to achieve comprehensive fault characterization.

[0047] Based on the node impedance matrix and voltage change, the fault root node is located by minimizing the fault impact function; breaking through the limitations of traditional device-level positioning, the fault propagation path and impact range in the power grid (such as voltage fluctuations at upstream and downstream nodes) are quantified to avoid blind maintenance decisions.

[0048] The fault type branch is embedded in the material failure model of wind power equipment to correct the original output probability of the neural network; the loss function of the spatial position branch introduces a regularization term in the grid topology space to constrain the predicted coordinates to be within a reasonable equipment range; this significantly reduces the false alarm rate and improves the engineering credibility of the diagnostic results.

[0049] The drone swarm collaborative inspection subsystem is used to realize functions such as three-dimensional path planning, intelligent obstacle avoidance and automatic charging; it solves the problems of low efficiency and unreliable communication of traditional inspections and supports real-time monitoring of large-scale wind farms. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0051] Figure 1 Schematic diagram of the system framework of the present invention. DETAILED DESCRIPTION

[0052] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0053] It should be understood that the step numbers used herein are only for convenience of description and are not intended to limit the order in which the steps are to be executed.

[0054] It should be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, the singular forms "a", "an" and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0055] The terms “include” and “comprising” indicate the presence of described features, integers, steps, operations, elements and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0056] The term "and / or" refers to and includes any and all possible combinations of one or more of the associated listed items.

[0057] Example 1:

[0058] In order to make the purpose, technical solutions and advantages of the present invention more clear, the following will be combined with the specific embodiments of the present application and refer to the attached Figure 1 , clearly and completely describe the technical solution of the present invention.

[0059] To solve the problems of the prior art, the present invention provides a wind farm diagnosis system for multimodal feature fusion and fault propagation analysis, comprising:

[0060] The drone swarm collaborative inspection subsystem collects visible light, infrared, and LiDAR multimodal data from wind farm equipment through drone swarms;

[0061] As a preferred implementation of this embodiment, the drone swarm collaborative inspection subsystem is deployed at a wind farm booster station base station, including:

[0062] The drone nest has a built-in drone charging station and a weather monitoring module. The drone charging efficiency can be adjusted according to the data from the weather monitoring module. When the wind speed is detected to be greater than 8m / s, the charging efficiency is increased by 40%.

[0063] The task scheduling center allocates inspection tasks based on wind turbine coordinates.

[0064] As a preferred implementation of this embodiment, the drone swarm collaborative inspection subsystem further includes:

[0065] In the path planning submodule, the drone performs three-dimensional geographic model path planning based on the map software API interface; it obtains elevation data through the Google Maps API, constructs a Delaunay triangulation terrain model, and generates the energy-optimized inspection path based on the terrain model.

[0066] The intelligent obstacle avoidance module integrates LiDAR point cloud and millimeter-wave radar feedback data, and uses obstacle avoidance algorithms to avoid moving obstacles.

[0067] The dual-link communication submodule utilizes 5G and LoRa dual-link communication, supporting master-slave distributed nest networking. The 5G primary link has a base station handover latency of less than 50ms and supports 4K video backhaul. The LoRa backup link uses a +20dB gain antenna with a communication range of 15km. This master-slave distributed nest networking involves setting up a master nest at the booster station and slave nests every 3km.

[0068] Fault diagnosis platform, including:

[0069] Feature extraction unit: YOLOv5, Transformer, and PointNet++ networks are used to extract local features of visible light images, global thermodynamic features of infrared images, and spatial structural features of LiDAR, respectively. A dynamic gating mechanism is used for weighted fusion to output a multimodal fusion feature vector. Specifically, YOLOv5 is mainly used for surface defects of collector lines (such as damaged insulators and broken wires); Transformer thermodynamic features focus on internal structural abnormalities of wind turbine blades (such as delamination and cracks); and PointNet++ extracts the three-dimensional spatial deformation features of blades and insulators (such as deformation and distortion).

[0070] As a preferred implementation of this embodiment, the dynamic gating mechanism dynamically allocates the fusion weight of the multimodal fusion feature vector based on the visible light image clarity, infrared image temperature anomaly intensity, and LiDAR point cloud density, specifically:

[0071] F fused =α·F yolo +β·F vit +γ·F pointnet ;

[0072] in:

[0073]

[0074] Where: F fusedis the multimodal fusion feature vector; α, β, and γ are the weight factors of visible light, infrared light, and LiDAR respectively; F yo1o 、F vit 、F pointnet are the feature vectors of visible light, infrared light and LiDAR respectively; vis clarity is the score of the visible light image clarity, ranging from 0 to 1; ΔT ir is the maximum temperature difference of the infrared image, which is used to detect abnormal heating intensity of the equipment; σ is the Sigmoid function; ρ lidar is the LiDAR point cloud density.

[0075] Fault analysis unit: Based on the multimodal fusion feature vector, a multi-task neural network is used to output the fault type (such as blade crack, collector line short circuit, etc.), fault location (locating blade damage coordinates or collector line fault section) and fault confidence score;

[0076] As a preferred implementation of this embodiment, the multi-task neural network of the fault analysis unit includes: a fault type branch, a spatial location branch, and a confidence branch, wherein:

[0077] Fault type branch: A fully connected layer with Softmax activation is used to output the fault probability distribution. A wind power equipment material failure model is embedded in the output layer to impose fatigue curve constraints on the crack probability of wind turbine blades. The wind power equipment material failure model includes constraints on the fatigue life of wind turbine blades and crack morphology, specifically:

[0078] Fatigue life constraint function:

[0079]

[0080] Where: P is the failure probability of the original output of the neural network; N is the number of stress cycles experienced by the wind power equipment; Modify the original probability P, if N<10 7 , reduce the crack probability proportionally, if N≥10 7 , retaining the original probability.

[0081] Crack morphology constraint function:

[0082]

[0083] Where: P ’ is the failure probability after correction of fatigue life constraint, that is, the output of fatigue life constraint function.

[0084] The final output of the output layer is:

[0085] P out =Softmax(λ·g morphology (ffatigue (W T h+b))+(1-λ)·(W T h+b));

[0086] Where: P out is the failure probability of the final output of the output layer; (W T h+b) is the original output of the fully connected layer; W is the weight matrix; h is the eigenvector of the previous hidden layer; b is the bias term; λ is the weight factor constrained by the material failure model of wind power equipment, which is used to balance the output of the material failure model and the original network.

[0087] Spatial location branch: A deconvolution network is used for bounding box regression to output the coordinates of the fault location. The grid topology space constraint is introduced into the loss function based on the topology of the collector line. The final loss function is:

[0088] L total =L reg +κ·R(Φ topo );

[0089] in:

[0090]

[0091] Where: L reg is the basic regression function, and this embodiment adopts smooth L1 loss; κ is the topology constraint weight coefficient; R is the topology constraint regularization term; Φ topo is the topology constraint function; (u, v) is the center coordinate of the predicted bounding box; k is the device index; δ k is the topological weight of the kth device; P k is the standard coordinate of the kth device; τ is the spatial constraint radius coefficient.

[0092] Confidence branch: uses the Gaussian distribution parameter estimation layer to output the confidence evaluation result of the fault.

[0093] Fault location unit: locates the root cause of the fault and assesses the impact scope based on the fault type, fault location, fault confidence score, grid topology model, and fault tracing algorithm;

[0094] As a preferred implementation of this embodiment, the fault tracing algorithm specifically performs:

[0095] Construct a node impedance matrix based on electrical connection relationships;

[0096] Calculate the voltage change at the fault point;

[0097] The fault location is determined by minimizing the fault impact function, which is:

[0098]

[0099] Where: FaultLoc is the value that minimizes the fault impact function; ΔV i is the voltage change at node i; Z ij is the impedance from node i to node j.

[0100] As a preferred implementation of this embodiment, the fault diagnosis platform further includes a reverse optimization mechanism:

[0101] When the fault location unit detects a positioning deviation, it is fed back to the feature extraction unit to enhance local feature extraction;

[0102] When the false alarm rate of the fault analysis unit exceeds the threshold, the gating weight retraining of the feature extraction unit is triggered.

[0103] The dynamic decision engine generates maintenance decision instructions based on the evaluation results of the fault location unit and the wind turbine operation data.

[0104] Example 2:

[0105] This embodiment provides a wind farm diagnosis method based on multimodal feature fusion and fault propagation analysis, including the following steps:

[0106] Collect visible light, infrared and LiDAR multimodal data through drone swarms;

[0107] A dynamic gating fusion model is used to extract cross-modal feature vectors;

[0108] Identify fault type, fault location, and fault confidence score based on multi-task neural network;

[0109] Calculate the fault propagation path based on the grid topology impedance matrix;

[0110] Generate maintenance decisions based on fault propagation layer assessment results and wind turbine operating data.

[0111] Example 3:

[0112] This embodiment provides an electronic device having a computer program stored thereon. When the computer program is executed by a processor, the wind farm diagnosis method for multimodal feature fusion and fault propagation analysis as described in any embodiment of the present invention is implemented.

[0113] Example 4:

[0114] This embodiment provides a computer-readable medium for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the wind farm diagnosis method of multimodal feature fusion and fault propagation analysis as described in any embodiment of the present invention.

[0115] In the embodiments of the present application, "at least one" refers to one or more, and "more" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. Among them, A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can represent: a, b, c, a and b, a and c, b and c or a and b and c, where a, b, c can be single or multiple.

[0116] Those skilled in the art will appreciate that the various units and algorithm steps described in the embodiments disclosed herein can be implemented using a combination of electronic hardware, computer software, and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0117] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0118] In the several embodiments provided in this application, if any function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of this application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory; hereinafter referred to as: ROM), random access memory (Random Access Memory; hereinafter referred to as: RAM), disk or optical disk, and other media that can store program code.

[0119] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention's description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. Wind farm diagnosis system based on multimodal feature fusion and fault propagation analysis, characterized by: include: The drone swarm collaborative inspection subsystem collects visible light, infrared, and LiDAR multimodal data from wind farm equipment through drone swarms; Fault diagnosis platform, including: Feature extraction unit: YOLOv5, Transformer, and PointNet++ networks are used to extract local features of visible light images, global thermodynamic features of infrared images, and spatial structural features of LiDAR. A dynamic gating mechanism is used for weighted fusion to output a multimodal fusion feature vector. Fault analysis unit: Based on the multimodal fusion feature vector, the multi-task neural network outputs the fault type, fault location and fault confidence score; Fault location unit: locates the root cause of the fault and assesses the impact scope based on the fault type, fault location, fault confidence score, grid topology model, and fault tracing algorithm; The dynamic decision engine generates maintenance decision instructions based on the evaluation results of the fault location unit and the wind turbine operation data.

2. The wind farm diagnostic system for multimodal feature fusion and fault propagation analysis according to claim 1 is characterized by: The drone swarm collaborative inspection subsystem is deployed at the wind farm booster station base station and includes: A drone nest with built-in drone charging station and weather monitoring module; The task scheduling center allocates inspection tasks based on wind turbine coordinates.

3. The wind farm diagnostic system for multimodal feature fusion and fault propagation analysis according to claim 1 is characterized by: The drone swarm collaborative inspection subsystem also includes: Path planning submodule: The drone performs 3D geographic model path planning based on the map software API interface; The intelligent obstacle avoidance module integrates LiDAR point cloud and millimeter-wave radar feedback data, and uses obstacle avoidance algorithms to avoid moving obstacles. The dual-link communication submodule adopts 5G and LoRa dual-link communication and supports master-slave distributed machine nest networking.

4. The wind farm diagnostic system for multimodal feature fusion and fault propagation analysis according to claim 1 is characterized by: The dynamic gating mechanism dynamically allocates the fusion weight of the multimodal fusion feature vector according to the visible light image clarity, infrared image temperature anomaly intensity and LiDAR point cloud density.

5. The wind farm diagnostic system for multimodal feature fusion and fault propagation analysis according to claim 1 is characterized by: The multi-task neural network of the fault analysis unit includes: a fault type branch, a spatial location branch, and a confidence branch, wherein: Fault type branch: A fully connected layer with Softmax activation is used to output the fault probability distribution. A wind power equipment material failure model is embedded in the output layer to apply fatigue curve constraints to the crack probability of wind turbine blades. Spatial location branch: A deconvolutional network is used to perform bounding box regression and output the coordinates of the fault location. The grid topology space constraint is introduced into the loss function. Confidence branch: uses the Gaussian distribution parameter estimation layer to output the confidence evaluation result of the fault.

6. The wind farm diagnostic system for multimodal feature fusion and fault propagation analysis according to claim 1 is characterized by: The fault tracing algorithm specifically performs: Construct a node impedance matrix based on electrical connection relationships; Calculate the voltage change at the fault point; The fault location is determined by minimizing the fault impact function, which is: Where: FaultLoc is the value that minimizes the fault impact function; ΔV i is the voltage change at node i; Z ij is the impedance from node i to node j.

7. The wind farm diagnostic system for multimodal feature fusion and fault propagation analysis according to claim 1 is characterized by: The fault diagnosis platform also includes a reverse optimization mechanism: When the fault location unit detects a positioning deviation, it is fed back to the feature extraction unit to enhance local feature extraction; When the false alarm rate of the fault analysis unit exceeds the threshold, the gating weight retraining of the feature extraction unit is triggered.

8. A wind farm diagnosis method based on multimodal feature fusion and fault propagation analysis, characterized in that: The following steps are involved: Collect visible light, infrared and LiDAR multimodal data through drone swarms; A dynamic gating fusion model is used to extract cross-modal feature vectors; Identify fault type, fault location, and fault confidence score based on multi-task neural network; Calculate the fault propagation path based on the grid topology impedance matrix; Generate maintenance decisions based on fault propagation layer assessment results and wind turbine operating data.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the wind farm diagnosis method of multimodal feature fusion and fault propagation analysis according to claim 8 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the wind farm diagnosis method of multimodal feature fusion and fault propagation analysis as claimed in claim 8 is implemented.

Images