A marine wind turbine fault defect identification system based on multi-source data fusion

Abstract

The application provides a kind of offshore wind turbine fault defect identification system based on multi-source data fusion, system includes: data acquisition unit, time-frequency feature extraction unit, time series modeling unit is used to time series modeling to vibration characteristics and acoustic characteristics to obtain time sequence characteristics;Multi-modal fusion unit is used to fuse time sequence characteristics and time aggregation feature vectors by tensor fusion network to generate fusion features;Defect identification unit is used for classification and positioning of fusion features, output fault type and confidence;Structured pruning and dynamic gating unit is used to perform structured pruning and dynamic gating scheduling on time series modeling unit and multi-modal fusion unit according to current task complexity and feature redundancy, lightweight network structure design and dynamic pruning strategy are adopted, the model inference speed and accuracy are considered, which is convenient for real-time operation on resource-limited edge devices, reduces the personnel investment and maintenance cost of offshore wind farm operation and maintenance.

Description

Technical Field

[0001] This application relates to the field of wind turbine fault identification technology, and in particular to a fault defect identification system for offshore wind turbines based on multi-source data fusion. Background Technology

[0002] With the rapid development of the global offshore wind power industry, offshore wind turbines, as an important carrier of marine renewable energy, are facing various operational uncertainties and potential equipment failures caused by the complex marine environment. Traditional turbine condition monitoring methods often rely on single sensors or single-modal data (such as simple vibration signals or infrared images). In harsh sea conditions, the signals are easily affected by noise interference, and the limited information dimensions make it difficult to meet the accuracy and reliability of fault diagnosis for actual operation and maintenance needs. Especially in conditions far from land and where manual inspection is inconvenient, existing methods often cannot achieve rapid identification and location of early defects in key components such as turbine blades, gearboxes, and generators, affecting the safe and stable operation of wind turbines and maintenance cost control.

[0003] Currently, related technologies utilize multimodal sensors (including vibration, acoustic, and video sensors) deployed at key locations within the turbine unit. Combined with advanced time-frequency analysis and neural network models, this enables the effective extraction and coupling of heterogeneous data from multiple sources, significantly improving the robustness and generalization ability of fault detection and classification under complex operating conditions. However, existing multi-source fusion methods often suffer from redundant algorithm models, insufficient real-time performance, and poor deployment flexibility, making it difficult to meet the practical needs of offshore wind farms for low power consumption and real-time edge-side inference. Summary of the Invention

[0004] In view of this, it is necessary to provide a fault and defect identification system for offshore wind turbines based on multi-source data fusion, which can at least overcome one of the above defects.

[0005] In a first aspect, embodiments of this application provide a fault and defect identification system for offshore wind turbines based on multi-source data fusion, applied to monitoring offshore wind turbines, the system comprising:

[0006] The data acquisition unit is used to collect the operating data of the offshore wind turbine, including vibration signals, operating noise signals and image signals.

[0007] The time-frequency feature extraction unit is used to extract features from the running data to obtain vibration features, acoustic features, and image features;

[0008] The temporal modeling unit is used to perform time-series modeling on the vibration features and the acoustic features to obtain temporal features, and is also used to perform temporal pooling on the image features to obtain a temporal aggregated feature vector.

[0009] A multimodal fusion unit is used to fuse the temporal features and the temporal aggregated feature vector through a tensor fusion network to generate fused features;

[0010] The defect identification unit is used to classify and locate the fused features, and output the fault type and confidence level;

[0011] The structured pruning and dynamic gating unit is used to perform structured pruning and dynamic gating scheduling on the temporal modeling unit and the multimodal fusion unit according to the current task complexity and feature redundancy. Specifically, the structured pruning and dynamic gating unit is used to calculate the importance score of redundant channels during forward inference. The formula for calculating the importance score is:

[0012]

[0013] Among them, S i w represents the importance score. ij This indicates that the i-th channel corresponds to the j-th weight parameter, and N represents the total number of channel weight parameters;

[0014] Based on a preset threshold, channels with importance scores below the preset threshold are dynamically pruned;

[0015] The channel's switching is controlled by a gating function, the importance score, and the preset threshold. The expression for the gating function is:

[0016] G i =σ(S i -θ)

[0017] Among them, G i Let σ be the gating state of the i-th channel, σ be the sigmoid activation function, and θ be the preset threshold.

[0018] In one embodiment of this application, the structured pruning and dynamic gating unit includes a pruning scheduling module;

[0019] The pruning scheduling module is used to obtain the channel importance in historical running data. The channel importance distribution is obtained through the Fisher information matrix, and the calculation formula for the channel importance measurement parameter is as follows:

[0020]

[0021] Where, θ i For model parameters, F i For parameter θ i The importance score, E[·] is the expected value operator, used to calculate the average value of the input data p(x,y). Let be the log-likelihood function. For the loss function with respect to parameter θ i The gradient;

[0022] The pruning scheduling module is also used to reduce overall inference latency by optimizing the objective, and the formula for minimizing the optimization objective is:

[0023]

[0024] in, The computational loss introduced by pruning, M i C is an indicator for whether the i-th channel is retained. i The channel's computational weight.

[0025] In one embodiment of this application, the structured pruning and dynamic gating unit further includes a threshold adaptive adjustment module;

[0026] The threshold adaptive adjustment module is used to adjust the preset threshold according to the operating status of the offshore wind turbine.

[0027] The formula used by the threshold adaptive adjustment module to adjust the preset threshold is:

[0028]

[0029] Where, θ t+1 θ represents the preset threshold at time t+1. t The preset threshold for time t is represented by β, which is the smoothing coefficient.

[0030] The preset threshold is used to calculate the gating function.

[0031] In one embodiment of this application, a collaborative working mechanism is configured between the structured pruning and dynamic gating unit and the multimodal fusion unit, specifically:

[0032] The multimodal fusion unit receives input feature vectors from each modality and modulates the features of each channel through the corresponding gating state to obtain feature tensors;

[0033] The modulated feature tensor is decomposed into a low-rank decomposition using a tensor fusion network to obtain the decomposition vector.

[0034] The structured pruning and dynamic gating unit retains the feature components corresponding to valid channels whose gating states are higher than the preset threshold, based on the gating state of each channel.

[0035] In one embodiment of this application, the expression of the tensor fusion network is:

[0036]

[0037] Among them, T fuse Let R be the fused feature tensor, and a be the decomposition rank. r b r c r The trainable vectors represent image features, vibration features, and acoustic features, respectively.

[0038] In one embodiment of this application, when the structured pruning and dynamic gating unit calculates the weight gradient of each channel, it proportionally weights the gradient of the channel according to the gating state corresponding to each channel.

[0039] When the gating state of the channel is lower than the preset threshold, the gradient of the channel is adjusted by exponential decay.

[0040] In one embodiment of this application, the structured pruning and dynamic gating unit is further used for:

[0041] Obtain the signal-to-noise ratio of the vibration signal and the proportion of background noise energy of the operating noise signal;

[0042] When the signal-to-noise ratio of the vibration signal is lower than the signal-to-noise ratio threshold or the background noise energy ratio is lower than the background noise threshold, a signal quality factor is generated.

[0043] The importance score for each channel is weighted and adjusted using the following formula:

[0044]

[0045] Among them, S i ′ is the corrected importance score, Q is the signal quality factor, Q∈[0,1].

[0046] In one embodiment of this application, the data acquisition unit includes a multi-channel sensor array and an adaptive sampling module;

[0047] The sensors in the multi-channel sensor array are located in the unmanned aerial vehicle and platform, as well as in the blades, gearboxes, and main bearings of the offshore wind turbine.

[0048] The adaptive sampling module is used to adjust the sensor sampling parameters according to the real-time signal-to-noise ratio. The adaptive sampling module is also used to wake up the sensor according to historical abnormal data records and abnormal triggering mechanisms.

[0049] In one embodiment of this application, the system further includes an alarm module, which is used to output an anomaly level based on the confidence level output by the defect identification unit, and to perform an alarm operation based on the anomaly level.

[0050] In one embodiment of this application, the structured pruning and dynamic gating unit is further configured to obtain the accuracy of the confidence level and update the structured pruning and dynamic gating scheduling strategy based on the accuracy of the confidence level.

[0051] This application provides a fault and defect identification system for offshore wind turbines based on multi-source data fusion. By introducing structured pruning and dynamic gating techniques, combined with a signal quality linkage optimization mechanism, it significantly improves the identification efficiency and robustness under complex sea conditions. The system calculates importance scores based on the mean absolute value of channel weights and incorporates dynamic weighted correction of signal quality factors, compressing the importance scores of channels corresponding to low-quality signals, thereby suppressing the activation probability of redundant channels. Simultaneously, through dynamic calibration of the gating function threshold, unreliable channels are shut down in scenarios such as salt spray corrosion and typhoon interference, reducing noise feature interference. Furthermore, a tensor fusion network is used to perform low-rank decomposition of image, vibration, and acoustic features, reducing computational complexity while preserving key cross-modal interaction information, combined with an online adaptive model update strategy. This effectively solves problems such as sensor signal contamination caused by high salt spray and computing power limitations caused by strong winds and waves in offshore wind power scenarios, providing high-precision, low-power intelligent diagnostic capabilities for defects such as blade cracks and gearbox wear, significantly reducing operation and maintenance costs and the risk of missed fault reports. Attached Figure Description

[0052] Figure 1 This is a schematic diagram of a module of an offshore wind turbine fault and defect identification system based on multi-source data fusion, provided in an embodiment of this application.

[0053] Figure 2 A schematic diagram of a fault and defect identification system for offshore wind turbines based on multi-source data fusion, provided as another embodiment of this application.

[0054] Figure 3 A schematic diagram of an electronic device provided in an embodiment of this application.

[0055] Explanation of main component symbols

[0056] Offshore wind turbine faults based on multi-source data fusion

[0057] Defect identification system 10; 10a

[0058] Data acquisition unit 100

[0059] Multichannel sensor array 110

[0060] Adaptive sampling module 120

[0061] Time-frequency feature extraction unit 200

[0062] Temporal modeling unit 300

[0063] Multimodal fusion unit 400

[0064] Defect Identification Unit 500

[0065] Structured pruning and dynamic gating unit 600

[0066] Pruning scheduling module 610

[0067] Threshold adaptive adjustment module 620

[0068] Alarm module 700

[0069] Electronic devices 20

[0070] Processor 21

[0071] Memory 22 Detailed Implementation

[0072] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them.

[0073] It should be noted that, in the embodiments of this application, "at least one" refers to one or more, and "more than one" refers to two or more. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the specification of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application.

[0074] It should be noted that in the embodiments of this application, the terms "first," "second," etc., are used only for descriptive purposes and should not be construed as indicating or implying relative importance, nor as indicating or implying order. Features specified as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0075] Based on the embodiments described in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0076] With the rapid development of the global offshore wind power industry, offshore wind turbines, as an important carrier of marine renewable energy, are facing various operational uncertainties and potential equipment failures caused by the complex marine environment. Traditional turbine condition monitoring methods often rely on single sensors or single-modal data (such as simple vibration signals or infrared images). In harsh sea conditions, the signals are easily affected by noise interference, and the limited information dimensions make it difficult to meet the accuracy and reliability of fault diagnosis for actual operation and maintenance needs. Especially in conditions far from land and where manual inspection is inconvenient, existing methods often cannot achieve rapid identification and location of early defects in key components such as turbine blades, gearboxes, and generators, affecting the safe and stable operation of wind turbines and maintenance cost control.

[0077] Currently, related technologies utilize multimodal sensors (including vibration, acoustic, and video sensors) deployed at key locations within the turbine unit. Combined with advanced time-frequency analysis and neural network models, this enables the effective extraction and coupling of heterogeneous data from multiple sources, significantly improving the robustness and generalization ability of fault detection and classification under complex operating conditions. However, existing multi-source fusion methods often suffer from redundant algorithm models, insufficient real-time performance, and poor deployment flexibility, making it difficult to meet the practical needs of offshore wind farms for low power consumption and real-time edge-side inference.

[0078] In view of this, the offshore wind turbine fault and defect identification system based on multi-source data fusion provided in this application fully considers the operating status of multiple components of the unit in the complex offshore environment. By fusing multi-modal data such as vibration, sound and video images, it achieves high-precision perception and fault location of the operating status of key components of the wind turbine.

[0079] The offshore wind turbine fault identification system provided in this application, based on multi-source data fusion, overcomes the limitations of single sensors, such as environmental interference and incomplete information, by introducing multi-modal sensing technologies including vibration, acoustics, and vision. This significantly improves the accuracy and robustness of fault diagnosis in harsh marine environments. Through ensemble algorithms, it achieves deep correlation analysis and spatiotemporal feature coupling of multi-source heterogeneous data, effectively capturing weak abnormal signals in the wind turbine's operating status and enhancing the sensitivity to early defects.

[0080] Furthermore, the system fully considers actual offshore deployment conditions, employing a lightweight network structure design and dynamic pruning strategies. It balances model inference speed and accuracy, facilitating real-time operation on resource-constrained edge devices. This reduces personnel input and maintenance costs for offshore wind farm operations, extends equipment lifespan, and improves the safety and economy of wind turbines. Overall, the system possesses comprehensive advantages such as high precision, low power consumption, interpretability, and ease of deployment, demonstrating significant engineering application value and promising prospects for promoting intelligent operation and maintenance in the offshore wind power industry.

[0081] Figure 1 This is a schematic diagram of a fault and defect identification system module for offshore wind turbines based on multi-source data fusion, provided in an embodiment of this application. Figure 1 The offshore wind turbine fault and defect identification system 10 based on multi-source data fusion shown includes at least the following parts: data acquisition unit 100, time-frequency feature extraction unit 200, time-series modeling unit 300, multi-modal fusion unit 400, defect identification unit 500, and structured pruning and dynamic gating unit 600.

[0082] In one embodiment of this application, the data acquisition unit 100 is used to acquire the operating data of the offshore wind turbine, including vibration signals, operating noise signals and image signals.

[0083] Specifically, the data acquisition unit 100 includes vibration sensors, acoustic sensors, and visible light cameras installed on key parts such as blades and gearboxes, as well as cameras mounted on the outside of the cabin or on a drone, for real-time acquisition of vibration, noise, and appearance images of various parts of the equipment.

[0084] Understandably, through the coordinated deployment of the aforementioned multi-source sensors, it is possible to comprehensively perceive the unit's mechanical status, operating acoustic characteristics, and changes in its external structure, ensuring the integrity and accuracy of subsequent analysis data.

[0085] In one embodiment of this application, the time-frequency feature extraction unit 200 is used to extract features from the running data to obtain vibration features, acoustic features and image features.

[0086] Specifically, the time-frequency feature extraction unit 200 uses short-time Fourier transform (STFT) and wavelet transform (WT) to obtain the time-frequency distribution of vibration and acoustic signals, respectively, and uses convolutional neural network (CNN) to extract spatial features such as texture, shape and color for image signals.

[0087] Understandably, by extracting features from the time and frequency domains and the spatial domain, transient anomaly information in vibration and acoustic signals and structural anomaly details in video images can be effectively preserved, providing rich multimodal feature inputs for subsequent time-series modeling and fusion.

[0088] In one embodiment of this application, the time-series modeling unit 300 is used to perform time-series modeling on vibration features and acoustic features to obtain time-series features, and is also used to perform time pooling on image features to obtain time-aggregated feature vectors.

[0089] Specifically, the temporal modeling unit 300 inputs the time-frequency features of vibration and acoustics into a gated recurrent unit (GRU) to extract key patterns that change over time. It then performs temporal max pooling or average pooling operations on the spatial features of consecutive frames to obtain aggregated feature vectors describing information from multiple frames.

[0090] Understandably, temporal modeling can capture the state evolution trend of a device over a long period of time, while temporal pooling compresses redundant information in video frame sequences, retains key frame features, and improves the compactness of feature representation.

[0091] In one embodiment of this application, the multimodal fusion unit 400 is used to fuse temporal features and temporal aggregated feature vectors through a tensor fusion network to generate fused features.

[0092] Specifically, the multimodal fusion unit 400 concatenates the temporal feature vectors of vibration and acoustics with the temporal aggregated features of the image through a tensor fusion network (TFN) and performs cross-modal interactive calculations to output a high-dimensional fusion feature vector containing multi-source information.

[0093] Understandably, the multimodal fusion unit 400 improves the ability to distinguish fault modes of offshore wind turbines by modeling the interdependencies between different modes and making full use of the complementarity of various data sources.

[0094] In one embodiment of this application, the defect identification unit 500 is used to classify and locate the fused features and output the fault type and confidence level.

[0095] Specifically, the defect identification unit 500 uses a multi-layer fully connected network combined with the Softmax activation function to map the fused features to a preset fault category space, and outputs the confidence probability of the corresponding category, thereby determining whether a fault exists and its possible location.

[0096] Understandably, this unit achieves rapid and automated identification and early warning of different faults and defects through end-to-end supervised learning, thereby improving the efficiency and reliability of remote intelligent operation and maintenance of wind turbine units.

[0097] In one embodiment of this application, the structured pruning and dynamic gating unit 600 is used to perform structured pruning and dynamic gating scheduling on the temporal modeling unit 300 and the multimodal fusion unit 400 according to the current task complexity and feature redundancy.

[0098] Specifically, the structured pruning and dynamic gating unit is used to calculate the importance score of redundant channels during forward inference. The formula for calculating the importance score is as follows:

[0099]

[0100] Among them, S i w represents the importance score. ij This indicates that the i-th channel corresponds to the j-th weight parameter, and N represents the total number of weight parameters for that channel.

[0101] In one embodiment of this application, the structured pruning and dynamic gating unit 600 is further used to dynamically prune channels with importance scores below a preset threshold based on a preset threshold. The switching of channels is controlled by a gating function, importance score, and preset threshold. The expression for the gating function is:

[0102] G i =σ(S i -θ)

[0103] Among them, G i Let σ be the gating state of the i-th channel, σ be the sigmoid activation function, and θ be the preset threshold.

[0104] Specifically, through the aforementioned structured pruning and dynamic gating mechanisms, the system can adaptively shut down unnecessary channels or units while ensuring accuracy, thereby reducing the computational complexity and storage overhead of the model and adapting to the real-time inference needs of edge devices.

[0105] Understandably, this unit achieves dynamic scheduling of computing resources based on input complexity, balancing the robustness of the model, inference speed, and energy consumption, and improving the practicality and scalability of unattended monitoring of offshore wind turbines.

[0106] The offshore wind turbine fault and defect identification system 10 based on multi-source data fusion provided in this application embodiment achieves efficient and intelligent diagnosis under complex offshore operating conditions through multi-source data collaborative analysis and dynamic resource optimization strategies. The system employs structured pruning and dynamic gating techniques, combined with dual constraints of channel importance scores and signal quality factors, to dynamically close redundant computational paths, compressing the computational load of edge device inference stages while ensuring identification accuracy. A tensor fusion network is used to perform low-rank decomposition of vibration, acoustic, and image features, reducing storage overhead while preserving key cross-modal information and solving the feature interference problem caused by modal conflicts in traditional methods. The gating function and signal quality linkage mechanism can dynamically adapt to harsh sea conditions such as salt spray corrosion and typhoon interference, avoiding false alarms and missed alarms caused by sensor noise pollution. The above technical solutions effectively solve the problems of signal distortion caused by high salt spray and computing power limitations caused by strong winds and waves in offshore wind power scenarios, achieving high-precision, low-power real-time detection of defects such as blade cracks and gearbox wear, significantly reducing operation and maintenance costs and the risk of missed faults, and providing reliable technical support for the intelligent and unmanned operation and maintenance of offshore wind turbines.

[0107] Figure 2 This is a schematic diagram of a fault and defect identification system module for offshore wind turbines based on multi-source data fusion, provided in another embodiment of this application. Figure 2 The offshore wind turbine fault and defect identification system 10a shown is based on multi-source data fusion, and is similar to... Figure 1The difference between the offshore wind turbine fault and defect identification system 10 based on multi-source data fusion shown is that the offshore wind turbine fault and defect identification system 10a based on multi-source data fusion also includes: an alarm module 700, wherein the data acquisition unit 100 includes a multi-channel sensor array 110 and an adaptive sampling module 120, and the structured pruning and dynamic gating unit 600 also includes a pruning scheduling module 610 and a threshold adaptive adjustment module 620.

[0108] In one embodiment of this application, the structured pruning and dynamic gating unit 600 includes a pruning scheduling module 610.

[0109] In one embodiment of this application, the pruning scheduling module 610 is used to obtain the channel importance in historical running data. The channel importance distribution is obtained through the Fisher information matrix, and the calculation formula for the channel importance measurement parameter is as follows:

[0110]

[0111] Where, θ i For model parameters, F i For parameter θ i The importance score, E[·] is the expected value operator, used to calculate the average value of the input data p(x,y). Let be the log-likelihood function. For the loss function with respect to parameter θ i The gradient.

[0112] Specifically, the pruning scheduling module 610 dynamically adjusts the pruning threshold based on the channel importance distribution in historical operating data. Its core process includes the following steps: The pruning scheduling module 610 extracts historical data from the system operation log, including vibration signals, acoustic signals, and image data collected by multi-source sensors, as well as corresponding model inference results (such as fault type and confidence level). The data is normalized to eliminate feature offsets caused by differences in model, installation location, or sea state between different wind turbines, ensuring the universality of channel importance assessment. The dataset is divided into time windows, each containing multimodal features of the equipment under stable operating conditions (such as normal operation) and abnormal operating conditions (such as blade cracks and gearbox wear), used to train the pruning strategy.

[0113] The pruning scheduling module 610 is also used to calculate the parameter θ for each channel based on the Fisher information matrix calculation formula. i Perform importance scoring F i Importance score F for all channels i The values ​​are then normalized to fall within the [0,1] interval for easier threshold setting. The importance score F is then determined based on the normalized values. i The channels are divided into high-importance channels (F).i ≥0.7), moderately important channels (0.3≤F) i <0.7) and redundant channels (F i <0.3), and set different pruning intensities (such as redundant channel pruning rate).

[0114] ≥60%, and pruning rate of medium importance channels ≤30%.

[0115] The pruning scheduling module 610 also sets differentiated pruning budgets based on different modal feature channels. Image feature channels: Prioritize retaining key channels related to spatial anomalies such as blade cracks and tower corrosion (e.g., channels in high-response regions of ResNet). Vibration feature channels: Retain high-frequency vibration feature channels related to gearbox wear and bearing failure. Acoustic feature channels: Retain frequency band channels related to mechanical impact noise and aerodynamic noise.

[0116] Understandably, the pruning scheduling module 610 achieves accurate identification and pruning of redundant channels through joint optimization of the Fisher information matrix and computational constraints. Specifically: Channel importance assessment quantitatively evaluates the contribution of model parameters to the classification results using Fisher information, providing a basis for ranking channel importance for the pruning scheduling module 610. Multimodal collaborative pruning: The pruning scheduling module 610 dynamically adjusts the pruning budget for each mode by combining the physical meaning of multimodal features (e.g., in the blade crack task, the image channel pruning rate is ≤20%, and the vibration channel pruning rate is ≤10%), avoiding over-pruning of a single mode that leads to missed detection of key defects. Online learning adaptation: During system operation, the pruning scheduling module 610 continuously monitors the changes in recognition performance brought about by the pruning strategy through an online learning algorithm, and dynamically updates the threshold and gating function parameters, enabling the pruning and gating strategies to adaptively adjust under different wind turbine units and external environmental changes, improving the system's universality and generalization ability.

[0117] In one embodiment of this application, the pruning scheduling module 610 is further used to reduce the overall inference latency by optimizing the objective, and the formula for minimizing the optimization objective is:

[0118]

[0119] in, The computational loss introduced by pruning, M i An indicator (M) for whether the i-th channel is retained. i =0 indicates pruning, M i =1 indicates reserved), C i The computational weight for this channel (such as the number of floating-point operations or memory accesses).

[0120] Specifically, the pruning scheduling module 610 minimizes the overall inference latency by optimizing the objective. Its implementation methods include dynamic pruning and gating collaboration, efficient utilization of multimodal features, and online learning adaptation capabilities. The pruning scheduling module 610 maximizes the model compression ratio while satisfying accuracy constraints.

[0121] Understandably, the dynamic optimization strategy of the pruning and scheduling module 610 enables efficient compression and intelligent scheduling of multimodal data from offshore wind turbines. The system can reduce single-frame inference latency on edge devices while maintaining a high F1 score, significantly outperforming fixed-threshold pruning schemes.

[0122] In addition, the online learning mechanism of pruning and gating enables the system to adapt to changes in different wind field environments, reducing maintenance costs caused by sensor aging or contamination, and providing a highly robust and low-energy-consumption intelligent diagnostic solution for unattended monitoring of offshore wind turbines.

[0123] In one embodiment of this application, the structured pruning and dynamic gating unit further includes a threshold adaptive adjustment module 620. The threshold adaptive adjustment module 620 is used to adjust a preset threshold according to the operating status of the offshore wind turbine. The preset threshold is used to calculate the gating function.

[0124] The formula for adjusting the preset threshold by the threshold adaptive adjustment module is as follows:

[0125]

[0126] Where, θ t+1 θ represents the preset threshold for updating at time t+1. t The preset threshold for time t is represented by β, which is the smoothing coefficient.

[0127] Specifically, the threshold adaptive adjustment module 620 dynamically adjusts the preset threshold in the gating function based on the real-time operating status of the offshore wind turbine. The threshold adaptive adjustment module 620 receives multi-dimensional operating status parameters from the data acquisition unit 100 in real time, including: mechanical status parameters: gearbox vibration amplitude, blade rotation period fluctuation rate, main bearing temperature change rate; environmental disturbance parameters: salt spray concentration, wind speed, visibility, humidity; and task priority parameters: the urgency of the current fault type (e.g., blade cracks are a level 1 task, oil spills are a level 3 task). The threshold adaptive adjustment module 620 uses an exponentially weighted moving average (EWMA) algorithm to update the preset threshold. When the smoothing coefficient β is close to 1, the threshold update relies more on historical values, suitable for short-term environmental disturbance fluctuations (e.g., gusts); when the smoothing coefficient β is close to 0, the threshold update relies more on the current importance average. Suitable for long-term environmental changes (such as continuous increase in salt spray concentration).

[0128] It is understandable that this embodiment significantly improves the environmental adaptability of the structured pruning and dynamic gating unit 600 through the dynamic threshold update mechanism of the threshold adaptive adjustment module 620. Through β and The joint optimization automatically adjusts the pruning intensity under severe sea conditions such as typhoon interference and salt spray pollution, so that the compression ratio of model parameters can be dynamically switched within a certain range, and the inference latency can be controlled within the range of 120 to 200 ms.

[0129] In one embodiment of this application, a collaborative working mechanism is configured between the structured pruning and dynamic gating unit 600 and the multimodal fusion unit 400. The multimodal fusion unit 400 receives input feature vectors from various modalities and modulates the features of each channel according to the corresponding gating state to obtain feature tensors. The modulated feature tensors are then subjected to low-rank decomposition through a tensor fusion network to obtain decomposed vectors. The structured pruning and dynamic gating unit 600 retains the feature components corresponding to valid channels whose gating states are higher than a preset threshold, based on the gating state of each channel.

[0130] Specifically, the collaborative working mechanism between the structured pruning and dynamic gating unit 600 and the multimodal fusion unit 400 includes receiving input feature vectors from image features, vibration features, and acoustic features. Each modal feature is modulated according to the channel gating state. The modulated image, vibration, and acoustic features are then concatenated into a third-order tensor modally.

[0131] In one embodiment of this application, the expression for the tensor fusion network is:

[0132]

[0133] Among them, T fuse Let R be the fused feature tensor, and a be the decomposition rank. r b r c r The trainable vectors represent image features, vibration features, and acoustic features, respectively.

[0134] Specifically, the tensor fusion network performs low-rank decomposition on the modulated feature tensors. During the decomposition process, only the feature components corresponding to valid channels with gate states greater than the gate state threshold are retained, while tensor elements of redundant channels are ignored, thereby reducing the computational complexity of the decomposition. For example, when leaf icing causes severe local reflection in the image, the structured pruning and dynamic gating unit 600 automatically shields relevant channels, forcing the fusion process to rely on high-frequency impact features in the vibration signal to ensure the stability of defect identification. The threshold adaptive adjustment module 620 dynamically adjusts the decomposition rank of the tensor fusion network according to the device load status. If the edge device GPU utilization is ≥70%, the decomposition rank is reduced from 128 to 64. If a primary task (such as a leaf crack) is detected, the decomposition rank is increased to 256 to enhance cross-modal feature interaction capabilities.

[0135] It is understood that the collaborative working mechanism provided in this application embodiment can automatically shut down relevant channels and enhance the feature contributions of other modes (such as vibration signal spectral distortion analysis) when a single modal signal is contaminated (e.g., water ingress into a camera causing image blurring), thus avoiding missed detections due to sensor failure. Joint optimization of the tensor fusion network and gating function parameters improves the model's recognition accuracy for new fault types (e.g., fatigue at tower weld points) by more than 15%, without requiring retraining of the entire model.

[0136] In one embodiment of this application, when the structured pruning and dynamic gating unit 600 calculates the weighted gradient of each channel, it proportionally weights the gradient of the channel according to the gating state corresponding to each channel. When the gating state of a channel is lower than a preset threshold, the gradient of the channel is adjusted exponentially.

[0137] Specifically, the structured pruning and dynamic gating unit 600 optimizes the pruning and gating strategies during the model training phase in the following way: During backpropagation, the weight gradient Δw for the i-th channel is... ij Perform a weighted update. The weighted update formula is:

[0138]

[0139] Where, Δw ij G represents the original gradient value. i The gating state of this channel (G) i ∈[0,1]), This is the weighted gradient. If G... i If the value is close to 1 (high gating state), the weight gradient retains its original update strength, ensuring sufficient training of key channels. If G... i When the gating value is close to 0 (low gating state), the weight gradient is compressed, reducing the interference of redundant channels on model training.

[0140] When the channel's gating value is less than a preset value (e.g., 0.3), the gradient is adjusted exponentially.

[0141]

[0142] Here, α is the decay coefficient (usually ranging from 1 to 3), which controls the gradient suppression strength of redundant channels. For redundant channels with low gating states, the gradient decay factor significantly reduces the weight update magnitude, accelerating their convergence to zero. For moderately important channels, the gradient decay factor moderately compresses the update amount, preventing them from excessively participating in key feature learning.

[0143] Combining the knowledge distillation method, we introduce the KL divergence loss between the teacher model output and the student model output:

[0144]

[0145] in, Predict probabilities for the teacher model. Output for the student model. The classification accuracy and model consistency are jointly optimized through a loss function to ensure that the difference in prediction distribution between the student model and the teacher model after pruning is ≤5%.

[0146] Understandably, by using gating states to weight and decay the gradients of redundant channels, the update magnitude of their weights is significantly lower than that of effective channels, ensuring the learning ability of the pruned student model on key features. For example, if the G of a certain vibration signal channel... i =0.2. If it is lower than 0.3, the gradient update amount is only 30% of the original value, and the weights converge to zero quickly, avoiding interference with subsequent inference.

[0147] In one embodiment of this application, the structured pruning and dynamic gating unit 600 is further configured to: acquire the signal-to-noise ratio (SNR) of the vibration signal and the background noise energy ratio of the running noise signal. When the SNR of the vibration signal is lower than a SNR threshold or the background noise energy ratio is lower than a background noise threshold, a signal quality factor is generated. The importance score of each channel is weighted and corrected using the following formula:

[0148]

[0149] Among them, S i ′ is the corrected importance score, Q is the signal quality factor, Q∈[0,1].

[0150] Specifically, the signal quality factor is used to quantify the quality of sensor data. When Q is less than a preset value, such as 0.6, the channel importance score is weighted and corrected.

[0151] For example, if the signal-to-noise ratio of the vibration signal is less than a preset value, then Q = 0.4, and the channel importance score is compressed to 40% of its original value, triggering a more intense pruning strategy. The same principle applies to the background noise energy proportion of the acoustic signal, which will not be elaborated here.

[0152] Understandably, the threshold adaptive adjustment module 620 dynamically updates the pruning threshold and the gate function bias term based on joint optimization using formulas, forming a closed-loop control of "signal quality → gradient compression → channel pruning". This embodiment achieves enhanced training consistency and improved signal quality adaptability through structured pruning, gradient weighting by the dynamic gate unit 600, and importance score correction driven by signal quality. When resources are limited by UAVs or edge devices, combined with joint optimization, the model parameter compression ratio reaches over 50%, and the single-frame inference latency is ≤200ms, meeting the real-time diagnostic requirements of offshore wind turbines.

[0153] In one embodiment of this application, the data acquisition unit 100 includes a multi-channel sensor array 110 and an adaptive sampling module 120. The sensors in the multi-channel sensor array 110 are located on the blades, gearboxes, and main bearings of unmanned aerial vehicles (UAVs) and offshore wind turbines. The adaptive sampling module 120 is used to adjust the sensor sampling parameters according to the real-time signal-to-noise ratio. The adaptive sampling module 120 is also used to activate the sensors based on historical abnormal data records and anomaly triggering mechanisms.

[0154] Specifically, the multi-channel sensor array 110 of the data acquisition unit 100 is deployed on key components of the offshore wind turbine as follows: triaxial vibration sensors are installed at the blade root and tip to collect bending mode and torsional vibration signals, while acoustic emission sensors are attached to the blade surface to capture high-frequency sound waves generated by crack propagation. A drone equipped with a high-definition visible light camera and an infrared thermal imager performs multi-angle inspections of the blades to ensure clear images are acquired even in low light or fog conditions. Accelerometers are placed at the input shaft, output shaft, and planetary gear train of the gearbox, covering different transmission chain fault characteristic frequency bands, and acoustic microphone arrays are installed on their housings to monitor gear meshing noise and abnormal bearing acoustic signatures. Acoustic emission sensors are evenly distributed around the outer ring of the main shaft bearing to capture microcrack propagation signals, while temperature and oil pressure sensors are integrated into the bearing housing to assist in determining lubrication status. The adaptive sampling module 120 monitors the signal-to-noise ratio of the vibration signal and the background noise energy ratio of the acoustic signal in real time, triggering a sampling parameter adjustment mechanism when a decrease in signal quality is detected. For example, when the signal-to-noise ratio of the vibration signal is lower than a preset threshold, the sampling rate is reduced to 1kHz to reduce redundant data. If the background noise of the acoustic signal exceeds a certain percentage, non-target frequency bands (such as wave noise 20–200Hz) are shut down, and only equipment fault-related frequency bands (1–5kHz) are retained. The image resolution dynamic adaptation strategy switches according to visibility and light intensity. When the visibility is less than 50 meters or the light intensity is less than 100 lux, the infrared thermal imaging mode is activated and combined with the dark channel defogging algorithm to enhance image clarity. When the blade rotation angle is in the overhead area, 4K resolution is used to shoot the root stress area, and when rotating laterally, the resolution is switched to 1080p to reduce redundant data transmission. The multi-channel sensor array 110 has a built-in GPS timing module to add precise timestamps (error ≤1ms) to the vibration signal, acoustic signal, and image frames, and compensates for asynchronously acquired multi-source data through a timestamp alignment algorithm. For example, vibration signals are completed with linear interpolation to fill in low-latency frames, image frames and acoustic signals are matched with nearest neighbor timestamps, and a dynamic buffer pool mechanism automatically adjusts the buffer window size according to the arrival order of sensor data to avoid timing misalignment caused by communication delays.

[0155] It is understandable that the collaborative design of the multi-channel sensor array 110 of the data acquisition unit 100 and the adaptive sampling module 120 achieves significant beneficial effects through the following technical means: the sensor deployment scheme covers the key fault characteristics of the blades, gearbox, and main bearing. For example, vibration sensors at the blade root and tip can capture low-frequency bending modes (1–100Hz) and high-frequency torsional vibrations (1–20kHz), respectively; the gearbox accelerometer and acoustic microphone array jointly monitor gear wear and tooth breakage impact sounds; and the main bearing acoustic emission sensor and temperature sensor work together to capture crack propagation and abnormal lubrication conditions. The adaptive sampling module 120 and the structured pruning and dynamic gating unit 600 form a closed-loop control, with the signal quality factor Q directly correlated with the importance score correction logic. For example, when blade icing causes image blurring, the signal quality factor Q decreases to 0.5, the channel importance score is compressed to 50% of its original value, triggering the pruning scheduling module 610 to close relevant image channels and enhance the learning of high-frequency characteristics of the vibration signal. If the standard deviation of grayscale values ​​in a local area of ​​the leaf image changes by more than 15%, the adaptive sampling module 120 activates the drone continuous shooting mode (frame rate increased to 60fps), marking abnormal frames for feature enhancement analysis by the defect identification unit 500. The timestamp synchronization mechanism and dynamic buffer pool strategy ensure the temporal consistency of multi-source data, controlling the timestamp error between vibration signals and image frames to within 1ms, avoiding mismatches caused by temporal misalignment during cross-modal feature fusion. In resource-constrained scenarios, the adaptive sampling module 120 and the pruning scheduling module 610 work together to optimize. For primary tasks (such as leaf breakage), high-frequency sampling and minimum pruning ratio are triggered; for tertiary tasks (such as oil contamination), low sampling rate and maximum pruning ratio are used, reducing inference latency to 100ms. This linkage mechanism is particularly crucial during typhoon season or periods of high salt spray concentration. When salt spray pollution causes lens blurring, the adaptive sampling module 120 automatically switches to infrared thermal imaging, and the pruning scheduling module 610 compresses the image feature channel pruning rate to 60%, while simultaneously increasing the vibration signal sampling rate to 20kHz, ensuring stable identification of critical defects.

[0156] In one embodiment of this application, the system further includes an alarm module 700, which is used to output an anomaly level based on the confidence level output by the defect identification unit, and to perform an alarm operation based on the anomaly level.

[0157] Specifically, the alarm module 700 executes multi-level early warning logic based on the confidence level output by the defect identification unit 500. Its core mechanism includes: classifying fault levels by setting a pre-set confidence threshold, and dynamically adjusting the alarm triggering conditions by combining multimodal redundancy check and signal quality factor Q. For example, a level 1 early warning corresponds to a serious defect with a confidence level ≥ 90%, a level 2 early warning corresponds to a potential defect with a confidence level of 70%–90%, and a level 3 early warning corresponds to a low-confidence abnormal event with a confidence level of 50%–70%. At the same time, the lower limit of the threshold is dynamically adjusted based on the confidence distribution within a sliding window of historical data. When the median confidence level of a certain type of fault drops by more than 10%, the threshold of the corresponding level is automatically reduced (e.g., the level 1 early warning threshold is adjusted from 90% to 85%). The alarm information generation rules include fault type codes (e.g., blade crack FL-001, gearbox wear FL-002), key component numbers (e.g., blade number B-03, gearbox number GB-01), location coordinates (image defect pixel coordinates or vibration sensor ID), and timestamps. Alarm information is synchronously pushed to the SCADA system via the OPC UA protocol and sent to maintenance personnel's mobile terminals via the MQTT protocol. Alarm suppression logic is also configured (e.g., if the same component triggers a Level 3 warning five times consecutively within 24 hours, it is upgraded to a Level 2 warning). Furthermore, the alarm module 700 is linked with the structured pruning and dynamic gating unit 600. When the vibration signal spectral distortion exceeds three times the historical average standard deviation and the image recognition confidence level is 82%, a Level 1 warning is triggered, and the pruning scheduling module 610 is forced to close redundant channel calculation paths to ensure the preservation of key features.

[0158] Understandably, when the alarm module 700 detects a level one warning (such as a broken leaf), it sends a resource preemption command to the pruning scheduling module 610, freezes the redundant channel pruning strategy, and restores the full-function mode to ensure the inference stability of high-priority tasks.

[0159] In one embodiment of this application, the structured pruning and dynamic gating unit 600 is further used to obtain the accuracy of the confidence level and update the structured pruning and dynamic gating scheduling strategy based on the accuracy of the confidence level.

[0160] Specifically, the confidence level generated by the alarm module 700 is compared with the actual labels reported by maintenance personnel to calculate the prediction error and generate a confidence accuracy index. For example, if the average confidence level for blade cracks is 82%, but the actual proportion of such defects in the review results is only 50%, it is determined that the current confidence level has an overestimation bias, triggering a pruning strategy adjustment. The structured pruning and dynamic gating unit 600 combines the signal quality factor Q and the confidence bias to correct the calculation weight of the channel importance score. For example, when salt spray pollution causes a decrease in image signal quality, the signal quality factor Q is compressed to below 0.5, triggering the pruning scheduling module 610 to reduce the pruning intensity of the image channel while increasing the pruning ratio of the vibration signal channel to ensure the model's learning ability on key features. The pruning scheduling module 610 combines the confidence accuracy and the signal quality factor Q to periodically optimize the pruning and gating strategies. For example, if the salt spray concentration in a wind farm continues to rise and the image signal quality factor Q remains below 0.6 for an extended period, the proportion of image features in the pruning budget is automatically reduced, while the retention rate of vibration signal channels is increased to avoid erroneous pruning due to sensor aging or contamination. When the confidence accuracy of image features deteriorates, the pruning scheduling module 610 reduces the pruning ratio of image channels (e.g., from 60% to 30%), while simultaneously increasing the pruning budget for vibration signal channels (e.g., from 20% to 40%). This ensures that the model can still rely on highly robust modes (such as vibration signal spectral distortion features) for stable diagnosis under harsh sea conditions such as blade icing and salt spray corrosion.

[0161] Understandably, the confidence level output by the defect identification unit 500 directly affects the warning level classification of the alarm module 700, while the false alarm rate and false negative rate data of the alarm module 700 inversely drive the pruning scheduling module 610 to adjust the pruning threshold and gating function parameters. For example, when the false alarm rate of the first-level warning for blade cracks exceeds 5% for three consecutive weeks, the pruning scheduling module 610 raises the pruning threshold and closes redundant calculation paths related to frost areas in the image feature channel, forcing the model to recalibrate based on the spectral distortion characteristics of the vibration signal.

[0162] The offshore wind turbine fault identification system 10a based on multi-source data fusion provided in this application, building upon the existing offshore wind turbine fault identification system 10 based on multi-source data fusion, further integrates an online learning mechanism. It combines historical data, real-time monitoring data, and manual inspection reports to periodically evaluate the model's identification accuracy and inference speed under different sea conditions, seasons, and operational phases. When a model performance degradation is detected (e.g., a decrease in validation set accuracy exceeding 2%), the system automatically triggers a transfer learning process and updates the classification head and attention module. It also supports incremental learning to adapt to new fault modes. A dual-buffer architecture enables model version switching, loading the updated version when edge device resources are sufficient and switching to a lightweight mode when resources are limited, ensuring long-term stability under complex offshore wind farm conditions.

[0163] Figure 3 This is an electronic device 20 provided in one embodiment of this application. For example... Figure 3 As shown, the electronic device 20 includes at least the following components: a processor 21 and a memory 22.

[0164] In this embodiment, the memory 22 is used to store executable instructions of the processor 21, which, when configured to execute instructions, implement... Figure 1 or Figure 2 The functions of the units / modules shown are illustrated.

[0165] In one embodiment of this application, the program operating in the electronic device 20 may be a program that controls a central processing unit (CPU) or similar device to achieve the functions described in the above-described embodiments of the present invention (a program that enables the computer to function). Information processed by these devices is then temporarily stored in random access memory (RAM) during processing, and subsequently stored in various ROMs such as read-only memory (Flash ROM) and hard disk drives (HDDs), and read, corrected, and written by the CPU as needed.

[0166] It should be noted that a portion of the electronic device 20 described above can also be implemented using a computer. In this case, the program for implementing the control function can be recorded on a computer-readable recording medium, and the program recorded on the recording medium can be read into the computer and executed.

[0167] It should be noted that the term "computer" as used here refers to a computer built into electronic device 20, employing hardware including an operating system and peripheral devices. Furthermore, "computer-readable recording media" refers to removable media such as floppy disks, magneto-optical disks, ROMs, and CD-ROMs, as well as storage devices such as hard drives built into the computer.

[0168] Furthermore, a "computer-readable recording medium" can include: a medium that dynamically stores a program for a short period of time, such as a communication line used when transmitting a program via a network such as the Internet or a communication line such as a telephone line; or a medium that stores a program for a fixed period of time, such as volatile memory inside a computer that serves as a server or client in this case. In addition, the aforementioned program can be a program used to implement the above-mentioned functions, or it can be a program that can implement the above-mentioned functions by combining with programs already recorded in the computer.

[0169] Furthermore, the electronic device 20 in the above embodiments can also be implemented as an assembly (device group) composed of multiple devices. Each device constituting the device group can possess some or all of the functions or functional blocks of the electronic device 20 in the above embodiments. As a device group, it is sufficient to have all the functions or functional blocks of the electronic device 20.

[0170] It is understood that the offshore wind turbine fault identification systems 10 and 10a based on multi-source data fusion provided in this application, by introducing multi-modal sensing technologies such as vibration, acoustics, and vision, overcome the limitations of single sensors, such as environmental interference and incomplete information, and significantly improve the accuracy and robustness of fault diagnosis in harsh marine environments. Through ensemble algorithms, deep correlation analysis and spatiotemporal feature coupling of multi-source heterogeneous data are achieved, effectively capturing weak abnormal signals of wind turbine operating status and enhancing the sensitivity to early defects.

[0171] Furthermore, the offshore wind turbine fault identification systems 10 and 10a, based on multi-source data fusion, fully consider actual offshore deployment conditions. They employ a lightweight network structure design and dynamic pruning strategies, balancing model inference speed and accuracy. This allows for real-time operation on resource-constrained edge devices, reducing personnel input and maintenance costs for offshore wind farms, extending equipment lifespan, and improving the safety and economy of wind turbines. Overall, the system possesses comprehensive advantages such as high precision, low power consumption, interpretability, and ease of deployment, demonstrating significant engineering application value and promising prospects for promoting intelligent operation and maintenance in the offshore wind power industry.

[0172] Those skilled in the art should recognize that the above embodiments are only used to illustrate this application and are not intended to limit this application. Any appropriate changes and variations made to the above embodiments within the essential spirit and scope of this application fall within the scope of protection claimed in this application.

Claims

1. A fault and defect identification system for offshore wind turbines based on multi-source data fusion, applied to monitoring offshore wind turbines, characterized in that, The system includes: The data acquisition unit is used to collect the operating data of the offshore wind turbine, including vibration signals, operating noise signals and image signals. The time-frequency feature extraction unit is used to extract features from the running data to obtain vibration features, acoustic features, and image features; The temporal modeling unit is used to perform time-series modeling on the vibration features and the acoustic features to obtain temporal features, and is also used to perform temporal pooling on the image features to obtain a temporal aggregated feature vector. A multimodal fusion unit is used to fuse the temporal features and the temporal aggregated feature vector through a tensor fusion network to generate fused features; The defect identification unit is used to classify and locate the fused features, and output the fault type and confidence level; The structured pruning and dynamic gating unit is used to perform structured pruning and dynamic gating scheduling on the temporal modeling unit and the multimodal fusion unit according to the current task complexity and feature redundancy. Specifically, the structured pruning and dynamic gating unit is used to calculate the importance score of redundant channels during forward inference. The formula for calculating the importance score is: ； in, For importance scores, This indicates that the i-th channel corresponds to the j-th weight parameter. Indicates the total number of channel weight parameters; Based on a preset threshold, channels with importance scores below the preset threshold are dynamically pruned; The channel's switching is controlled by a gating function, the importance score, and the preset threshold. The expression for the gating function is: ； in, This represents the gating state of the i-th channel. It is the Sigmoid activation function. The preset threshold; The structured pruning and dynamic gating unit includes a pruning scheduling module; The pruning scheduling module is used to obtain the channel importance in historical running data. The channel importance distribution is obtained through the Fisher information matrix, and the calculation formula for the channel importance measurement parameter is as follows: ； in, For model parameters, For parameters Importance rating The expected value operator is used to evaluate input data. Find the average value. Let be the log-likelihood function. For the loss function with respect to parameters The gradient; The pruning scheduling module is also used to reduce overall inference latency by optimizing the objective, and the formula for minimizing the optimization objective is: ； in, The computational cost introduced by pruning, This is an indicator for whether the i-th channel is retained. Weights for channel computation; The structured pruning and dynamic gating unit also includes a threshold adaptive adjustment module; The threshold adaptive adjustment module is used to adjust the preset threshold according to the operating status of the offshore wind turbine. The formula used by the threshold adaptive adjustment module to adjust the preset threshold is: ； in, This represents the preset threshold for updating at time t+1. This represents the preset threshold at time t. For smoothing coefficients; The preset threshold is used to calculate the gating function.

2. The offshore wind turbine fault and defect identification system based on multi-source data fusion according to claim 1, characterized in that, The structured pruning and dynamic gating unit and the multimodal fusion unit are configured with a collaborative working mechanism, specifically: The multimodal fusion unit receives input feature vectors from each modality and modulates the features of each channel through the corresponding gating state to obtain feature tensors; The modulated feature tensor is decomposed into a low-rank decomposition using a tensor fusion network to obtain the decomposition vector. The structured pruning and dynamic gating unit retains the feature components corresponding to valid channels whose gating states are higher than the preset threshold, based on the gating state of each channel.

3. The offshore wind turbine fault and defect identification system based on multi-source data fusion according to claim 2, characterized in that, The expression for the tensor fusion network is: ； in, To fuse the feature tensor, R is the decomposition rank. The trainable vectors represent image features, vibration features, and acoustic features, respectively.

4. The offshore wind turbine fault and defect identification system based on multi-source data fusion according to claim 3, characterized in that, When the structured pruning and dynamic gating unit calculates the weight gradient of each channel, it proportionally weights the gradient of the channel according to the gating state corresponding to each channel. When the gating state of the channel is lower than the preset threshold, the gradient of the channel is adjusted by exponential decay.

5. The offshore wind turbine fault and defect identification system based on multi-source data fusion according to claim 4, characterized in that, The structured pruning and dynamic gating unit is also used for: Obtain the signal-to-noise ratio of the vibration signal and the proportion of background noise energy of the operating noise signal; When the signal-to-noise ratio of the vibration signal is lower than the signal-to-noise ratio threshold or the background noise energy ratio is lower than the background noise threshold, a signal quality factor is generated. The importance score for each channel is weighted and adjusted using the following formula: ； in, This is the corrected importance score. For signal quality factor, .

6. The offshore wind turbine fault and defect identification system based on multi-source data fusion according to claim 1, characterized in that, The data acquisition unit includes a multi-channel sensor array and an adaptive sampling module; The sensors in the multi-channel sensor array are located in the unmanned aerial vehicle and platform, as well as in the blades, gearboxes, and main bearings of the offshore wind turbine. The adaptive sampling module is used to adjust the sensor sampling parameters according to the real-time signal-to-noise ratio. The adaptive sampling module is also used to wake up the sensor according to historical abnormal data records and abnormal triggering mechanisms.

7. The offshore wind turbine fault and defect identification system based on multi-source data fusion according to claim 1, characterized in that, The system also includes: An alarm module is used to output an anomaly level based on the confidence level output by the defect identification unit, and to perform an alarm operation based on the anomaly level.

8. The offshore wind turbine fault and defect identification system based on multi-source data fusion according to claim 1, characterized in that, The structured pruning and dynamic gating unit is also used to obtain the accuracy of the confidence level and update the structured pruning and dynamic gating scheduling strategy based on the accuracy of the confidence level.

Images