A Multimodal Fault Diagnosis Method for Wind Turbines Based on Acoustic Print Features and Vibration Data Fusion

Abstract

This invention discloses a multimodal fault diagnosis method for wind turbines based on the fusion of acoustic signature features and vibration data. The method comprises the following steps: S1, synchronously collecting acoustic signature and vibration signals generated during wind turbine operation using multiple sensors; S2, preprocessing the collected acoustic signature and vibration signals; S3, fusing the preprocessed acoustic signature and vibration signals using a feature-level weighted fusion algorithm; S4, coarse-grained fault screening within a multi-level framework; S5, fine-grained fault diagnosis within the multi-level framework; S6, after fine-grained fault diagnosis, the system outputs corresponding fault warning signals based on the fault type, specific location, and severity; S7, dynamically adjusting the fault diagnosis model by monitoring the wind turbine's operating status and fault conditions in real time. This invention can accurately identify the fault type of the wind turbine and assess its severity, improving the overall intelligence level and real-time response capability of the fault diagnosis system.

Description

Technical Field

[0001] This invention relates to the field of wind turbine technology, and in particular to a multimodal fault diagnosis method for wind turbines based on the fusion of acoustic signature features and vibration data. Background Technology

[0002] In existing technologies, wind turbine fault diagnosis mainly relies on single-mode data, such as monitoring the wind turbine's operating status through vibration signals or acoustic signals. Traditional wind turbine fault diagnosis methods are usually based on vibration signal analysis, mainly judging the health status of the equipment by monitoring the changes in the vibration frequency, amplitude, and vibration spectrum of the wind turbine's mechanical components. Although traditional methods can detect wind turbine faults to a certain extent, they are prone to misdiagnosis or missed diagnosis due to their reliance on single-mode data and limitations caused by environmental noise, data acquisition accuracy, and external interference.

[0003] Another common method for wind turbine fault diagnosis is based on the monitoring of acoustic signals. By collecting the acoustic signals generated during wind turbine operation and analyzing the changes in frequency, amplitude, and harmonic characteristic signals, the fault of the equipment can be determined. However, due to the complex operating environment of wind turbines and the large amount of external noise interference, the accuracy and reliability of fault diagnosis using acoustic signals alone are limited. Existing acoustic signal processing methods often cannot make full use of the complementary information between other modal signals, resulting in insufficient sensitivity of fault detection.

[0004] Another drawback of existing technologies is their weak ability to warn of early-stage faults. Most traditional fault diagnosis systems can only issue an alarm when obvious faults occur in the equipment, and they usually cannot effectively detect minor or early-stage faults. This means that the wind turbine is only discovered after serious mechanical damage has occurred, which increases maintenance costs and shortens the service life of the equipment. Traditional fault diagnosis systems mostly rely on predefined rules or models and cannot be dynamically adjusted according to the actual operating conditions of the equipment. They are difficult to adapt to complex and ever-changing operating conditions, which reduces the accuracy and efficiency of fault diagnosis.

[0005] In summary, the existing technologies have the following main drawbacks: First, the use of single-modal data results in low accuracy and robustness of fault diagnosis, making it susceptible to external environmental factors and noise. Second, the existing technologies lack the ability to provide early warnings for early or minor faults, typically issuing alarms only when the fault is already apparent. Finally, traditional fault diagnosis systems lack the ability to adapt and adjust, making it difficult to cope with the complex and ever-changing wind turbine operating environment and unable to dynamically optimize the fault diagnosis model to improve detection effectiveness. Summary of the Invention

[0006] One objective of this invention is to propose a multimodal fault diagnosis method for wind turbines based on the fusion of acoustic signature features and vibration data. This invention can accurately identify the fault type of the wind turbine and assess its severity, thereby improving the overall intelligence level and real-time response capability of the fault diagnosis system.

[0007] A method for multimodal fault diagnosis of wind turbines based on the fusion of acoustic signature features and vibration data according to an embodiment of the present invention includes the following steps:

[0008] S1. Acoustic and vibration signals generated during the operation of the fan are collected synchronously by multiple sensors;

[0009] S2. Preprocess the collected acoustic and vibration signals;

[0010] S3. The preprocessed acoustic signature signal and vibration signal are fused using a feature-level weighted fusion algorithm to generate a multimodal feature vector. The multimodal feature vector integrates the key feature information of the acoustic signature signal and vibration signal.

[0011] S4. Coarse-grained fault screening in a multi-level framework: Using a pre-established simple fault mode library, fault types are screened through multi-modal feature vectors to preliminarily determine potential fault situations.

[0012] S5. Fine-grained fault diagnosis in a multi-level framework: For potential faults screened out at the coarse-grained level, a fault diagnosis model based on historical operating data and an unsupervised learning model is used to conduct in-depth analysis of multi-modal features, locate the specific location of the wind turbine fault according to the specific fault mode, and assess the severity of the fault.

[0013] S6. After fine-grained fault diagnosis is completed, the system outputs corresponding fault warning signals according to the type, specific location and severity of the fault. When a minor or early fault is detected, a warning signal is generated in a timely manner and fault information and corresponding maintenance suggestions are provided.

[0014] S7. The system monitors the operating status and fault conditions of the fan in real time, dynamically adjusts the fault diagnosis model, and updates the fault diagnosis model parameters based on newly collected acoustic and vibration signal data.

[0015] Optionally, S1 further includes:

[0016] S11. Multiple acoustic sensors M and vibration sensors arranged near the wind turbine blades, bearings and other key components are used to synchronously collect the acoustic and vibration signals generated during the operation of the wind turbine. The acoustic sensors are used to capture the sound wave signals of the wind turbine under different operating conditions, and the vibration sensors are used to detect the vibration amplitude and frequency of the wind turbine's mechanical components.

[0017] S12, the acoustic signal S(t, x, y, z) is the acoustic signal generated by the wind turbine blades, bearings and other key components during operation:

[0018] ;

[0019] in, Let be the amplitude of the nth acoustic mode. Where is the attenuation coefficient, and r is the sound wave propagation distance. Let n be the frequency of the nth acoustic mode. The phase angle, Let P(x, y, z) be the angle between the direction of the sound source and the direction of the sensor, P(x, y, z) be the distribution function of the sound source in space, and N be the number of modes.

[0020] S13, Vibration signal Vibration data of mechanical components during wind turbine operation:

[0021] ;

[0022] in, Let m be the amplitude of the m-th vibration mode. Let m be the frequency of the m-th vibration mode. The phase angle, The frequency shift of the vibration mode. The nonlinear exponent of frequency drift, denoted as the attenuation coefficient of the vibration signal, d as the distance or displacement of the mechanical structure, and M as the number of modes.

[0023] Optionally, S2 further includes:

[0024] S21. The acquired acoustic signature signal S(t, x, y, z) is denoised by using a bandpass filter to filter out environmental noise components within the frequency range, retaining the effective acoustic signal components during fan operation. The filtered signal is denoted as... ;

[0025] S22. The denoised voiceprint signal Perform feature extraction, extracting frequency features. Amplitude characteristics Harmonic characteristics Frequency characteristics The amplitude characteristics are determined by analyzing the dominant frequency component of the signal. Harmonic characteristics were obtained through signal envelope detection. The amplitude information of each harmonic is obtained by Fourier transform;

[0026] S23. The collected vibration signals To denoise the vibration signal, wavelet transform was used to decompose and reconstruct the signal, removing high-frequency noise components to obtain the denoised vibration signal. ;

[0027] S24. The denoised vibration signal Feature extraction is performed to extract the time-domain features of acceleration. Acceleration spectrum characteristics and velocity spectrum characteristics acceleration time-domain characteristics The peak acceleration of the vibration signal, and the acceleration spectrum characteristics. The velocity spectrum characteristics are obtained through Fast Fourier Transform. The velocity characteristics are obtained by integrating the acceleration spectrum.

[0028] S25. Perform signal segmentation processing on the pre-processed acoustic and vibration signals, and divide the signals into multiple time windows according to the fan's operating cycle and working status.

[0029] Optionally, S3 further includes:

[0030] S31. The preprocessed frequency characteristics Amplitude characteristics Harmonic characteristics and acceleration time domain characteristics Acceleration spectrum characteristics and velocity spectrum characteristics For the input features, initially set the weighting coefficients for each feature. The weighting coefficients represent the importance of each feature in multimodal feature fusion;

[0031] S32. The weighting coefficients of each feature are iteratively adjusted using the Bayesian optimization algorithm. The Bayesian optimization model aims to minimize the error of wind turbine fault mode identification with the objective function.

[0032] S33. The Bayesian optimization algorithm evaluates each set of weighted coefficients by constructing a probabilistic model and gradually optimizes the weighted coefficients based on the observed data. The updated weighted coefficients... This minimizes the objective loss function, allowing the weighted fused multimodal feature vector to reflect the operating status of the wind turbine under different working conditions.

[0033] S34. Based on the optimized weighting coefficients For frequency characteristics Amplitude characteristics Harmonic characteristics and acceleration time domain characteristics Acceleration spectrum characteristics and velocity spectrum characteristics Perform feature weighted fusion to generate the final multimodal feature vector. :

[0034] ;

[0035] in, This is the weighted fusion of multimodal feature vectors, which contains all the key feature information of the acoustic and vibration signals.

[0036] Optionally, S32 further includes:

[0037] S321. In Bayesian optimization, the objective function of the weighted feature vector fusion model is defined as:

[0038] ;

[0039] Among them, L( ) is the loss function optimized by Bayes. To achieve multimodal feature vectors through weighted fusion, For the weight vector, The output prediction of the fault diagnosis model is given by y, where y is the actual fault label. Let X be the distribution density of the feature vector X, where X is the feature space vector;

[0040] S322. In the Bayesian optimization process, the objective function is optimized by constructing a Gaussian process regression model. The objective function is approximated, and iterative updates are performed using the criterion of maximizing the expected improvement:

[0041] ;

[0042] in, For the desired improvement value, Let be the multimodal feature vector at the t-th iteration. These are parameters used to adjust and optimize the degree of exploration;

[0043] S323, Weight Vector The update is expressed by maximizing the expected improvement value as:

[0044] .

[0045] Optionally, S4 further includes:

[0046] S41, Based on multimodal feature vectors The common fault types are coarsely screened using a pre-established simple fault mode library M. The simple fault mode library includes fault type feature modes such as wind turbine bearing damage, wind turbine blade cracks, and wind turbine imbalance.

[0047] S42. Determine the similarity between the multimodal feature vector and the simple fault mode using the similarity calculation formula. Combine the acoustic signature features and vibration features to define the weighted similarity for each simple fault mode. :

[0048] ;

[0049] in, Let i be the feature vector of the i-th simple fault mode. The weighting coefficients for each feature dimension. These are nonlinear adjustment parameters for acoustic signature signals or vibration signals. Here, T represents the frequency component and T represents the time period.

[0050] S43. For each group of simple fault modes, filter them based on similarity values ​​and set a similarity threshold. And define the fault severity coefficient. :

[0051] ;

[0052] in, This represents the severity coefficient of simple failure mode i. To adjust the coefficient, A threshold associated with simple fault mode i; when the threshold is exceeded, it is considered a potential fault. A value close to 1 indicates a higher risk of failure.

[0053] S44, for the calculated The system performs classification by clustering the results of coarse-grained fault screening, defining cluster centers to determine the cluster affiliation of simple fault modes, and determining the specific classification of faults by judging the cluster affiliation value, thus generating coarse-grained fault screening results.

[0054] Optionally, S5 further includes:

[0055] S51. For multimodal feature vectors identified as potential faults through coarse-grained fault screening. In-depth analysis is conducted using an unsupervised learning model built based on historical operational data. The fault diagnosis model D, trained using unsupervised learning methods, is then analyzed. Output the clustering results of the failure modes;

[0056] S52. Utilize self-organizing mapping networks to process historical operational data. Cluster analysis is performed to generate a topological mapping of wind turbine failure modes. The self-organizing mapping network then converts high-dimensional multimodal feature vectors. Mapped onto a low-dimensional grid, grid nodes The cluster centers are represented, and each node represents a failure mode.

[0057] S53, By calculating multimodal feature vectors With self-organizing mapping network nodes Euclidean distance Identify the current fault mode:

[0058] ;

[0059] in, The distance is Euclidean distance, used to measure the distance between the current failure mode and the nodes of the self-organizing map network. The node with the minimum distance is the representative of the failure type.

[0060] S54. Based on the nearest self-organizing map network node distance Assess the severity of the fault. This is the fault severity coefficient, with a value range of [0, 1]. The closer the value is to 1, the more severe the fault. It is used to perform fault assessment and fine-grained diagnosis in combination with historical data.

[0061] Optionally, S6 further includes:

[0062] S61. When minor or early-stage faults are identified through fine-grained fault diagnosis, the severity coefficient is used to determine the severity of the fault. Set the value of the warning threshold. ,when At that time, the system generates an early warning signal;

[0063] S62. The generated warning signal includes the fault type and fault severity. The fault types include the following categories:

[0064] Wind turbine bearing wear: a common minor fault type, usually accompanied by low-frequency vibration and medium-amplitude acoustic wave signals;

[0065] Wind turbine blade cracks: an early-stage failure, manifested as harmonic variations in high-frequency acoustic signatures and slight shifts in the vibration signal spectrum;

[0066] Fan imbalance: a minor fault type, manifested by low-frequency vibration characteristics and amplitude changes in periodic acoustic signals;

[0067] Motor overheating or malfunction: Identified by abnormal increase in noise components in the acoustic signal and persistent deviation in vibration characteristics;

[0068] The severity of the fault is categorized into the following levels:

[0069] Minor fault: Preventive checks are required, but they will not immediately affect the operation of the fan.

[0070] Moderate fault: It needs to be maintained in a timely manner to prevent the malfunction from worsening.

[0071] Critical fault: Immediately trigger the alarm and arrange for a shutdown for maintenance;

[0072] S63. Based on the fault type and severity, and combined with historical maintenance data in the system, maintenance suggestions are automatically generated. For minor faults, the system suggests routine checks and monitoring. For moderate and severe faults, the system will suggest replacing relevant components and provide detailed maintenance steps, including shutdown checks, component replacement, and lubrication.

[0073] The beneficial effects of this invention are:

[0074] (1) This invention breaks through the limitations of traditional single-mode fault diagnosis methods by performing feature-level weighted fusion of acoustic features and vibration data. The Bayesian optimization algorithm is used to dynamically adjust the weighting coefficient of each feature, so that the feature vector after multi-modal feature fusion can more accurately reflect the operating status of the wind turbine. In traditional fault diagnosis methods, relying solely on acoustic or vibration data is easily affected by external interference such as environmental noise and data errors. However, this invention improves the accuracy and anti-interference ability of fault diagnosis through feature fusion. Under complex working conditions, the adaptive optimization of the weighting coefficient ensures the robustness and stability of the diagnosis results under different wind turbine operating conditions, significantly reducing the risk of misdiagnosis and missed diagnosis.

[0075] (2) By introducing a fault severity coefficient and a warning threshold, this invention effectively achieves accurate identification and timely warning of early or minor faults in wind turbines. Compared with traditional systems that can only alarm when the fault is obvious, this invention, based on fine-grained fault screening, uses an unsupervised learning model and a self-organizing mapping network to conduct in-depth analysis of fault modes and combine historical operating data. The system can capture the most subtle abnormal signals during wind turbine operation and quantify the risk level of the fault through the severity coefficient, generating timely warning signals and corresponding maintenance suggestions. This greatly improves the sensitivity and accuracy of fault warning and avoids major mechanical damage caused by failure to detect early faults in time during wind turbine operation.

[0076] (3) This invention combines historical data and real-time monitoring data to dynamically update the weights and parameters of the fault diagnosis model, enabling the diagnosis model to adapt to different wind turbine operating conditions. In traditional fault diagnosis systems, fault modes and diagnosis models are usually static and difficult to adapt to changes in operating conditions during long-term wind turbine operation, resulting in decreased diagnostic accuracy. By using Bayesian optimization and self-organizing mapping network to dynamically adjust the parameters of the diagnosis model, it ensures that the system can accurately identify the fault type of the wind turbine and assess its severity under new operating conditions and new fault modes, thereby improving the overall intelligence level and real-time response capability of the fault diagnosis system. Attached Figure Description

[0077] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0078] Figure 1 The flowchart shows a method for multimodal fault diagnosis of wind turbines based on the fusion of acoustic signature features and vibration data proposed in this invention.

[0079] Figure 2 This is a schematic diagram of the structure of a wind turbine fault diagnosis model, which is a wind turbine multimodal fault diagnosis method based on the fusion of acoustic signature features and vibration data proposed in this invention. Detailed Implementation

[0080] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0081] refer to Figure 1-2 A multimodal fault diagnosis method for wind turbines based on the fusion of acoustic signature features and vibration data includes the following steps:

[0082] S1. Acoustic and vibration signals generated during the operation of the fan are collected synchronously by multiple sensors;

[0083] S2. Preprocess the collected acoustic and vibration signals;

[0084] S3. The preprocessed acoustic signature signal and vibration signal are fused using a feature-level weighted fusion algorithm to generate a multimodal feature vector. The multimodal feature vector integrates the key feature information of the acoustic signature signal and vibration signal.

[0085] S4. Coarse-grained fault screening in a multi-level framework: Using a pre-established simple fault mode library, fault types are screened through multi-modal feature vectors to preliminarily determine potential fault situations.

[0086] S5. Fine-grained fault diagnosis in a multi-level framework: For potential faults screened out at the coarse-grained level, a fault diagnosis model based on historical operating data and an unsupervised learning model is used to conduct in-depth analysis of multi-modal features, locate the specific location of the wind turbine fault according to the specific fault mode, and assess the severity of the fault.

[0087] S6. After fine-grained fault diagnosis is completed, the system outputs corresponding fault warning signals according to the type, specific location and severity of the fault. When a minor or early fault is detected, a warning signal is generated in a timely manner and fault information and corresponding maintenance suggestions are provided.

[0088] S7. The system monitors the operating status and fault conditions of the fan in real time, dynamically adjusts the fault diagnosis model, and updates the fault diagnosis model parameters based on newly collected acoustic and vibration signal data.

[0089] In this embodiment, S1 further includes:

[0090] S11. Multiple acoustic sensors M and vibration sensors arranged near the wind turbine blades, bearings and other key components are used to synchronously collect the acoustic and vibration signals generated during the operation of the wind turbine. The acoustic sensors are used to capture the sound wave signals of the wind turbine under different operating conditions, and the vibration sensors are used to detect the vibration amplitude and frequency of the wind turbine's mechanical components.

[0091] S12, the acoustic signal S(t, x, y, z) is the acoustic signal generated by the wind turbine blades, bearings and other key components during operation:

[0092] ;

[0093] in, Let be the amplitude of the nth acoustic mode. Where is the attenuation coefficient, and r is the sound wave propagation distance. Let n be the frequency of the nth acoustic mode. The phase angle, Let P(x, y, z) be the angle between the direction of the sound source and the direction of the sensor, P(x, y, z) be the distribution function of the sound source in space, and N be the number of modes.

[0094] S13, Vibration signal Vibration data of mechanical components during wind turbine operation:

[0095] ;

[0096] in, Let m be the amplitude of the m-th vibration mode. Let m be the frequency of the m-th vibration mode. The phase angle, The frequency shift of the vibration mode. The nonlinear exponent of frequency drift, denoted as the attenuation coefficient of the vibration signal, d as the distance or displacement of the mechanical structure, and M as the number of modes.

[0097] In this embodiment, S2 further includes:

[0098] S21. The acquired acoustic signature signal S(t, x, y, z) is denoised by using a bandpass filter to filter out environmental noise components within the frequency range, retaining the effective acoustic signal components during fan operation. The filtered signal is denoted as... ;

[0099] S22. The denoised voiceprint signal Perform feature extraction, extracting frequency features. Amplitude characteristics Harmonic characteristics Frequency characteristics The amplitude characteristics are determined by analyzing the dominant frequency component of the signal. Harmonic characteristics were obtained through signal envelope detection. The amplitude information of each harmonic is obtained by Fourier transform;

[0100] S23. The collected vibration signals To denoise the vibration signal, wavelet transform was used to decompose and reconstruct the signal, removing high-frequency noise components to obtain the denoised vibration signal. ;

[0101] S24. The denoised vibration signal Feature extraction is performed to extract the time-domain features of acceleration. Acceleration spectrum characteristics and velocity spectrum characteristics acceleration time-domain characteristics The peak acceleration of the vibration signal, and the acceleration spectrum characteristics. The velocity spectrum characteristics are obtained through Fast Fourier Transform. The velocity characteristics are obtained by integrating the acceleration spectrum.

[0102] S25. Perform signal segmentation processing on the pre-processed acoustic and vibration signals, and divide the signals into multiple time windows according to the fan's operating cycle and working status.

[0103] In this embodiment, S3 further includes:

[0104] S31. The preprocessed frequency characteristics Amplitude characteristics Harmonic characteristics and acceleration time domain characteristics Acceleration spectrum characteristics and velocity spectrum characteristics For the input features, initially set the weighting coefficients for each feature. The weighting coefficients represent the importance of each feature in multimodal feature fusion;

[0105] S32. The weighting coefficients of each feature are iteratively adjusted using the Bayesian optimization algorithm. The Bayesian optimization model aims to minimize the error of wind turbine fault mode identification with the objective function.

[0106] S33. The Bayesian optimization algorithm evaluates each set of weighted coefficients by constructing a probabilistic model and gradually optimizes the weighted coefficients based on the observed data. The updated weighted coefficients... This minimizes the objective loss function, allowing the weighted fused multimodal feature vector to reflect the operating status of the wind turbine under different working conditions.

[0107] S34. Based on the optimized weighting coefficients For frequency characteristics Amplitude characteristics Harmonic characteristics and acceleration time domain characteristics Acceleration spectrum characteristics and velocity spectrum characteristics Perform feature weighted fusion to generate the final multimodal feature vector. :

[0108] ;

[0109] in, This is the weighted fusion of multimodal feature vectors, which contains all the key feature information of the acoustic and vibration signals.

[0110] In this embodiment, S32 further includes:

[0111] S321. In Bayesian optimization, the objective function of the weighted feature vector fusion model is defined as:

[0112] ;

[0113] Among them, L( ) is the loss function optimized by Bayes. To achieve multimodal feature vectors through weighted fusion, For the weight vector, The output prediction of the fault diagnosis model is given by y, where y is the actual fault label. Let X be the distribution density of the feature vector X, where X is the feature space vector;

[0114] S322. In the Bayesian optimization process, the objective function is optimized by constructing a Gaussian process regression model. The objective function is approximated, and iterative updates are performed using the criterion of maximizing the expected improvement:

[0115] ;

[0116] in, For the desired improvement value, Let be the multimodal feature vector at the t-th iteration. These are parameters used to adjust and optimize the degree of exploration;

[0117] S323, Weight Vector The update is expressed by maximizing the expected improvement value as:

[0118] .

[0119] In this embodiment, S4 further includes:

[0120] S41, Based on multimodal feature vectors The common fault types are coarsely screened using a pre-established simple fault mode library M. The simple fault mode library includes fault type feature modes such as wind turbine bearing damage, wind turbine blade cracks, and wind turbine imbalance.

[0121] S42. Determine the similarity between the multimodal feature vector and the simple fault mode using the similarity calculation formula. Combine the acoustic signature features and vibration features to define the weighted similarity for each simple fault mode. :

[0122] ;

[0123] in, Let i be the feature vector of the i-th simple fault mode. The weighting coefficients for each feature dimension. These are nonlinear adjustment parameters for acoustic signature signals or vibration signals. Here, T represents the frequency component and T represents the time period.

[0124] S43. For each group of simple fault modes, filter them based on similarity values ​​and set a similarity threshold. And define the fault severity coefficient. :

[0125] ;

[0126] in, This represents the severity coefficient of simple failure mode i. To adjust the coefficient, A threshold associated with simple fault mode i; when the threshold is exceeded, it is considered a potential fault. A value close to 1 indicates a higher risk of failure.

[0127] S44, for the calculated The system performs classification by clustering the results of coarse-grained fault screening, defining cluster centers to determine the cluster affiliation of simple fault modes, and determining the specific classification of faults by judging the cluster affiliation value, thus generating coarse-grained fault screening results.

[0128] In this embodiment, S5 further includes:

[0129] S51. For multimodal feature vectors identified as potential faults through coarse-grained fault screening. In-depth analysis is conducted using an unsupervised learning model built based on historical operational data. The fault diagnosis model D, trained using unsupervised learning methods, is then analyzed. Output the clustering results of the failure modes;

[0130] S52. Utilize self-organizing mapping networks to process historical operational data. Cluster analysis is performed to generate a topological mapping of wind turbine failure modes. The self-organizing mapping network then converts high-dimensional multimodal feature vectors. Mapped onto a low-dimensional grid, grid nodes The cluster centers are represented, and each node represents a failure mode.

[0131] S53, By calculating multimodal feature vectors With self-organizing mapping network nodes Euclidean distance Identify the current fault mode:

[0132] ;

[0133] in, The distance is Euclidean distance, used to measure the distance between the current failure mode and the nodes of the self-organizing map network. The node with the minimum distance is the representative of the failure type.

[0134] S54. Based on the nearest self-organizing map network node distance Assess the severity of the fault. This is the fault severity coefficient, with a value range of [0, 1]. The closer the value is to 1, the more severe the fault. It is used to perform fault assessment and fine-grained diagnosis in combination with historical data.

[0135] In this embodiment, S6 further includes:

[0136] S61. When minor or early-stage faults are identified through fine-grained fault diagnosis, the severity coefficient is used to determine the severity of the fault. Set the value of the warning threshold. ,when At that time, the system generates an early warning signal;

[0137] S62. The generated warning signal includes the fault type and fault severity. The fault types include the following categories:

[0138] Wind turbine bearing wear: a common minor fault type, usually accompanied by low-frequency vibration and medium-amplitude acoustic wave signals;

[0139] Wind turbine blade cracks: an early-stage failure, manifested as harmonic variations in high-frequency acoustic signatures and slight shifts in the vibration signal spectrum;

[0140] Fan imbalance: a minor fault type, manifested by low-frequency vibration characteristics and amplitude changes in periodic acoustic signals;

[0141] Motor overheating or malfunction: Identified by abnormal increase in noise components in the acoustic signal and persistent deviation in vibration characteristics;

[0142] The severity of the fault is categorized into the following levels:

[0143] Minor fault: Preventive checks are required, but they will not immediately affect the operation of the fan.

[0144] Moderate fault: It needs to be maintained in a timely manner to prevent the malfunction from worsening.

[0145] Critical fault: Immediately trigger the alarm and arrange for a shutdown for maintenance;

[0146] S63. Based on the fault type and severity, and combined with historical maintenance data in the system, maintenance suggestions are automatically generated. For minor faults, the system suggests routine checks and monitoring. For moderate and severe faults, the system will suggest replacing relevant components and provide detailed maintenance steps, including shutdown checks, component replacement, and lubrication.

[0147] Example 1:

[0148] In August 2023, the operation and maintenance team of a coastal wind farm discovered that the failure rate of blades and bearing components of the regional wind turbines was relatively high due to long-term exposure to high humidity and high salt environment. In order to improve the stability of the equipment and extend the service life of the wind turbines, the team introduced a multi-modal fault diagnosis system based on the fusion of acoustic features and vibration data.

[0149] On August 12, during the operation of the No. 18 wind turbine at the wind farm, with blade number "Blade-18-A", the system monitored the turbine's operating status in real time. The turbine's acoustic and vibration sensors were installed at the blade root, bearing, and near the motor, respectively. The data acquisition system collected data every 5 minutes and input it into a multimodal data fusion algorithm for real-time processing.

[0150] At 10:21 AM on August 15th, the system detected an anomaly in the acoustic signature signal of wind turbine No. 18. The frequency characteristics suddenly increased, and the amplitude fluctuated significantly. The vibration sensor showed signs of frequency shift in the acceleration spectrum, especially a sudden increase in the vibration signal in the low-frequency range (0-200Hz). After analysis by the fusion algorithm, the system generated a fault severity coefficient of 0.42, exceeding the preset warning threshold of 0.4. At this point, the system determined that there might be a crack in the blade and generated a warning signal.

[0151] Data analysis and maintenance decisions:

[0152] Within 5 minutes of receiving the warning signal, the system will push detailed fault information to on-site maintenance personnel via SMS and email. The fault report includes the following information:

[0153] Fault time: 10:21 AM on August 15, 2023;

[0154] Fan number: Fan No. 18;

[0155] Blade number: Blade-18-A;

[0156] Fault type: Blade crack;

[0157] Fault severity: 0.42 (moderate fault);

[0158] Vibration signal characteristics: acceleration spectrum shift, enhanced low-frequency vibration;

[0159] Voiceprint signal characteristics: amplitude fluctuations, increased frequency characteristics;

[0160] Based on the maintenance recommendations provided by the system, the maintenance team arranged a routine inspection that afternoon. At 2:30 p.m., the maintenance personnel found a slight crack at the root of the blade of wind turbine No. 18. The crack was 8 centimeters long, but it had not yet spread to the critical structure. The team used special materials to repair the crack according to the recommendations, which prevented more serious failures in the future. The entire inspection and repair process lasted about 3 hours, and the wind turbine did not stop running.

[0161] On the adjacent No. 19 wind turbine, no multimodal fault diagnosis system was installed; instead, traditional vibration signal monitoring methods were used. While No. 19 wind turbine operated during the same period, the blade cracks were not detected in time, and the turbine continued to run until early September when it experienced a sudden shutdown. Upon inspection, severe cracks were found on the turbine blades, with a crack length of 35 centimeters, necessitating a shutdown for blade replacement. The entire repair process took 48 hours and cost 30,000 RMB.

[0162] Table 1. Comparison data between fan No. 18 (using a multi-modal fault diagnosis system) and fan No. 19 (using a traditional fault diagnosis method).

[0163] project Fan No. 18 (Multimodal System) Fan No. 19 (traditional method) Detection time August 15, 2023 September 3, 2023 Fault type Blade cracks Blade cracks Detection method Multimodal data fusion Single vibration signal monitoring Voiceprint signal Frequency increases, amplitude fluctuates Unable to monitor Vibration signal Low-frequency acceleration enhancement, spectral shift No significant changes Fault Severity Coefficient 0.42 none Warning time August 15, 2023, 10:21 AM No warning Crack length 8 centimeters 35 cm Repair time 3 hours 48 hours Downtime No downtime 48 hours Repair costs 3000 yuan 30,000 yuan Should an early warning signal be generated? yes no Maintenance Recommendations Routine inspection, repairing cracks No, shut down immediately.

[0164] On September 5, 2023, the maintenance team re-inspected wind turbine No. 18. After the blades were repaired, the turbine operated normally without any new fault signals. Through the diagnosis and comparison of this simulated fault, the team found that the early warning mechanism of the multimodal fault diagnosis system can effectively prevent major damage to key components of the wind turbine. During a month of continuous operation monitoring, the system generated a total of three early fault warnings, corresponding to minor bearing wear and blade cracks, respectively, and maintenance measures were taken before the faults became severe.

[0165] The testing and application of this wind turbine fault diagnosis system have proven the effectiveness of the invention. The multimodal data fusion technology significantly improves the accuracy and sensitivity of wind turbine fault diagnosis. Compared with traditional vibration monitoring technology, the system can detect potential faults in wind turbine operation earlier and generate maintenance suggestions, greatly reducing downtime and maintenance costs caused by wind turbine faults. In addition, the system optimizes the operation and maintenance cycle of wind turbines, reduces unnecessary inspection and maintenance frequency, and thus improves power generation efficiency.

[0166] This invention overcomes the limitations of traditional single-modal fault diagnosis methods by performing feature-level weighted fusion of acoustic signature features and vibration data. It employs a Bayesian optimization algorithm to dynamically adjust the weighting coefficients of each feature, enabling the fused feature vector to more accurately reflect the wind turbine's operating status. Traditional fault diagnosis methods, relying solely on acoustic signature or vibration data, are easily affected by environmental noise and external interference from data errors. This invention, however, improves the accuracy and anti-interference capability of fault diagnosis through feature fusion. Under complex operating conditions, adaptive optimization of the weighting coefficients ensures the robustness and stability of diagnostic results under different wind turbine operating conditions, significantly reducing the risk of misdiagnosis and missed diagnosis.

[0167] This invention effectively identifies and provides timely warnings for early or minor faults in wind turbines by introducing a fault severity coefficient and a warning threshold. Compared to traditional systems that can only issue alarms when faults are obvious, this invention, based on fine-grained fault screening, uses an unsupervised learning model and a self-organizing mapping network to conduct in-depth analysis of fault modes and combine them with historical operating data. The system can capture the most subtle abnormal signals during wind turbine operation and quantify the risk level of the fault through a severity coefficient, generating timely warning signals and corresponding maintenance suggestions. This significantly improves the sensitivity and accuracy of fault warnings and avoids major mechanical damage caused by the failure to detect early faults in a timely manner during wind turbine operation.

[0168] This invention dynamically updates the weights and parameters of the fault diagnosis model by combining historical data and real-time monitoring data, enabling the diagnostic model to adapt to different wind turbine operating conditions. In traditional fault diagnosis systems, fault modes and diagnostic models are usually static, making it difficult to adapt to changes in operating conditions during long-term wind turbine operation, resulting in decreased diagnostic accuracy. By using Bayesian optimization and self-organizing mapping networks to dynamically adjust the parameters of the diagnostic model, this invention ensures that the system can accurately identify the fault type of the wind turbine and assess its severity under new operating conditions and new fault modes, thereby improving the overall intelligence level and real-time response capability of the fault diagnosis system.

[0169] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A fan multi-modal fault diagnosis method based on voiceprint feature and vibration data fusion, characterized in that, The method comprises the following steps: S1, synchronously collecting the soundprint signals and vibration signals generated during the operation of the fan through multiple sensors; S2, preprocessing the collected soundprint signals and vibration signals; S3, fusing the preprocessed soundprint signals and vibration signals through a feature level weighted fusion algorithm to generate a multi-modal feature vector, which comprehensively integrates the key feature information of the soundprint signals and vibration signals; S4, coarse-grained fault screening under a multi-level framework: using a pre-established simple fault mode library, screening the fault type through the multi-modal feature vector, and preliminarily judging the potential fault condition; S5, fine-grained fault diagnosis under a multi-level framework: for the potential fault screened by the coarse-grained level, a fault diagnosis model based on historical operation data and an unsupervised learning model is used to deeply analyze the multi-modal features, determine the specific position of the fan fault according to the specific fault model, and evaluate the severity of the fault; S6, after the fine-grained fault diagnosis is completed, the system outputs the corresponding fault warning signal according to the type, specific position and severity of the fault, generates a warning signal in time when a slight or early fault is detected, and provides fault information and corresponding maintenance suggestions; S7, the system dynamically adjusts the fault diagnosis model by real-time monitoring the operation state and fault condition of the fan, and updates the parameters of the fault diagnosis model according to the newly collected soundprint signals and vibration signals.

2. The fan multi-modal fault diagnosis method based on the fusion of voiceprint features and vibration data according to claim 1, characterized in that, The S1 further comprises: S11, synchronously collecting the soundprint signals and vibration signals generated during the operation of the fan by using multiple acoustic sensors and vibration sensors arranged near the fan blades, bearings and other key components, the acoustic sensors being used to capture the sound wave signals of the fan under different working conditions, and the vibration sensors being used to detect the vibration amplitude and frequency of the mechanical components of the fan; S12, the soundprint signal S(t, x, y, z) is an acoustic signal generated by the fan blades, bearings and other key components during operation: ； wherein, is the amplitude of the nth acoustic mode, is the attenuation coefficient, r is the acoustic wave propagation distance, is the frequency of the nth acoustic mode, is the phase angle, is the angle between the sound source direction and the sensor direction, P(x, y, z) is the distribution function of the sound source in space, and N is the number of modes. S13, vibration signal Vibration data of mechanical parts during operation of the fan: ； wherein, Amis the amplitude of the mth vibration mode, fmis the frequency of the mth vibration mode, is the phase angle, is the frequency drift of the vibration mode, is the non-linear index of the frequency drift, is the attenuation coefficient of the vibration signal, d is the distance or displacement of the mechanical structure, and M is the number of modes.

3. The fan multi-modal fault diagnosis method based on the fusion of voiceprint features and vibration data according to claim 2, characterized in that, The S2 further comprises: S21, the collected voiceprint signal S(t, x, y, z) is denoised, a band-pass filter is used to filter the environmental noise components in the frequency range, and the effective acoustic signal components during the fan operation are retained, and the filtered signal is denoted as ; S22, the voiceprint signal after denoising perform feature extraction, extract frequency features , amplitude features and harmonic features , frequency features determined by analyzing the main frequency component of the signal, amplitude features obtained by signal envelope detection, harmonic features obtain the amplitude information of each order harmonic through Fourier transform; S23, the collected vibration signal S23, the collected vibration signal ; S24, the de-noised vibration signal feature extraction is performed to extract acceleration time domain features , acceleration spectrum features and speed spectrum features , acceleration time domain features are peak accelerations of the vibration signal, acceleration spectrum features are obtained by fast Fourier transform, speed spectrum features are speed features obtained by integrating the acceleration spectrum; S25, performing signal segmentation processing on the preprocessed soundprint signals and vibration signals, and dividing the signals into multiple time windows according to the operation period and working state of the fan.

4. The fan multi-modal fault diagnosis method based on the fusion of voiceprint features and vibration data according to claim 3, characterized in that, The S3 further comprises: S31. The preprocessed frequency characteristics Amplitude characteristics Harmonic characteristics and acceleration time domain characteristics Acceleration spectrum characteristics and velocity spectrum characteristics For the input features, initially set the weighting coefficients for each feature. The weighting coefficients represent the importance of each feature in multimodal feature fusion; S32, iteratively adjusting the weighting coefficient of each feature through a Bayesian optimization algorithm, and taking the error minimization of the fan fault mode recognition as the objective function of the Bayesian optimization model; S33, the Bayesian optimization algorithm evaluates each set of weighted coefficients by constructing a probabilistic model, and gradually optimizes the weighted coefficients according to the observation data, and the updated weighted coefficients Minimize the target loss function, so that the multi-modal feature vector after weighted fusion reflects the running state of the fan under different working conditions. S34、According to the optimized weighting coefficient , frequency characteristics , amplitude characteristics , harmonic characteristics , and acceleration time domain characteristics , acceleration spectrum characteristics , and velocity spectrum characteristics , the feature weighting fusion is carried out to generate the final multi-modal feature vector : ； wherein, is the weighted fused multi-modal feature vector, containing all key feature information of the voiceprint signal and the vibration signal.

5. The fan multi-modal fault diagnosis method based on the fusion of voiceprint features and vibration data according to claim 4, characterized in that, The S32 further comprises: S321, defining the objective function of the weighted feature vector fusion model in the Bayesian optimization as: ； wherein L( ) is a loss function of Bayesian optimization, is a multi-modal feature vector fused by weighting, is a weight vector, is an output prediction of the fault diagnosis model, y is an actual fault label, is a distribution density of the feature vector X, and X is a feature space vector. S322、In the Bayesian optimization process, the optimization of the target function is performed by constructing a Gaussian process regression model The target function is approximated and iteratively updated by maximizing the expected improvement criterion: ； wherein, is a desired improvement value, is a multimodal feature vector at the tth iteration, is a parameter for adjusting the degree of optimization exploration; S323、weight vector The update of the weight vector is expressed by maximizing the expected improvement value as 。 6. The fan multi-modal fault diagnosis method based on the fusion of voiceprint features and vibration data according to claim 1, characterized in that, The S4 further comprises: S41、based on the multi-modal feature vector A pre-established simple fault mode library is used to coarsely screen common fault types, and the simple fault mode library includes fault type characteristic modes of fan bearing damage, fan blade crack and fan imbalance. S42, determine the similarity between the multi-modal feature vector and the simple fault mode through a similarity calculation formula, combine the voiceprint feature and the vibration feature, and define the weighted similarity of each simple fault mode : ； wherein, is a feature vector of the i-th simple failure mode, is a weighting coefficient of each feature dimension, is a nonlinear adjustment parameter of the voiceprint signal or the vibration signal, is a frequency component, and T is a time period. S43, for each set of simple failure modes, filtering by the value of similarity, setting a threshold of similarity and defining a failure severity coefficient : ； wherein is a severity coefficient for the simple failure mode i, is an adjustment coefficient, is a threshold value related to the simple failure mode i, beyond which a potential failure is judged, when approaches 1, a higher risk of failure is indicated; S44, classifying the calculated The classification is performed by clustering the results of the coarse-grained fault screening, defining a cluster center to determine the cluster attribution of the simple fault mode, determining the specific classification of the fault by judging the value of the cluster attribution, and generating the coarse-grained fault screening result.

7. The fan multi-modal fault diagnosis method based on the fusion of voiceprint features and vibration data according to claim 1, characterized in that, The S5 further comprises: S51、For the multi-modal feature vector determined as a potential fault by the coarse-grained fault screening , in-depth analysis using an unsupervised learning model constructed based on historical operation data, a fault diagnosis model D trained by an unsupervised learning method, and a clustering result of the fault mode output by the fault diagnosis model D S52, using a self-organizing mapping network on historical operation data to perform clustering analysis and generate a topological mapping of fan failure modes, the self-organizing mapping network mapping high-dimensional multi-modal feature vectors onto a low-dimensional grid, the grid nodes representing cluster centers, each node representing a failure mode; S53, identifying a current failure mode by computing a multi-modal feature vector with self-organizing map network nodes euclidean distance , identifying a current failure mode: ； wherein, is the Euclidean distance for measuring the distance between the current fault mode and the nodes of the self-organizing map, and the node corresponding to the minimum distance is the representative of the fault type. S54. According to the distance to the closest self-organizing map network node Assess the severity of the fault.​

Images