Fan multi-mode fault diagnosis method based on voiceprint feature and vibration data fusion

Abstract

The invention discloses a fan multi-mode fault diagnosis method based on voiceprint feature and vibration data fusion. The method comprises the steps that S1, voiceprint signals and vibration signals generated in the fan operation process are synchronously collected through a plurality of sensors; s2, preprocessing the collected voiceprint signal and the vibration signal; s3, fusing the preprocessed voiceprint signal and vibration signal through a feature level weighted fusion algorithm; s4, performing coarse-grained fault screening under a multi-level framework; s5, performing fine-grained fault diagnosis under a multi-level framework; s6, after the fine-grained fault diagnosis is completed, the system outputs a corresponding fault early warning signal according to the type, the specific part and the severity of the fault; and S7, the system dynamically adjusts the fault diagnosis model by monitoring the running state and the fault condition of the fan in real time. According to the method, the fault type of the fan can be accurately identified, the severity of the fan can be evaluated, and the overall intelligent level and the real-time response capability of a fault diagnosis system are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of fans, and in particular to a multi-modal fault diagnosis method for fans based on the fusion of voiceprint features and vibration data. Background Art

[0002] In the prior art, fan fault diagnosis mainly relies on data from a single mode. For example, vibration signals or soundprint signals are used to monitor the operating status of the fan. Traditional fan fault diagnosis methods are usually based on the analysis of vibration signals, and mainly judge the health status of the equipment by monitoring the vibration frequency, amplitude and vibration spectrum changes of the mechanical parts of the fan. Although traditional methods can detect fan faults to a certain extent, they rely only on data from a single mode and are limited by environmental noise, data acquisition accuracy and external interference, which makes misdiagnosis or missed diagnosis a possibility.

[0003] Another common fan fault diagnosis method is based on the monitoring of soundprint signals. It collects the acoustic signals generated by the fan during operation, analyzes the changes in frequency, amplitude, and harmonic characteristic signals to determine the equipment fault. Due to the complex operating environment of the fan and the large external noise interference, the accuracy and reliability of fault diagnosis using only soundprint signals are limited. The soundprint signal processing methods in the existing technology often cannot fully utilize the complementary information between other modal signals, resulting in insufficient sensitivity of fault detection.

[0004] Another shortcoming of the existing technology is that the early warning capability of faults is weak. Most traditional fault diagnosis systems can only sound an alarm when the equipment has obvious faults, and usually cannot effectively detect minor or early faults. This causes the fan to be discovered only after serious mechanical damage occurs, increasing maintenance costs and shortening the service life of the equipment. Most traditional fault diagnosis systems rely on pre-defined rules or models and cannot be dynamically adjusted according to the actual operating conditions of the equipment. They are difficult to adapt to complex and changeable working conditions, resulting in reduced accuracy and efficiency of fault diagnosis.

[0005] In summary, the existing technology has the following main shortcomings: first, the use of single modal data leads to low accuracy and robustness of fault diagnosis, which is easily affected by external environment and noise; second, the existing technology has insufficient warning capabilities for early or minor faults, and usually only issues an alarm when the fault is already obvious; finally, the traditional fault diagnosis system lacks the ability of adaptive adjustment, making it difficult to cope with the complex and changeable wind turbine operating environment, and unable to dynamically optimize the fault diagnosis model to improve the detection effect. Summary of the invention

[0006] One object of the present invention is to propose a multimodal fault diagnosis method for a fan based on the fusion of voiceprint features and vibration data. The present invention can accurately identify the fault type of the fan and evaluate its severity, thereby improving the overall intelligence level and real-time response capability of the fault diagnosis system.

[0007] A multi-modal fault diagnosis method for a wind turbine based on the fusion of voiceprint features and vibration data according to an embodiment of the present invention comprises the following steps:

[0008] S1. Synchronously collect the soundprint signal and vibration signal generated during the operation of the fan through multiple sensors;

[0009] S2. Preprocessing the collected voiceprint signals and vibration signals;

[0010] S3, fusing the preprocessed voiceprint signal and vibration signal through a feature-level weighted fusion algorithm to generate a multimodal feature vector, which integrates key feature information of the voiceprint signal and the vibration signal;

[0011] S4. Coarse-grained fault screening under a multi-level framework: Using the pre-established simple fault mode library, the fault type is screened through multi-modal feature vectors to preliminarily determine the potential fault conditions;

[0012] S5. Fine-grained fault diagnosis under a multi-level framework: For potential faults screened out at the coarse-grained level, a fault diagnosis model based on historical operation data and an unsupervised learning model is used to conduct an in-depth analysis of multimodal features, locate the specific part of the fan fault according to the specific fault model, and evaluate the severity of the fault;

[0013] S6. After the fine-grained fault diagnosis is completed, the system outputs the corresponding fault warning signal according to the type, specific location and severity of the fault. When a minor or early fault is detected, a warning signal is generated in time, and fault information and corresponding maintenance suggestions are provided;

[0014] S7. The system monitors the operating status and fault conditions of the fan in real time, dynamically adjusts the fault diagnosis model, and updates the fault diagnosis model parameters according to the newly collected sound print signal and vibration signal data.

[0015] Optionally, the S1 further comprises:

[0016] S11. Use multiple acoustic sensors M and vibration sensors arranged near the fan blades, bearings and other key components to synchronously collect the sound print signals and vibration signals generated during the operation of the fan. The acoustic sensors are used to capture the sound wave signals of the fan under different working conditions, and the vibration sensors are used to detect the vibration amplitude and frequency of the mechanical parts of the fan.

[0017] S12, soundprint signal S (t, x, y, z) is the acoustic signal generated by the fan blades, bearings and other key components during operation:

[0018]

[0019] Among them, A s,n is the amplitude of the nth acoustic mode, α n is the attenuation coefficient, r is the distance of sound wave propagation, f s,n is the frequency of the nth acoustic mode, φ s,n is the phase angle, θ s,n is the angle between the sound source direction and the sensor direction, P(x,y,z) is the distribution function of the sound source in space, and N is the number of modes;

[0020] S13, vibration signal V(t,ω,λ) is the vibration data of mechanical parts during the operation of the fan:

[0021]

[0022] Among them, A v,m is the amplitude of the mth vibration mode, f v,m is the frequency of the mth vibration mode, φ v,m is the phase angle, ω m is the frequency drift of the vibration mode, λ is the nonlinear index of the frequency drift, β m is the attenuation coefficient of the vibration signal, d is the distance or displacement of the mechanical structure, and M is the number of modes.

[0023] Optionally, the S2 further comprises:

[0024] S21. De-noise the collected voiceprint signal S(t,x,y,z). Use a bandpass filter to filter the ambient noise components within the frequency range and retain the effective acoustic signal components when the fan is running. The filtered signal is recorded as S d (t,x,y,z);

[0025] S22, the denoised voiceprint signal S d (t,x,y,z) performs feature extraction and extracts frequency feature F s , Amplitude characteristic A s and harmonic characteristics H s , frequency characteristic F s By analyzing the main frequency component of the signal, the amplitude characteristic A s Through signal envelope detection, the harmonic feature H s The amplitude information of each order harmonic is obtained through Fourier transform;

[0026] S23, denoising the collected vibration signal V(t,ω,λ), using wavelet transform method to decompose and reconstruct the vibration signal to remove high-frequency noise components, and obtain the denoised vibration signal V d (t,ω,λ);

[0027] S24, the vibration signal V after denoising d (t,ω,λ) to extract the acceleration time domain feature A v,td , acceleration spectrum characteristics A v,fd and velocity spectrum characteristics V v,fd , acceleration time domain characteristic A v,td is the peak acceleration of the vibration signal, acceleration spectrum feature A v,fd The velocity spectrum feature V is obtained by fast Fourier transform. v,fd is the velocity characteristic obtained by integrating the acceleration spectrum;

[0028] S25, performing signal segmentation processing on the pre-processed soundprint signal and vibration signal, and dividing the signal into multiple time windows according to the operation cycle and working status of the fan.

[0029] Optionally, the S3 further includes:

[0030] S31, the preprocessed frequency feature F s , Amplitude characteristic A s and harmonic characteristics H s And acceleration time domain characteristics A v,td , acceleration spectrum characteristics A v,fd and velocity spectrum characteristics V v,fd For input features, the weight coefficients w1, w2, w3, w4, w5, w6 of each feature are initially set. The weight coefficients represent the importance of each feature in multimodal feature fusion;

[0031] S32, iteratively adjusting the weight coefficient of each feature through a Bayesian optimization algorithm, wherein the Bayesian optimization model minimizes the error of wind turbine fault mode identification with an objective function;

[0032] S33, Bayesian optimization algorithm evaluates each set of weighted coefficients by constructing a probability model, and gradually optimizes the weighted coefficients based on the observed data. The updated weighted coefficients The objective loss function is minimized so that the weighted fused multi-modal feature vector reflects the operating status of the wind turbine under different working conditions;

[0033] S34, according to the optimized weighting coefficient Frequency characteristic F s , Amplitude characteristic A s and harmonic characteristics H sAnd acceleration time domain characteristics A v,td , acceleration spectrum characteristics A v,fd and velocity spectrum characteristics V v,fd Perform feature weighted fusion to generate the final multimodal feature vector X fusion :

[0034]

[0035] Among them, X fusion It is a weighted fused multimodal feature vector, which contains all the key feature information of the voiceprint signal and the vibration signal.

[0036] Optionally, the S32 further includes:

[0037] S321. The objective function of the weighted feature vector fusion model is defined as follows in Bayesian optimization:

[0038]

[0039] Among them, L(w i ) is the loss function of Bayesian optimization, X fusion =w i X is the multimodal feature vector through weighted fusion, w i ={w1,w2,w3,w4,w5,w6} is the weight vector, f(X fusion ,w i ) is the output prediction of the fault diagnosis model, y is the actual fault label, p(X|w i ) is the distribution density of the feature vector X, X is the feature space vector;

[0040] S322. In the Bayesian optimization process, the objective function is optimized by constructing a Gaussian process regression model g(w) to approximate the objective function and iteratively updating it by maximizing the expected improvement criterion:

[0041]

[0042] Where EI(w) is the expected improvement value, EI(w) is the multimodal feature vector at the tth iteration, and ξ is the parameter used to adjust the degree of optimization exploration;

[0043] S323, the update of the weight vector w is expressed by maximizing the expected improvement value:

[0044]

[0045] Optionally, the S4 further includes:

[0046] S41, based on multimodal feature vector X fusion,Use the pre-established simple fault mode library M to perform coarse-grained screening of common fault types. The simple fault mode library includes the fault type characteristic modes of fan bearing damage, fan blade cracks and fan imbalance;

[0047] S42. Determine the similarity between the multimodal feature vector and the simple fault mode through the similarity calculation formula, combine the voiceprint feature and the vibration feature, and define the weighted similarity Sim of each simple fault mode. w (X fusion ,M i ):

[0048]

[0049] Among them, M i ={M i,1 ,M i,2 ,,M i,n} is the characteristic vector of the i-th simple fault mode, w k is the weighting coefficient for each feature dimension, λ k is the nonlinear adjustment parameter of the voiceprint signal or vibration signal, f k is the frequency component, T is the time period;

[0050] S43, for each group of simple fault modes, screen them by similarity value and set a similarity threshold θ i , and define the fault severity coefficient γ i =:

[0051]

[0052] Among them, γ i is the severity coefficient of simple failure mode i, α is the adjustment coefficient, θ i is the threshold value associated with simple fault mode i. When it exceeds the threshold value, it is judged as a potential fault. i When it is close to 1, it indicates a higher risk of failure;

[0053] S44, the calculated γ i Classification is performed by clustering the results of coarse-grained fault screening, defining the cluster center to determine the clustering affiliation of simple fault modes, and determining the specific classification of the fault by judging the value of clustering affiliation to generate coarse-grained fault screening results.

[0054] Optionally, the S5 further includes:

[0055] S51, for the multi-modal feature vector X determined as a potential fault through coarse-grained fault screening potential, using an unsupervised learning model built based on historical operation data for in-depth analysis, the fault diagnosis model D trained by unsupervised learning method, through D(X potential ) Output the clustering results of the fault mode;

[0056] S52, using the self-organizing map network to analyze the historical operation data X history Cluster analysis is performed to generate a topological map of the fan failure mode. The self-organizing map network transforms the high-dimensional multimodal feature vector X potential Mapped to a low-dimensional grid, the grid nodes N i represents the cluster center, and each node represents a failure mode;

[0057] S53, by calculating the multimodal feature vector X potential With the self-organizing map network node N i The Euclidean distance Dist(X potential ,N i ), identify the current failure mode:

[0058] Dist(X potential ,N i )=∥X potential -N i ∥;

[0059] Among them, ∥·∥ is the Euclidean distance, which is used to measure the distance between the current fault mode and the self-organizing map network node. The node corresponding to the minimum distance is the representative of the fault type;

[0060] S54, according to the closest self-organizing map network node N i The distance Dist(X potential ,N i ) evaluates the severity of the fault, the fault severity coefficient γ severity Defined as:

[0061]

[0062] Among them, γ severity is the fault severity coefficient, and its value range is [0,1]. The closer the value is to 1, the more serious the fault is. Fault assessment and fine-grained diagnosis are performed in combination with historical data.

[0063] Optionally, the S6 further includes:

[0064] S61. When a minor or early fault is identified through fine-grained fault diagnosis, the fault severity coefficient γ severity The value of the warning threshold θ is set alert , when γ severity ≥θ alert When the system generates an early warning signal;

[0065] S62. The generated warning signal includes the fault type and fault severity. The fault type includes the following categories:

[0066] Fan bearing wear: a common type of minor fault, usually accompanied by low-frequency vibration and medium-amplitude soundprint signals;

[0067] Fan blade crack: early failure, manifested in harmonic variation of high-frequency sound pattern characteristics and slight shift in the vibration signal spectrum;

[0068] Fan imbalance: a minor fault type, manifested in low-frequency vibration characteristics and amplitude changes of periodic sound pattern signals;

[0069] Motor overheating or failure: Identified by abnormal increase of noise components in the soundprint signal and persistent deviation of vibration characteristics;

[0070] Fault severity levels include the following:

[0071] Minor glitches: γ severity ∈[0.2,0.4], preventive inspection is required but does not immediately affect the operation of the fan;

[0072] Moderate failure: γ severity ∈[0.4,0.6], timely maintenance is required to avoid further deterioration of the fault;

[0073] Serious failure: γ severity >0.6, an alarm is triggered immediately and shutdown and maintenance are arranged;

[0074] S63. Maintenance recommendations are automatically generated based on the fault type and severity, combined with historical maintenance data in the system. For minor faults, the system recommends routine inspection and monitoring. For moderate and severe faults, the system will recommend replacement of related parts and provide detailed maintenance steps, including shutdown inspection, part replacement and lubrication operations.

[0075] The beneficial effects of the present invention are:

[0076] (1) The present invention breaks through the limitations of traditional single-modal fault diagnosis methods by performing feature-level weighted fusion of voiceprint features and vibration data, and uses a Bayesian optimization algorithm to dynamically adjust the weighting coefficient of each feature, so that the feature vector after multi-modal feature fusion can more accurately reflect the operating status of the fan. In traditional fault diagnosis methods, relying solely on voiceprint or vibration data is easily affected by external interference such as environmental noise and data errors. The present invention improves the accuracy and anti-interference ability of fault diagnosis through feature fusion. Under complex working conditions, the weighting coefficient is adaptively optimized to ensure the robustness and stability of the diagnosis results under different fan operating conditions, significantly reducing the risk of misdiagnosis and missed diagnosis.

[0077] (2) The present invention effectively realizes accurate identification and timely warning of early or minor faults of the fan by introducing fault severity coefficients and warning thresholds. Compared with the traditional system that can only alarm when the fault is obvious, the present invention, based on fine-grained fault screening, uses unsupervised learning models and self-organizing mapping networks to conduct in-depth analysis of fault modes and combines historical operation data. The system can capture the most subtle abnormal signals during the operation of the fan, quantify the risk level of the fault through the severity coefficient, generate timely warning signals and corresponding maintenance suggestions, thereby greatly improving the sensitivity and accuracy of fault warnings, and avoiding major mechanical damage caused by the failure to detect early faults in time during the operation of the fan.

[0078] (3) The present invention combines historical data and real-time monitoring data to dynamically update the weights and parameters of the fault diagnosis model, so that the diagnosis model can adapt to different fan operating conditions. In traditional fault diagnosis systems, the fault modes and diagnosis models are usually static and difficult to adapt to changes in operating conditions during the long-term operation of the fan, resulting in a decrease in diagnostic accuracy. By dynamically adjusting the parameters of the diagnosis model using Bayesian optimization and self-organizing mapping networks, it is ensured that under new operating conditions and new fault modes, the system can accurately identify the fault type of the fan and evaluate its severity, thereby improving the overall intelligence level and real-time response capability of the fault diagnosis system. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0080] Figure 1 This is a flow chart of a multi-modal fault diagnosis method for a fan based on the fusion of voiceprint features and vibration data proposed by the present invention;

[0081] Figure 2 This is a structural schematic diagram of a fan fault diagnosis model for a fan multimodal fault diagnosis method based on voiceprint features and vibration data fusion proposed in the present invention. DETAILED DESCRIPTION

[0082] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0083] refer to Figure 1-2 A multi-modal fault diagnosis method for a fan based on the fusion of voiceprint features and vibration data comprises the following steps:

[0084] S1. Synchronously collect the soundprint signal and vibration signal generated during the operation of the fan through multiple sensors;

[0085] S2. Preprocessing the collected voiceprint signals and vibration signals;

[0086] S3, fusing the preprocessed voiceprint signal and vibration signal through a feature-level weighted fusion algorithm to generate a multimodal feature vector, which integrates key feature information of the voiceprint signal and the vibration signal;

[0087] S4. Coarse-grained fault screening under a multi-level framework: Using the pre-established simple fault mode library, the fault type is screened through multi-modal feature vectors to preliminarily determine the potential fault conditions;

[0088] S5. Fine-grained fault diagnosis under a multi-level framework: For potential faults screened out at the coarse-grained level, a fault diagnosis model based on historical operation data and an unsupervised learning model is used to conduct an in-depth analysis of multimodal features, locate the specific part of the fan fault according to the specific fault model, and evaluate the severity of the fault;

[0089] S6. After the fine-grained fault diagnosis is completed, the system outputs the corresponding fault warning signal according to the type, specific location and severity of the fault. When a minor or early fault is detected, a warning signal is generated in time, and fault information and corresponding maintenance suggestions are provided;

[0090] S7. The system monitors the operating status and fault conditions of the fan in real time, dynamically adjusts the fault diagnosis model, and updates the fault diagnosis model parameters according to the newly collected sound print signal and vibration signal data.

[0091] In this embodiment, S1 further includes:

[0092] S11. Use multiple acoustic sensors M and vibration sensors arranged near the fan blades, bearings and other key components to synchronously collect the sound print signals and vibration signals generated during the operation of the fan. The acoustic sensors are used to capture the sound wave signals of the fan under different working conditions, and the vibration sensors are used to detect the vibration amplitude and frequency of the mechanical parts of the fan.

[0093] S12, soundprint signal S (t, x, y, z) is the acoustic signal generated by the fan blades, bearings and other key components during operation:

[0094]

[0095] Among them, A s,n is the amplitude of the nth acoustic mode, α n is the attenuation coefficient, r is the distance of sound wave propagation, f s,nis the frequency of the nth acoustic mode, φ s,n is the phase angle, θ s,n is the angle between the sound source direction and the sensor direction, P(x,y,z) is the distribution function of the sound source in space, and N is the number of modes;

[0096] S13, vibration signal V(t,ω,λ) is the vibration data of mechanical parts during the operation of the fan:

[0097]

[0098] Among them, A v,m is the amplitude of the mth vibration mode, f v,m is the frequency of the mth vibration mode, φ v,m is the phase angle, ω m is the frequency drift of the vibration mode, λ is the nonlinear index of the frequency drift, β m is the attenuation coefficient of the vibration signal, d is the distance or displacement of the mechanical structure, and M is the number of modes.

[0099] In this embodiment, S2 further includes:

[0100] S21. De-noise the collected voiceprint signal S(t,x,y,z). Use a bandpass filter to filter the ambient noise components within the frequency range and retain the effective acoustic signal components when the fan is running. The filtered signal is recorded as S d (t,x,y,z);

[0101] S22, the denoised voiceprint signal S d (t,x,y,z) performs feature extraction and extracts frequency feature F s , Amplitude characteristic A s and harmonic characteristics H s , frequency characteristic F s By analyzing the main frequency component of the signal, the amplitude characteristic A s Through signal envelope detection, the harmonic feature H s The amplitude information of each order harmonic is obtained through Fourier transform;

[0102] S23, denoising the collected vibration signal V(t,ω,λ), using wavelet transform method to decompose and reconstruct the vibration signal to remove high-frequency noise components, and obtain the denoised vibration signal V d (t,ω,λ);

[0103] S24, the vibration signal V after denoising d (t,ω,λ) to extract the acceleration time domain feature A v,td , acceleration spectrum characteristics A v,fdand velocity spectrum characteristics V v,fd , acceleration time domain characteristic A v,td is the peak acceleration of the vibration signal, acceleration spectrum feature A v,fd The velocity spectrum feature V is obtained by fast Fourier transform. v,fd is the velocity characteristic obtained by integrating the acceleration spectrum;

[0104] S25, performing signal segmentation processing on the pre-processed soundprint signal and vibration signal, and dividing the signal into multiple time windows according to the operation cycle and working status of the fan.

[0105] In this implementation, S3 further includes:

[0106] S31, the preprocessed frequency feature F s , Amplitude characteristic A s and harmonic characteristics H s And acceleration time domain characteristics A v,td , acceleration spectrum characteristics A v,fd and velocity spectrum characteristics V v,fd For input features, the weight coefficients w1, w2, w3, w4, w5, w6 of each feature are initially set. The weight coefficients represent the importance of each feature in multimodal feature fusion;

[0107] S32, iteratively adjusting the weight coefficient of each feature through a Bayesian optimization algorithm, wherein the Bayesian optimization model minimizes the error of wind turbine fault mode identification with an objective function;

[0108] S33, Bayesian optimization algorithm evaluates each set of weighted coefficients by constructing a probability model, and gradually optimizes the weighted coefficients based on the observed data. The updated weighted coefficients The objective loss function is minimized so that the weighted fused multi-modal feature vector reflects the operating status of the wind turbine under different working conditions;

[0109] S34, according to the optimized weighting coefficient Frequency characteristic F s , Amplitude characteristic A s and harmonic characteristics H s And acceleration time domain characteristics A v,td , acceleration spectrum characteristics A v,fd and velocity spectrum characteristics V v,fd Perform feature weighted fusion to generate the final multimodal feature vector X fusion :

[0110]

[0111] Among them, X fusionIt is a weighted fused multimodal feature vector, which contains all the key feature information of the voiceprint signal and the vibration signal.

[0112] In this implementation, S32 further includes:

[0113] S321. The objective function of the weighted feature vector fusion model is defined as follows in Bayesian optimization:

[0114]

[0115] Among them, L(w i ) is the loss function of Bayesian optimization, X fusion =w i X is the multimodal feature vector through weighted fusion, w i ={w1,w2,w3,w4,w5,w6} is the weight vector, f(X fusion ,w i ) is the output prediction of the fault diagnosis model, y is the actual fault label, p(X|w i ) is the distribution density of the feature vector X, X is the feature space vector;

[0116] S322. In the Bayesian optimization process, the objective function is optimized by constructing a Gaussian process regression model g(w) to approximate the objective function and iteratively updating it by maximizing the expected improvement criterion:

[0117]

[0118] Where EI(w) is the expected improvement value, EI(w) is the multimodal feature vector at the tth iteration, and ξ is the parameter used to adjust the degree of optimization exploration;

[0119] S323, the update of the weight vector w is expressed by maximizing the expected improvement value:

[0120]

[0121] In this implementation, S4 further includes:

[0122] S41, based on multimodal feature vector X fusion ,Use the pre-established simple fault mode library M to perform coarse-grained screening of common fault types. The simple fault mode library includes the fault type characteristic modes of fan bearing damage, fan blade cracks and fan imbalance;

[0123] S42. Determine the similarity between the multimodal feature vector and the simple fault mode through the similarity calculation formula, combine the voiceprint feature and the vibration feature, and define the weighted similarity Sim of each simple fault mode. w (X fusion ,Mi ):

[0124]

[0125] Among them, M i ={M i,1 ,M i,2 ,,M i,n} is the characteristic vector of the i-th simple fault mode, w k is the weighting coefficient for each feature dimension, λ k is the nonlinear adjustment parameter of the voiceprint signal or vibration signal, f k is the frequency component, T is the time period;

[0126] S43, for each group of simple fault modes, screen them by similarity value and set a similarity threshold θ i , and define the fault severity coefficient γ i =:

[0127]

[0128] Among them, γ i is the severity coefficient of simple failure mode i, α is the adjustment coefficient, θ i is the threshold value associated with simple fault mode i. When it exceeds the threshold value, it is judged as a potential fault. i When it is close to 1, it indicates a higher risk of failure;

[0129] S44, the calculated γ i Classification is performed by clustering the results of coarse-grained fault screening, defining the cluster center to determine the clustering affiliation of simple fault modes, and determining the specific classification of the fault by judging the value of clustering affiliation to generate coarse-grained fault screening results.

[0130] In this implementation, S5 further includes:

[0131] S51, for the multi-modal feature vector X determined as a potential fault through coarse-grained fault screening potential , using an unsupervised learning model built based on historical operation data for in-depth analysis, the fault diagnosis model D trained by unsupervised learning method, through D(X potential ) Output the clustering result of the fault mode;

[0132] S52, using the self-organizing map network to analyze the historical operation data X history Cluster analysis is performed to generate a topological map of the fan failure mode. The self-organizing map network transforms the high-dimensional multimodal feature vector X potential Mapped to a low-dimensional grid, the grid nodes N i represents the cluster center, and each node represents a failure mode;

[0133] S53, by calculating the multimodal feature vector X potential With the self-organizing map network node N i The Euclidean distance Dist(X potential ,N i ), identify the current failure mode:

[0134] Dist(X potential ,N i )=∥X potential -N i ∥;

[0135] Among them, ∥·∥ is the Euclidean distance, which is used to measure the distance between the current fault mode and the self-organizing map network node. The node corresponding to the minimum distance is the representative of the fault type;

[0136] S54, according to the closest self-organizing map network node N i The distance Dist(X potential ,N i ) evaluates the severity of the fault, the fault severity coefficient γ severity Defined as:

[0137]

[0138] Among them, γ severity is the fault severity coefficient, and its value range is [0,1]. The closer the value is to 1, the more serious the fault is. Fault assessment and fine-grained diagnosis are performed in combination with historical data.

[0139] In this implementation, S6 further includes:

[0140] S61. When a minor or early fault is identified through fine-grained fault diagnosis, the fault severity coefficient γ severity The value of the warning threshold θ is set alert , when γ severity ≥θ alert When the system generates an early warning signal;

[0141] S62. The generated warning signal includes the fault type and fault severity. The fault type includes the following categories:

[0142] Fan bearing wear: a common type of minor fault, usually accompanied by low-frequency vibration and medium-amplitude soundprint signals;

[0143] Fan blade crack: early failure, manifested in harmonic variation of high-frequency sound pattern characteristics and slight shift in the vibration signal spectrum;

[0144] Fan imbalance: a minor fault type, manifested in low-frequency vibration characteristics and amplitude changes of periodic sound pattern signals;

[0145] Motor overheating or failure: Identified by abnormal increase of noise components in the soundprint signal and persistent deviation of vibration characteristics;

[0146] Fault severity levels include the following:

[0147] Minor glitches: γ severity ∈[0.2,0.4], preventive inspection is required but does not immediately affect the operation of the fan;

[0148] Moderate failure: γ severity ∈[0.4,0.6], timely maintenance is required to avoid further deterioration of the fault;

[0149] Serious failure: γ severity >0.6, an alarm is triggered immediately and shutdown and maintenance are arranged;

[0150] S63. Maintenance recommendations are automatically generated based on the fault type and severity, combined with historical maintenance data in the system. For minor faults, the system recommends routine inspection and monitoring. For moderate and severe faults, the system will recommend replacement of related parts and provide detailed maintenance steps, including shutdown inspection, part replacement and lubrication operations.

[0151] Embodiment 1:

[0152] In August 2023, the operation and maintenance team of a coastal wind farm discovered that the failure rate of blades and bearing components of regional wind turbines was relatively high due to long-term exposure to high humidity and high salt environments. In order to improve the stability of the equipment and extend the service life of the wind turbines, the team introduced a multimodal fault diagnosis system based on the fusion of voiceprint features and vibration data.

[0153] On August 12, during the operation of wind turbine No. 18 of the wind farm, with blade number "Blade-18-A", the system monitored the operating status of the wind turbine in real time. The soundprint sensor and vibration sensor of the wind turbine were installed at the root of the blade, at the bearing and near the motor respectively. The data acquisition system collected data every 5 minutes and input it into the multimodal data fusion algorithm for real-time processing.

[0154] At 10:21 am on August 15, the system detected an abnormality in the soundprint signal of fan No. 18. The frequency characteristics suddenly increased, the amplitude fluctuated significantly, and the vibration sensor showed signs of frequency shift in the acceleration spectrum, especially the sudden increase in the vibration signal in the low frequency band (0-200Hz). After fusion algorithm analysis, the system generated a fault severity coefficient of 0.42, which exceeded the preset warning threshold of 0.4. At this time, the system judged that there might be cracks in the blades and generated a warning signal.

[0155] Data analysis and maintenance decision-making:

[0156] Within 5 minutes of receiving the warning signal, the system will push detailed fault information to the on-site operation and maintenance personnel via SMS and email. The fault report includes the following information:

[0157] Failure time: 10:21, August 15, 2023;

[0158] Fan number: Fan No. 18;

[0159] Blade number: Blade-18-A;

[0160] Fault type: Blade crack;

[0161] Fault severity: 0.42 (moderate fault);

[0162] Vibration signal characteristics: acceleration spectrum shift, low-frequency vibration enhancement;

[0163] Voiceprint signal characteristics: amplitude fluctuation, frequency characteristics increase;

[0164] According to the maintenance suggestions provided by the system, the operation and maintenance team arranged a routine inspection in the afternoon of the same day. At 2:30 p.m., the operation and maintenance personnel found a slight crack at the root of the blade of wind turbine No. 18 during the inspection. The crack was 8 cm long, but had not yet extended to the critical structure. The team used special materials to repair the crack according to the suggestions, avoiding subsequent more serious failures. The entire inspection and repair process lasted about 3 hours and the wind turbine did not stop running.

[0165] The adjacent fan No. 19 was not equipped with a multi-modal fault diagnosis system, and the traditional vibration signal monitoring method was used. Fan No. 19 was running in the same time period, but because the blade cracks were not detected in time, the fan continued to run until early September, when a sudden shutdown failure occurred. After investigation, it was found that the fan blades had serious cracks, with a length of 35 cm, causing the fan to shut down and replace the blades. The entire repair process took 48 hours and the maintenance cost was as high as RMB 30,000.

[0166] Table 1 Comparison data of fan No. 18 (using multi-modal fault diagnosis system) and fan No. 19 (using traditional fault diagnosis method)

[0167]

[0168] On September 5, 2023, the operation and maintenance team rechecked wind turbine No. 18. After the blades were repaired, they operated normally without any new fault signals. Through the diagnosis and comparison of this simulated fault, the team found that the early warning mechanism of the multimodal fault diagnosis system can effectively avoid major damage to key components of the wind turbine. During the one-month continuous operation monitoring, the system generated a total of 3 early fault warnings, corresponding to mild bearing wear and blade cracks, and maintenance measures were taken before the faults became serious.

[0169] The testing and application of the wind turbine fault diagnosis system have proved the effectiveness of the present invention. The multimodal data fusion technology has significantly improved the accuracy and sensitivity of wind turbine fault diagnosis. Compared with traditional vibration monitoring technology, the system can detect potential faults in wind turbine operation at an earlier stage and generate maintenance suggestions, which greatly reduces the downtime and maintenance costs of wind turbines caused by faults. In addition, through the system, the operation and maintenance cycle of the wind turbine can be optimized, reducing the frequency of unnecessary inspections and repairs, thereby improving power generation efficiency.

[0170] The present invention breaks through the limitations of traditional single-modal fault diagnosis methods by performing feature-level weighted fusion of voiceprint features and vibration data, and uses a Bayesian optimization algorithm to dynamically adjust the weighted coefficient of each feature, so that the feature vector after multi-modal feature fusion can more accurately reflect the operating status of the fan. In traditional fault diagnosis methods, only relying on voiceprint or vibration data is easily affected by external interference such as environmental noise and data errors. The present invention improves the accuracy and anti-interference ability of fault diagnosis through feature fusion, and through adaptive optimization of weighted coefficients under complex working conditions, ensures the robustness and stability of the diagnosis results under different fan operating conditions, and significantly reduces the risk of misdiagnosis and missed diagnosis.

[0171] The present invention effectively realizes accurate identification and timely warning of early or minor faults of fans by introducing fault severity coefficients and warning thresholds. Compared with traditional systems that can only alarm when faults are obvious, the present invention, based on fine-grained fault screening, uses unsupervised learning models and self-organizing mapping networks to conduct in-depth analysis of fault modes combined with historical operation data. The system can capture the most subtle abnormal signals during fan operation, quantify the risk level of the fault through the severity coefficient, and generate timely warning signals and corresponding maintenance suggestions, thereby greatly improving the sensitivity and accuracy of fault warnings and avoiding major mechanical damage caused by failure to detect early faults in time during fan operation.

[0172] The present invention combines historical data and real-time monitoring data to dynamically update the weights and parameters of the fault diagnosis model, so that the diagnosis model can adapt to different fan operating conditions. The fault modes and diagnosis models in traditional fault diagnosis systems are usually static, and it is difficult to adapt to changes in operating conditions during the long-term operation of the fan, resulting in a decrease in diagnostic accuracy. By dynamically adjusting the parameters of the diagnosis model using Bayesian optimization and self-organizing mapping networks, it is ensured that under new operating conditions and new fault modes, the system can accurately identify the fault type of the fan and evaluate its severity, thereby improving the overall intelligence level and real-time response capability of the fault diagnosis system.

[0173] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A multi-modal fault diagnosis method for a fan based on the fusion of voiceprint features and vibration data, characterized in that: The steps include: S1. Synchronously collect the soundprint signal and vibration signal generated during the operation of the fan through multiple sensors; S2. Preprocessing the collected voiceprint signals and vibration signals; S3, fusing the preprocessed voiceprint signal and vibration signal through a feature-level weighted fusion algorithm to generate a multimodal feature vector, which integrates key feature information of the voiceprint signal and the vibration signal; S4. Coarse-grained fault screening under a multi-level framework: Using the pre-established simple fault mode library, the fault type is screened through multi-modal feature vectors to preliminarily determine the potential fault conditions; S5. Fine-grained fault diagnosis under a multi-level framework: For potential faults screened out at the coarse-grained level, a fault diagnosis model based on historical operation data and an unsupervised learning model is used to conduct an in-depth analysis of multimodal features, locate the specific part of the fan fault according to the specific fault model, and evaluate the severity of the fault; S6. After the fine-grained fault diagnosis is completed, the system outputs the corresponding fault warning signal according to the type, specific location and severity of the fault. When a minor or early fault is detected, a warning signal is generated in time, and fault information and corresponding maintenance suggestions are provided; S7. The system monitors the operating status and fault conditions of the fan in real time, dynamically adjusts the fault diagnosis model, and updates the fault diagnosis model parameters according to the newly collected sound print signal and vibration signal data.

2. According to claim 1, a multi-modal fault diagnosis method for a wind turbine based on the fusion of voiceprint features and vibration data is characterized in that: The S1 further comprises: S11. Use multiple acoustic sensors M and vibration sensors arranged near the fan blades, bearings and other key components to synchronously collect the sound print signals and vibration signals generated during the operation of the fan. The acoustic sensors are used to capture the sound wave signals of the fan under different working conditions, and the vibration sensors are used to detect the vibration amplitude and frequency of the mechanical parts of the fan. S12, soundprint signal S (t, x, y, z) is the acoustic signal generated by the fan blades, bearings and other key components during operation: Among them, A s,n is the amplitude of the nth acoustic mode, α n is the attenuation coefficient, r is the distance of sound wave propagation, f s,n is the frequency of the nth acoustic mode, φ s,n is the phase angle, θ s,n is the angle between the sound source direction and the sensor direction, P(x,y,z) is the distribution function of the sound source in space, and N is the number of modes; S13, vibration signal V(t,ω,λ) is the vibration data of mechanical parts during the operation of the fan: Among them, A v,m is the amplitude of the mth vibration mode, f v,m is the frequency of the mth vibration mode, φ v,m is the phase angle, ω m is the frequency drift of the vibration mode, λ is the nonlinear index of the frequency drift, β m is the attenuation coefficient of the vibration signal, d is the distance or displacement of the mechanical structure, and M is the number of modes.

3. The method for multi-modal fault diagnosis of a wind turbine based on the fusion of voiceprint features and vibration data according to claim 1 is characterized in that: The S2 further comprises: S21. De-noise the collected voiceprint signal S(t,x,y,z). Use a bandpass filter to filter the ambient noise components within the frequency range and retain the effective acoustic signal components when the fan is running. The filtered signal is recorded as S d (t,x,y,z); S22, the denoised voiceprint signal S d (t,x,y,z) performs feature extraction and extracts frequency feature F s , Amplitude characteristic A s and harmonic characteristics H s , frequency characteristic F s By analyzing the main frequency component of the signal, the amplitude characteristic A s Through signal envelope detection, the harmonic feature H s The amplitude information of each order harmonic is obtained through Fourier transform; S23, denoising the collected vibration signal V(t,ω,λ), using wavelet transform method to decompose and reconstruct the vibration signal to remove high-frequency noise components, and obtain the denoised vibration signal V d (t,ω,λ); S24, the vibration signal V after denoising d (t,ω,λ) to extract the acceleration time domain feature A v,td , acceleration spectrum characteristics A v,fd and velocity spectrum characteristics V v,fd , acceleration time domain characteristic A v,td is the peak acceleration of the vibration signal, acceleration spectrum feature A v,fd The velocity spectrum feature V is obtained by fast Fourier transform. v,fd is the velocity characteristic obtained by integrating the acceleration spectrum; S25, performing signal segmentation processing on the pre-processed soundprint signal and vibration signal, and dividing the signal into multiple time windows according to the operation cycle and working status of the fan.

4. The method for multi-modal fault diagnosis of a wind turbine based on the fusion of voiceprint features and vibration data according to claim 1 is characterized in that: The S3 further comprises: S31, the preprocessed frequency feature F s , Amplitude characteristic A s and harmonic characteristics H s And acceleration time domain characteristics A v,td , acceleration spectrum characteristics A v,fd and velocity spectrum characteristics V v,fd For input features, the weight coefficients w1, w2, w3, w4, w5, w6 of each feature are initially set. The weight coefficients represent the importance of each feature in multimodal feature fusion; S32, iteratively adjusting the weight coefficient of each feature through a Bayesian optimization algorithm, wherein the Bayesian optimization model minimizes the error of wind turbine fault mode identification with an objective function; S33, Bayesian optimization algorithm evaluates each set of weighted coefficients by constructing a probability model, and gradually optimizes the weighted coefficients based on the observed data. The updated weighted coefficients The objective loss function is minimized so that the weighted fused multi-modal feature vector reflects the operating status of the wind turbine under different working conditions; S34, according to the optimized weighting coefficient Frequency characteristic F s , Amplitude characteristic A s and harmonic characteristics H s And acceleration time domain characteristics A v,td , acceleration spectrum characteristics A v,fd and velocity spectrum characteristics V v,fd Perform feature weighted fusion to generate the final multimodal feature vector X fusion : Among them, X fusion It is a weighted fused multimodal feature vector, which contains all the key feature information of the voiceprint signal and the vibration signal.

5. The method for multi-modal fault diagnosis of a wind turbine based on the fusion of voiceprint features and vibration data according to claim 1 is characterized in that: The S32 further comprises: S321. The objective function of the weighted feature vector fusion model is defined as follows in Bayesian optimization: Among them, L(w i ) is the loss function of Bayesian optimization, X fusion =w i X is the multimodal feature vector through weighted fusion, w i ={w1,w2,w3,w4,w5,w6} is the weight vector, f(X fusion ,w i ) is the output prediction of the fault diagnosis model, y is the actual fault label, p(X|w i ) is the distribution density of the feature vector X, X is the feature space vector; S322. In the Bayesian optimization process, the objective function is optimized by constructing a Gaussian process regression model g(w) to approximate the objective function and iteratively updating it by maximizing the expected improvement criterion: Where EI(w) is the expected improvement value, EI(w) is the multimodal feature vector at the tth iteration, and ξ is the parameter used to adjust the degree of optimization exploration; S323, the update of the weight vector w is expressed by maximizing the expected improvement value:

6. The method for multi-modal fault diagnosis of a wind turbine based on the fusion of voiceprint features and vibration data according to claim 1 is characterized in that: The S4 further comprises: S41, based on multimodal feature vector X fusion ,Use the pre-established simple fault mode library M to perform coarse-grained screening of common fault types. The simple fault mode library includes the fault type characteristic modes of fan bearing damage, fan blade cracks and fan imbalance; S42. Determine the similarity between the multimodal feature vector and the simple fault mode through the similarity calculation formula, combine the voiceprint feature and the vibration feature, and define the weighted similarity Sim of each simple fault mode. w (X fusion ,M i ): Among them, M i ={M i,1 ,M i,2 ,,M i,n } is the characteristic vector of the i-th simple fault mode, w k is the weighting coefficient for each feature dimension, λ k is the nonlinear adjustment parameter of the voiceprint signal or vibration signal, f k is the frequency component, T is the time period; S43, for each group of simple fault modes, screen them by similarity value and set a similarity threshold θ i , and define the fault severity coefficient γ i =: Among them, γ i is the severity coefficient of simple failure mode i, α is the adjustment coefficient, θ i is the threshold value associated with simple fault mode i. When it exceeds the threshold value, it is judged as a potential fault. i When it is close to 1, it indicates a higher risk of failure; S44, the calculated γ i Classification is performed by clustering the results of coarse-grained fault screening, defining the cluster center to determine the cluster affiliation of simple fault modes, and determining the specific classification of the fault by judging the value of cluster affiliation to generate coarse-grained fault screening results.

7. The method for multi-modal fault diagnosis of a wind turbine based on the fusion of voiceprint features and vibration data according to claim 1 is characterized in that: The S5 further comprises: S51, for the multi-modal feature vector X determined as a potential fault through coarse-grained fault screening potential , using an unsupervised learning model built based on historical operation data for in-depth analysis, the fault diagnosis model D trained by unsupervised learning method, through D(X potential ) Output the clustering results of the fault mode; S52, using the self-organizing map network to analyze the historical operation data X history Cluster analysis is performed to generate a topological map of the fan failure mode. The self-organizing map network transforms the high-dimensional multimodal feature vector X potential Mapped to a low-dimensional grid, the grid nodes N i represents the cluster center, and each node represents a failure mode; S53, by calculating the multimodal feature vector X potential With the self-organizing map network node N i The Euclidean distance Dist(X potential ,N i ), identify the current failure mode: Dist(X potential ,N i )=∥X potential -N i ∥; Among them, ∥(∥ is the Euclidean distance, which is used to measure the distance between the current fault mode and the self-organizing map network node. The node corresponding to the minimum distance is the representative of the fault type; S54, according to the closest self-organizing map network node N i The distance Dist(X potential ,N i ) evaluates the severity of the fault, the fault severity coefficient γ severity Defined as: Among them, γ severity is the fault severity coefficient, and its value range is [0,1]. The closer the value is to 1, the more serious the fault is. Fault assessment and fine-grained diagnosis are performed in combination with historical data.

8. The method for multi-modal fault diagnosis of a wind turbine based on the fusion of voiceprint features and vibration data according to claim 1 is characterized in that: The S6 further comprises: S61. When a minor or early fault is identified through fine-grained fault diagnosis, the fault severity coefficient γ severity The value of the warning threshold θ is set alert , when γ severity ≥θ alert When the system generates an early warning signal; S62. The generated warning signal includes the fault type and fault severity. The fault type includes the following categories: Fan bearing wear: a common type of minor fault, usually accompanied by low-frequency vibration and medium-amplitude soundprint signals; Fan blade crack: early failure, manifested in harmonic variation of high-frequency sound pattern characteristics and slight shift in the vibration signal spectrum; Fan imbalance: a minor fault type, manifested in low-frequency vibration characteristics and amplitude changes of periodic sound pattern signals; Motor overheating or failure: Identified by abnormal increase of noise components in the soundprint signal and persistent deviation of vibration characteristics; Fault severity levels include the following: Minor glitches: γ severity ∈[0.2,0.4], preventive inspection is required but does not immediately affect the operation of the fan; Moderate failure: γ severity ∈[0.4,0.6], timely maintenance is required to avoid further deterioration of the fault; Serious failure: γ severity >0.6, an alarm is triggered immediately and shutdown and maintenance are arranged; S63. Maintenance recommendations are automatically generated based on the fault type and severity, combined with historical maintenance data in the system. For minor faults, the system recommends routine inspection and monitoring. For moderate and severe faults, the system will recommend replacement of related parts and provide detailed maintenance steps, including shutdown inspection, part replacement and lubrication operations.

Images