Fan blade fault diagnosis method based on voiceprint recognition

Abstract

The invention discloses a fan blade fault diagnosis method based on voiceprint recognition, and the method comprises the steps: collecting a voiceprint signal of a fan blade through an acoustic sensor, carrying out the marking of the voiceprint signal, carrying out the preprocessing of the voiceprint signal, and carrying out the processing of the preprocessed voiceprint signal, and obtaining a Mel time-frequency spectrogram; constructing a hybrid model used for identifying fan blade faults, and training and testing the hybrid model; and finally, fan blade fault detection is carried out by using the hybrid model which is tested to be qualified. The method has the advantages of real-time monitoring, high accuracy and the like, and provides powerful guarantee for stable operation of the wind turbine generator.

Description

Technical Field

[0001] The present invention relates to the technical field of wind power equipment condition monitoring, and particularly relates to a fault diagnosis method for wind turbine blades based on the combination of online voiceprint recognition and deep learning. Background Technique

[0002] As a key part of clean energy, wind power generation plays an important role in the transformation of the energy structure and the development of renewable energy, and is of great significance for alleviating energy shortages, promoting the green transformation of energy, and ensuring energy security. The wind turbine blade, as a key component for capturing wind energy, is vulnerable to erosion, lightning strikes, and fatigue cracks in a complex environment, resulting in performance degradation and failures, which affect the efficiency and safety of wind farms. Therefore, real-time monitoring of the health status of wind turbine blades and accurate fault diagnosis are crucial for improving the operation and maintenance efficiency of wind power, reducing costs, and ensuring safe and stable operation.

[0003] With the development of the wind power industry and intelligent technologies, significant progress has been made in wind turbine blade fault diagnosis technology, presenting diverse methods. Widely used technologies include computer vision, acoustic emission, infrared thermography, vibration analysis, and ultrasonic technology. Machine vision relies on image sequence information and has high requirements for imaging and data processing. Especially in a complex background, the computational load is large and the accuracy depends on image processing. Therefore, the development of diagnostic technologies based on voiceprint signals is of great significance for improving the accuracy and efficiency of wind turbine blade fault diagnosis.

[0004] Currently, there have been a large number of studies on fault detection based on the voiceprint signals of wind turbine blades at home and abroad, and a variety of effective blade fault detection methods have been proposed.

[0005] In the article titled "Research on a Highly Generalizable Abnormal Detection Method for Wind Turbine Blades Based on Voiceprint", the authors Zou Yijin et al. proposed a periodic audio cutting method based on clustering and median convergence to effectively cut the voiceprint and reduce the computational load. For fault detection, the steady-state difference method between the three blades of the wind turbine generator set was used to detect abnormalities, bypassing the problems of algorithm migration failure caused by changes in the object to be detected, channels, etc. However, it only detects and prompts the abnormal detection of wind turbine blades, does not classify specific abnormalities, nor does it identify the appropriate operating state of the wind turbine, and fails to identify the problem during the operation and control of the wind turbine, such as pitch control, yaw control, shutdown process, and stop process.

[0006] Wang Zongyao et al. proposed a method in the patent application with the patent number 202411484598.6 and the title "A Fault Diagnosis Method for Wind Turbine Blades Based on Voiceprint Signals". This method uses a sound acquisition device to obtain voiceprint signals and removes noise through adaptive filtering. Then, the complete ensemble empirical mode decomposition is used to decompose the signal into intrinsic mode functions, which are then converted into two-dimensional mirror snowflake diagrams to achieve the mapping from one-dimensional signals to two-dimensional images. Subsequently, the images are normalized, and the penalty parameter and kernel parameter of the support vector machine (SVM) are optimized using the improved red-billed blue magpie optimization algorithm (IRBMO) to construct the IRBMO-SVM model. Finally, the two-dimensional snowflake diagram is input into the model for fault classification. Although the improved IRBMO algorithm introduces a chaotic mapping and a reverse learning mechanism, its global search ability and convergence speed still need to be further verified, especially in high-dimensional parameter spaces where local optimal problems may exist.

[0007] In the article titled "A Method for Abnormal Recognition of Wind Turbine Blades Based on Voiceprint Feature Fusion", the authors Yu Hongwu et al. proposed a method that combines complementary ensemble empirical mode decomposition with the voiceprint features of wind turbine blades. First, voiceprint data under normal and four abnormal working conditions are collected, and the signal quality is optimized through operations such as noise reduction using a Butterworth band-pass filter, framing, and adding a Hamming window. The complementary ensemble empirical mode decomposition algorithm is used to perform secondary noise reduction on the signal. After decomposition, effective mode components (IMFs) are selected through Pearson correlation coefficients, and Mel frequency cepstral coefficients, linear prediction cepstral coefficients, gammatone cepstral coefficients, short-time energy, and short-time average zero-crossing rate are extracted from each layer of IMF to form a 41-dimensional feature vector and fuse them. High fault recognition accuracy is achieved in the verification experiment, and the effect on early anomaly detection is significant. However, there is a lack of necessary quantitative analysis (such as principal component analysis) to optimize the feature dimension. Summary of the Invention

[0008] In view of this, the purpose of the present invention is to provide a fault diagnosis method for wind turbine blades based on voiceprint recognition to solve the technical problems of fault diagnosis and fault location of wind turbine blades.

[0009] The fault diagnosis method for wind turbine blades based on voiceprint recognition of the present invention includes the following steps:

[0010] 1) Collect the voiceprint signals of the wind turbine blades through acoustic sensors arranged at the roots of the wind turbine blades to obtain the voiceprint signals of normal blades and the voiceprint signals of faulty blades, and mark the types of faults and the distances from the faults to the blade roots for the voiceprint signals of the faulty blades;

[0011] 2) Preprocess the marked voiceprint signals in step 1), and the preprocessing includes denoising the voiceprint signals, framing the denoised signals, and windowing the framed signals;

[0012] 3) Perform a discrete Fourier transform on the preprocessed voiceprint signal to convert the signal from the time domain to the time-frequency domain, obtaining a time-frequency spectrum matrix;

[0013] And use a Mel filter bank to extract frequency-domain features from the preprocessed voiceprint signal, obtaining a Mel filter bank matrix;

[0014] Multiply the time-frequency spectrum matrix and the Mel filter bank matrix and then take the logarithm to obtain the Mel time-frequency spectrum diagram of the voiceprint signal;

[0015] 4) Use the Mixup method to perform data augmentation on the Mel time-frequency spectrum diagram containing the faulty voiceprint features obtained in step 3);

[0016] 5) Construct a hybrid model for identifying fan blade faults. The hybrid model is composed of an input layer, a convolutional neural network, a long short-term memory network, an attention layer, and an output layer connected. And train the hybrid model in the following way: Use the Mel time-frequency spectrum diagram obtained in step 3) and the enhanced data obtained in step 4) to form the training set and test set of the hybrid model, train and test the hybrid model, and obtain a qualified hybrid model;

[0017] 6) Process the voiceprint signal of the fan blade collected in real time by the acoustic sensor as described in step 2) and step 3), and then input the obtained Mel time-frequency spectrum diagram into the qualified hybrid model obtained in step 5), and the hybrid model obtains the fault detection result.

[0018] Further, in step 2), a Butterworth band-pass filter is used to filter and denoise the voiceprint signal. The formula of the filter is as follows:

[0019]

[0020] Where: H 2 (p) is the squared amplitude-frequency, ω c is the cut-off frequency, ω u is the upper cut-off frequency, ω l is the lower cut-off frequency, N is the order of the filter, and s is the complex frequency variable;

[0021] Further, the formula for frame segmentation of the denoised signal in step 2) is as follows:

[0022]

[0023] Where: M is the number of frames, N is the signal length, L is the frame length, and W is the overlap rate.

[0024] Further, in step 2), perform a Hamming window operation on the framed signal. The Hamming window formula is:

[0025]

[0026] Among them, A is the Hamming window length.

[0027] Furthermore, the transfer function of the Mel filter bank described in step 3) is as shown in formula (5):

[0028]

[0029] Where: H m (f) is the filter parameter, m represents the number of filter banks, x(m) represents the center frequency of the triangular filter, and the calculation method is as shown in formula (6)

[0030]

[0031] Where: f max and f min are respectively the maximum frequency value and the minimum frequency value of the filtering range; f s is the sampling frequency of the acoustic sensor, C is the frame length of the discrete Fourier transform; the calculation formula of the Mel value is as follows:

[0032]

[0033] Where, f is the frequency under the conventional scale, 0 ≤ f ≤ 8000; k is the frequency under the Mel scale.

[0034] Furthermore, in step 4), the data augmentation using the Mixup method is to fuse two Mel time-frequency diagrams into a new time-frequency diagram through the following formula:

[0035]

[0036] Where, x i and x j represent the feature vectors of any two Mel time-frequency diagram samples i and j respectively, y i and y j represent the label values of any two Mel time-frequency diagram samples i and j respectively; represents the feature vector of the extended time-frequency diagram, represents the label value of the extended time-frequency diagram, and λ represents the fusion coefficient.

[0037] Furthermore, in step 5), the processing process of the hybrid model for the input time-frequency diagram data is as follows:

[0038] a) The time-frequency diagram data sample enters the hybrid model through the input layer;

[0039] b) The convolutional neural network extracts features from the samples input by the input layer and inputs the extracted features into the long short-term memory network;

[0040] c) The long short-term memory network extracts features from the output of the convolutional neural network, and the long short-term memory network inputs the extracted feature vectors into the attention layer;

[0041] d) The attention layer assigns different parameters to different features to create an ideal weight parameter matrix, and the output of the attention layer enters the output layer;

[0042] e) The output layer outputs the fan blade status data.

[0043] Advantages of the present invention:

[0044] 1. The method for diagnosing faults of fan blades based on voiceprint recognition according to the present invention marks the information collected by the sound sensor for different fault types and fault positions, laying a foundation for subsequent training of the hybrid model to identify the fault type features and fault position features contained in the voiceprint data.

[0045] 2. The method for diagnosing faults of fan blades based on voiceprint recognition according to the present invention proposes a data processing method combining Mel time-frequency spectrum and Mixup data augmentation. The Mel filter bank is used to reduce the dimension of the voiceprint data, significantly reducing the sample size while retaining the voiceprint features; and by using Mixup data augmentation for the fault data, the fault data is extended; thus, it lays a foundation for subsequent training to obtain a hybrid model with high detection accuracy and high generalization ability.

[0046] 3. The method for diagnosing faults of fan blades based on voiceprint recognition according to the present invention proposes a hybrid model that combines the feature extraction ability of the convolutional neural network and the time series analysis ability of the long short-term memory network, and can effectively capture local and global features in the voiceprint signal, with high accuracy in fault diagnosis.

[0047] 4. The method for diagnosing faults of fan blades based on voiceprint recognition according to the present invention proposes a hybrid model that also combines an attention mechanism on the basis of the convolutional neural network-long short-term memory network, making the hybrid model have better robustness to environmental noise and interference; and by introducing the attention mechanism, the model can focus on the features most critical for fault diagnosis, further improving the accuracy of fault recognition. At the same time, introducing the attention mechanism can optimize the network structure, reduce the number of model parameters and the amount of calculation, and improve the operation efficiency of the model.

[0048] 5. The method for diagnosing faults of fan blades based on voiceprint recognition according to the present invention can identify the voiceprint features of multiple fault types by the trained hybrid model and give the position information of the fault voiceprint features, with strong applicability. Description of the Drawings

[0049] Figure 1 is the Mel filter bank.

[0050] Figure 2 It is the LSTM cell structure.

[0051] Figure 3 It is the attention mechanism structure.

[0052] Figure 4 It is the flowchart of the fault diagnosis method for wind turbine blades. Specific implementation manner

[0053] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0054] The fault diagnosis method for wind turbine blades based on voiceprint recognition in this embodiment includes the following steps:

[0055] 1) Collect the voiceprint signals of the wind turbine blades through the acoustic sensors arranged at the roots of the wind turbine blades to obtain the voiceprint signals of normal blades and the voiceprint signals of faulty blades, and mark the types of faults and the distances from the faults to the blade roots for the voiceprint signals of the faulty blades.

[0056] 2) Preprocess the marked voiceprint signals in step 1), and the preprocessing includes denoising the voiceprint signals, framing the denoised signals, and windowing the framed signals.

[0057] During the operation of wind turbine blades in the wild, they are not only affected by harsh environments such as rain, snow, sand, and strong winds, but also affected by the complex input forces of wind turbines, resulting in complex background noise in the collected voiceprint signals of wind turbine blades. Therefore, it is necessary to perform corresponding preprocessing on the collected voiceprint signals.

[0058] In this embodiment, a Butterworth band-pass filter is used to filter and denoise the voiceprint signals, and the formula of the filter is as follows:

[0059]

[0060]

[0061] Where: H 2 (p) is the squared amplitude-frequency, ω c is the cut-off frequency, ω u is the upper cut-off frequency, ω l is the lower cut-off frequency, N is the order of the filter; s is the complex frequency variable, which is used to describe the frequency response characteristics of the filter.

[0062] Framing is to divide the denoised voiceprint signals into several small segments. To ensure the smooth transition between two adjacent frames of signals and avoid signal loss, there needs to be an overlap between two frames. The formula for framing the denoised signals in this embodiment is as follows:

[0063]

[0064] Where: M is the number of frames, N is the signal length, L is the frame length, and W is the overlap rate.

[0065] In voiceprint recognition, usually 20 - 30 ms is taken as one frame. Compared with voice signals, the voiceprint signals of fan blades are more stable. The frame length can be appropriately increased to improve the recognition accuracy. Therefore, the voiceprint signals of fan blades in different states can be cut into samples with a duration of 2 seconds. The frame length is uniformly set to 250 ms, and the overlap rate is set to 50%.

[0066] The preprocessed signal needs to be subjected to discrete Fourier transform. However, directly performing the transform on it will cause large distortion. Therefore, a windowing operation needs to be performed on the framed signal, that is, each frame needs to be multiplied by a window function. The windowing operation will gradually change the amplitude of a frame signal to zero at both ends, reducing its truncation effect. In this embodiment, it is selected to first apply a Hamming window with less frequency leakage and high frequency resolution to each framed signal and then perform the transform to increase the continuity at both ends of the signal. The Hamming window formula is:

[0067]

[0068] Where A is the Hamming window length.

[0069] 3) Perform discrete Fourier transform on the preprocessed voiceprint signal to convert the signal from the time domain to the time - frequency domain, obtaining a time - frequency spectrum matrix. Each row in the time - frequency spectrum matrix represents a time frame, each column represents a frequency component, and the element value in the matrix represents the energy magnitude of the frequency component in that time frame.

[0070] And a Mel filter bank is used to extract frequency - domain features from the preprocessed voiceprint signal, obtaining a Mel filter bank matrix. The Mel filter bank is a group of filters designed based on the Mel frequency scale, and its purpose is to simulate the perception characteristics of the human ear for different frequencies. The number of rows of the Mel filter bank matrix is equal to the number of filters, and the number of columns is the same as the number of frequency components of the time - frequency spectrum matrix. The transfer function of the Mel filter bank is shown in formula (5):

[0071]

[0072] Where: H m (f) is the filter parameter, m represents the number of filter banks. In this embodiment, m is set to 30; x(m) represents the center frequency of the triangular filter, and the calculation method is shown in formula (6)

[0073]

[0074] Where: f max and f minThey are the maximum frequency value and the minimum frequency value of the filtering range; f s is the sampling frequency of the acoustic sensor, which is 16 kHz in this embodiment; C is the frame length of the discrete Fourier transform. The calculation formula of the Mel value is as follows:

[0075]

[0076] where f is the frequency under the conventional scale, 0 ≤ f ≤ 8000; k is the frequency under the Mel scale.

[0077] Multiply the time-frequency spectrum matrix by the Mel filter bank matrix and then take the logarithm to obtain the Mel time-frequency spectrum diagram of the voiceprint signal. Multiplying the time-frequency spectrum matrix by the Mel filter bank matrix and taking the logarithm is actually to weight and sum the energy values of each frequency component in the time-frequency spectrum matrix according to the weights of the Mel filters, obtain the energy values of each Mel frequency band, and convert the energy values into logarithmic energy values to obtain the Mel time-frequency spectrum of the voiceprint signal of the fan blade in different states. The finally obtained Mel time-frequency spectrum is a two-dimensional matrix, where each row represents a time frame and each column represents the logarithmic energy value of a Mel frequency band.

[0078] 4) Use the Mixup method to perform data augmentation on the Mel time-frequency spectrum diagram with fault voiceprint features obtained in step 3). The main purpose of data augmentation is to increase the diversity of data during the deep learning training process, thereby improving the generalization ability of the model. Using the Mixup method for data augmentation is to fuse two Mel time-frequency diagrams and expand them into a new spectrum. The newly generated sample after fusion contains the features of two original samples, but is different from the original; its essence is to perform linear interpolation on the two, enhance the linear expression between samples. In order to facilitate the next step of the fault diagnosis model to classify and train the spectrum, all coordinate parameters of the spectrum are removed, and only the image part of the Mel time-frequency spectrum is retained. The formula for fusing two Mel time-frequency diagrams into a new time-frequency diagram is as follows:

[0079]

[0080] where x i and x j represent the feature vectors of any two Mel time-frequency diagram samples i and j respectively, y i and y j represent the label values of any two Mel time-frequency diagram samples i and j respectively; represents the feature vector of the expanded time-frequency diagram, represents the label value of the expanded time-frequency diagram; λ represents the fusion coefficient, which is set to 0.5 in this embodiment.

[0081] 5) Construct a hybrid model for identifying faults in wind turbine blades. The hybrid model is composed of an input layer, a convolutional neural network, a long short-term memory network, an attention layer, and an output layer connected together, and the hybrid model is trained in the following way: Use the Mel spectrogram obtained in step 3) and the enhanced data obtained in step 4) to form the training set and test set of the hybrid model, and train and test the hybrid model to obtain a qualified hybrid model. The processing process of the hybrid model for the input time-frequency map data is as follows:

[0082] a) The time-frequency map data sample X = [x1, x2,..., x t enters the hybrid model through the input layer, and the subscript t represents the length of the sample.

[0083] b) The Convolutional Neural Networks (CNN) extracts features from the samples input by the input layer and inputs the extracted features into the long short-term memory network. In this embodiment, the convolutional neural network includes a convolutional layer, a pooling layer, and a fully connected layer. The convolutional layer is the core of the convolutional neural network. It uses the convolutional kernel matrix to abstract the hidden correlations in the input data and extract features. The convolutional operation of each layer is carried out through the rectified linear unit (ReLU) activation function as follows

[0084] f(x) = max(0, x) (9)

[0085] After the activation function processing, the filter generates the following features

[0086]

[0087] where, in the convolutional layer j, is the result of the l-th filter, f represents the non-linear function, the operator * represents convolution, is the l-th convolutional kernel between the i-th input mapping and the j-th output mapping, is the bias.

[0088] The convolutional layer extracts a large number of features of the input data. When performing feature operations, the computational efficiency is relatively low. Therefore, it is necessary to solve this problem through the pooling layer. The pooling layer is responsible for screening the features in the receptive field and extracting the most representative features in the region, thereby effectively reducing the dimension of the output features and the number of model parameters required. Pooling is divided into average pooling and max pooling. Average pooling can retain more object background information and reduce the large differences in estimated values caused by neighborhood limitations. Max pooling, on the other hand, can retain more texture information of the object while reducing the mean shift of the estimate caused by convolutional layer parameter errors. In this embodiment, voiceprint information is used for fault detection of wind turbine blades, so the max pooling method is adopted.

[0089] The fully connected layer is the last layer of the convolutional neural network in this embodiment. It connects each neuron to the neurons before and after its use, and calculates the weights and biases of the features, so as to obtain the output of the feature information. The output feature vector of the convolutional neural network is H C =[h C1 ,h C2 ,...,h Ci T ,and the calculation formula is

[0090]

[0091] P = max(C) + b P (12)

[0092] H C = f(W H ·P + b H ) (13)

[0093] Among them, C is the output of the convolutional layer, W C and b C are the weights and biases of the convolutional layer respectively, is the convolutional operator, P is the output of the pooling layer, max() is the max pooling mode, b P is the bias of the pooling layer; f(·) is the activation function of the fully connected layer, W H is the weight of the fully connected layer, b H is the sum bias of the fully connected layer.

[0094] c) The long short-term memory network extracts features from the output of the convolutional neural network. The long short-term memory network extracts the feature vector H L =[h L1 ,h L2 ,...,h Li T ​​Input attention layer. Long Short-Term Memory (LSTM) is a unique type of RNN memory. LSTM adds gating mechanisms that can remember information through the cell state. The forget gate can prevent excessive memory from affecting the neural network's processing of the current input. Each time a new input is received, based on the most recent input and output, LSTM first selects which previous memories to erase. The memory gate is a control unit that determines whether the data at time t (now) is included in the state. It can filter out invalid data in the current input and extract valid data. The neural layer in the LSTM unit used to determine the current output value is the output gate. After integrating the current input value with the output value of the previous time step using the sigmoid function, the output layer first extracts information from the vector and then uses the tanh function to compress and map the current cell state to the interval (-1, 1). LSTM introduces the sigmoid function through three gates and combines it with the tanh function, increasing the summation step size, reducing the likelihood of gradient vanishing and gradient explosion, and simultaneously solving the problems of short-term and long-term dependencies. The structure of the LSTM cell is as shown in Figure 2 shown, and its calculation formulas are shown in Formulas (14)-(19)

[0095] g t = σ(W g · [z t-1 , x t + b g ) (14)

[0096] i t = σ(W i · [z t-1 , x t + b i ) (15)

[0097] V t = σ(g t · V t-1 + i t · V z ) (16)

[0098] V z = tanh(W V · [z t-1 , x t + b V ) (17)

[0099] o t = σ(W o · [z t-1 , x t + b o ) (18)

[0100] z t = o t ·tanh(V t ) (19)

[0101] Among them, x t is the network input matrix, and σ is the activation function. V t-1 is the old cell state, which is updated to the new cell state V t through formula (14). tanh is the hyperbolic tangent activation function. W g , W i , W V , Wo are the network model parameters, and b g , b i , b V , b o is the offset vector of the network. The model updates the weights and biases by minimizing the objective function.

[0102] d) The attention layer assigns different parameters to different features to create an ideal weight parameter matrix. The attention layer is S = [s1, s2,..., s k T . The output of the attention layer enters the output layer. The attention layer can capture important information useful for the current task, highlight the important features affecting the effect, reduce the influence of useless features, enable the model to make the optimal choice, and improve the accuracy of the model. As shown in Figure 3 , in the attention mechanism structure, x1, x2,..., x i are the input feature values, h1, h2,..., h i are the hidden layer state values specific to the input features, a t is the weight value of the current input, which is equivalent to the hidden layer state value of the historical input, and h t ' is the hidden layer state value output by the final node. The calculation formula of the attention mechanism is:

[0103] e i = tanh(wh i + b) (20)

[0104]

[0105]

[0106] Among them, w and b are the weight parameters and biases, and e i is the attention probability distribution value determined by the input vector h i at the i-th moment, and s i is the final output feature.

[0107] ​e) The output layer outputs the fan blade status data Y, and the calculation formula of Y is as follows:

[0108] Y = f(W Y ·S + b Y ) (23)

[0109] Among them, W Y is the weight of the output layer, and b Y is the bias of the output layer.

[0110] 6) Process the acoustic fingerprint signal of the fan blade collected in real time by the acoustic sensor as described in steps 2) and 3), and then input the obtained Mel time-frequency spectrogram into the qualified hybrid model obtained in step 5), and the fault detection result is obtained by the hybrid model.

[0111] In a specific implementation, the fan blade fault diagnosis method in this embodiment can also be combined with remote monitoring technology, and the diagnosis result can be sent to the operation and maintenance personnel by means of text messages, etc. It is also possible to allow the operation and maintenance personnel to remotely access the acoustic fingerprint data and diagnosis results of the fan blades, so as to realize remote monitoring and fault handling, and improve the management efficiency of the wind farm.

[0112] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the purpose and scope of the technical solutions of the present invention, and they should all be covered by the scope of the claims of the present invention.

Claims

1. A fan blade fault diagnosis method based on voiceprint recognition, characterized in that: The following steps are involved: 1) The acoustic sensor arranged at the root of the fan blade collects the soundprint signal of the fan blade to obtain the soundprint signal of the normal blade and the soundprint signal of the faulty blade, and the soundprint signal of the faulty blade is marked with the fault type and the distance from the fault to the blade root; 2) Preprocessing the voiceprint signal marked in step 1), wherein the preprocessing includes denoising the voiceprint signal, framing the denoised signal, and windowing the framed signal; 3) Perform discrete Fourier transform on the preprocessed voiceprint signal to convert the signal from the time domain to the time-frequency domain to obtain the time-frequency spectrum matrix; The Mel filter bank is used to extract the frequency domain features of the preprocessed voiceprint signal to obtain the Mel filter bank matrix; Multiply the time-frequency spectrum matrix and the Mel filter bank matrix and take the logarithm to obtain the Mel time-frequency spectrum of the voiceprint signal; 4) The Mel-time spectrum diagram containing the fault voiceprint features obtained in step 3) is enhanced by using the Mixup method; 5) constructing a hybrid model for identifying wind turbine blade faults, wherein the hybrid model is composed of an input layer, a convolutional neural network, a long short-term memory network, an attention layer, and an output layer, and training the hybrid model in the following manner: using the Mel-time spectrum obtained in step 3) and the enhanced data obtained in step 4) to form a training set and a test set of the hybrid model, training and testing the hybrid model to obtain a qualified hybrid model; 6) The fan blade soundprint signal collected in real time by the acoustic sensor is processed as described in step 2) and step 3), and then the obtained Mel-time spectrum is input into the qualified hybrid model obtained in step 5), and the fault detection result is obtained by the hybrid model.

2. The method for diagnosing fan blade faults based on voiceprint recognition according to claim 1 is characterized in that: In step 2), a Butterworth bandpass filter is used to filter and denoise the voiceprint signal. The filter formula is as follows: Where: H 2 (p) is the square amplitude frequency, ω c is the cut-off frequency, ω u is the upper cutoff frequency, ω l is the lower cut-off frequency, N is the order of the filter, and s is the complex frequency variable.

3. The method for diagnosing fan blade faults based on voiceprint recognition according to claim 1, characterized in that: The formula for framing the denoised signal in step 2) is as follows: Where: M is the number of frames, N is the signal length, L is the frame length, and W is the overlap rate.

4. The method for diagnosing fan blade faults based on voiceprint recognition according to claim 1, characterized in that: In step 2), a Hamming window operation is performed on the framed signal, and the Hamming window formula is: Where A is the Hamming window length.

5. The method for diagnosing fan blade faults based on voiceprint recognition according to claim 1, characterized in that: The transfer function of the Mel filter bank described in step 3) is shown in formula (5): Where: H m (f) is the filter parameter, m is the number of filter banks, and x(m) is the center frequency of the triangular filter. The calculation method is shown in formula (6): Where: f max and f min are the maximum and minimum frequency values ​​of the filtering range respectively; f s is the sampling frequency of the acoustic sensor, C is the frame length of the discrete Fourier transform; the calculation formula of the Mel value is as follows: Where f is the frequency under the conventional scale, 0≤f≤8000; k is the frequency under the Mel scale.

6. The method for diagnosing fan blade faults based on voiceprint recognition according to claim 1, characterized in that: The data enhancement using the Mixup method in step 4) is to fuse the two Mel time-frequency graphs into a new time-frequency graph through the following formula: Among them, x i and x j Represents the feature vectors of any two Mel-spectrogram samples i and j, y i and j Represent the label values ​​of any two Mel-spectrogram samples i and j respectively; The eigenvector representing the extended time-frequency graph, represents the label value of the extended time-frequency graph, and λ represents the fusion coefficient.

7. The method for diagnosing fan blade faults based on voiceprint recognition according to claim 1, characterized in that: In step 5), the hybrid model processes the input time-frequency graph data as follows: a) The time-frequency graph data samples enter the hybrid model through the input layer; b) The convolutional neural network extracts features from the samples input into the input layer and inputs the extracted features into the long short-term memory network; c) The LSTM network extracts features from the output of the convolutional neural network, and the LSTM network inputs the extracted feature vector into the attention layer; d) The attention layer assigns different parameters to different features to create an ideal weight parameter matrix, and the output of the attention layer enters the output layer; e) The output layer outputs the status data of the fan blades.

Images