A method for detecting anomalies in wind turbine blades based on acoustic signatures

Abstract

This invention relates to a method for detecting abnormalities in wind turbine blades based on acoustic signatures, comprising the following steps: acquiring raw audio data of three wind turbine blades respectively, and converting the acquired raw audio data into Mel spectrograms; establishing a coarse extraction model based on a convolutional neural network, performing convolution operations on the Mel spectrograms to extract several first feature signals containing acoustic signature features; establishing a blade acoustic signature capture model based on a long short-term memory neural network, determining which first feature signal contains the sound of the wind turbine blade, and outputting the first feature signal containing the sound of the wind turbine blade as a second feature signal; calculating the signal-to-noise ratio (SNR) of the second feature signal, removing noise signals from the second feature signal based on the calculated SNR, and calculating the feature distribution of each second feature signal; calculating the Fréchet distance between each pair of second feature signals based on the feature distribution, and determining whether the wind turbine blade has experienced an abnormality based on this distance.

Description

Technical Field

[0001] This invention relates to a method for detecting abnormalities in wind turbine blades based on acoustic signatures, belonging to the field of wind turbine blade detection technology. Background Technology

[0002] The blades are a key component of wind turbines for harvesting wind energy. Because they operate in harsh natural environments and are subjected to various complex forces, they are highly susceptible to faults such as pinholes, wear, skin detachment, cracks, and breakage. By collecting sound data of the blades operating at the wind farm, abnormalities can be detected.

[0003] For example, patent number "CN114120974A" discloses a wind turbine blade fault diagnosis method based on deep learning. It collects the sound signal of the wind turbine and monitors it in real time. If a fault is found, it can immediately alarm. It has high real-time performance. It performs spectrum analysis on the sound signal to obtain feature parameter values ​​based on short-time stationary conditions, retaining more original sound features. At the same time, it reduces the redundancy brought by the time domain. It is trained by a deep learning model to automatically extract image features. It uses multiple similarity calculation and comparison methods to analyze the similarity of spectrum images from different dimensions to achieve blade fault diagnosis.

[0004] However, the problem with the above-mentioned existing technology is that the blades operate in an outdoor environment, which is a noisy environment, unlike the indoor environment. There is a lot of noise, and the noise is not a fixed noise source but is diverse. A lot of noise can have a great impact on the test results. Summary of the Invention

[0005] To address the problems existing in the prior art, this invention proposes a method for detecting abnormal wind turbine blades based on acoustic signatures. In the noisy environment where wind turbines operate, acoustic signature data of the wind turbine blades is extracted from the sound signal using a neural network model. This acoustic signature data is then denoised before anomaly detection, significantly improving the accuracy of wind turbine blade anomaly detection.

[0006] The technical solution of the present invention is as follows:

[0007] On the one hand, this invention proposes a method for detecting abnormalities in wind turbine blades based on acoustic signatures, comprising the following steps:

[0008] Raw audio data from the three blades of the wind turbine were collected separately and preprocessed to convert the three raw audio data into three Mel spectrograms.

[0009] A coarse extraction model based on a convolutional neural network is established. The input is a Mel spectrogram. The Mel spectrogram is convolved to extract several first feature signals containing voiceprint features from the Mel spectrogram.

[0010] A blade acoustic signature capture model based on a long short-term memory neural network is established to receive each first feature signal, determine which first feature signal contains the sound of the wind turbine blade, and output the first feature signal containing the sound of the wind turbine blade as the second feature signal.

[0011] Calculate the signal-to-noise ratio of the second feature signal, and remove the noise signal from the second feature signal based on the calculated signal-to-noise ratio;

[0012] Obtain the three second feature signals after removing the noise signal, and calculate the feature distribution of each second feature signal;

[0013] Based on the calculated characteristic distribution of each second characteristic signal, the Fréchet distance between each pair of second characteristic signals is calculated, and the calculated Fréchet distance is used to determine whether the wind turbine blades are abnormal.

[0014] In a preferred embodiment, the specific steps of acquiring the raw audio data corresponding to the three blades respectively, and preprocessing them to convert the three acquired raw audio data into three Mel spectrograms are as follows:

[0015] For any original audio data, perform a short-time Fourier transform to calculate the sound signal at a specified sampling frequency in the original audio data, obtain the power spectrum from the calculated sound signal, and then convert the power spectrum into a Mel spectrogram.

[0016] In a preferred embodiment, the coarse extraction model includes:

[0017] The selection layer is used to select n neighboring voiceprint feature vectors from the input Mel spectrogram;

[0018] The convolutional layer, containing several convolution operations and a ReLU activation function, is used to perform convolution operations on the selected n voiceprint feature vectors. The specific formula is as follows:

[0019] y i，j =max(x i *w j ,0)

[0020] In the formula, x i It is the i-th voiceprint feature vector, w j It is the j-th convolution operation with a kernel size of 1×1, y i，j It is the input x i and w j The first convolutional feature map extracted by performing convolution operations;

[0021] The feature fusion layer contains several convolutional operations, residual connective blocks, and ReLU activation functions. It is used to perform convolution operations on the first convolutional feature map to obtain features, and then fuse the first convolutional feature map with the currently obtained features. The specific formula is as follows:

[0022] z i，l =max(max(y) i，j *f j，k ,0)*w k，l ,0)+y i，j

[0023] In the formula, f j，k Is with w j The corresponding k-th convolution operation with a kernel size of 5×1, w k，l To be with f j，k The corresponding I-th filter with a convolution kernel size of 5×1; z i，l The result of the calculation;

[0024] For z i，l Perform convolution operation, outputting the result with x. i The second convolutional feature map of the same size is as follows:

[0025] v i =z i,l *w m

[0026] In the formula, v i For the second convolutional feature map, w m For a convolution operation with a kernel size of 1×1, v i Size and x i same;

[0027] The feature enhancement layer includes a sigmoid activation function, which calculates the first feature signal based on the second convolutional feature map, as detailed below:

[0028]

[0029] In the formula, v i 'For the i-th voiceprint feature vector x i The corresponding first characteristic signal, e is the natural constant.

[0030] In a preferred embodiment, an attention module is inserted into the long short-term memory neural network in the blade acoustic signature capture model to calculate the attention weight of the first feature signal containing the sound of the wind turbine blades. The specific formula is as follows:

[0031]

[0032] In the formula, attentioni Let a and W be the attention weights for the i-th first feature signal. x and W h For learning parameters, a T Let be the transpose of a, tanh() be the hyperbolic tangent function, and h be the internal state vector of the long short-term memory neural network.

[0033] As a preferred embodiment, the specific method for calculating the signal-to-noise ratio of the second feature signal and removing noise signals from the second feature signal based on the calculated signal-to-noise ratio is as follows:

[0034] Obtain the second feature signal, and calculate the prior signal-to-noise ratio (SNR) and posterior signal-to-noise ratio (SNR) of the second feature signal according to the following formulas:

[0035]

[0036] ξ k =0.98*ξ k,old +0.02*max(γ k -1,0)

[0037] Where, γ k It is the posterior signal-to-noise ratio estimated from the k-th frequency component in the second characteristic signal. It is the k-th frequency component, λ d (k) is the power spectrum of the k-th frequency corresponding to the noise signal; ξ k ξ is the prior signal-to-noise ratio estimated from the k-th frequency component in the second characteristic signal. k,old This represents the previously estimated a priori signal-to-noise ratio, with the first a priori signal-to-noise ratio value initialized to 0.

[0038] Calculate the spectral gain:

[0039]

[0040] Among them, gain k Let be the spectral gain of the k-th spectral component;

[0041] Multiplying the k-th frequency component in the second characteristic signal by the corresponding spectral gain eliminates the noise signal, specifically:

[0042]

[0043] in, To remove the k-th frequency component of the noise signal;

[0044] Based on eliminating each frequency component of the noise signal, a second characteristic signal is obtained after removing the noise signal.

[0045] In a preferred embodiment, the method for obtaining the three second feature signals after removing the noise signal and calculating the feature distribution of each second feature signal is as follows:

[0046] For any second feature signal, feature extraction is performed through convolution operation to obtain m n-dimensional feature vectors Z. The mean of each dimension of the m feature vectors Z is calculated to obtain an n-dimensional vector μ. Through the m n-dimensional feature vectors, an n*n-order feature covariance matrix is ​​calculated. The n-dimensional vector μ and the feature covariance matrix are used as the feature distribution of the corresponding second feature signal.

[0047] In a preferred embodiment, the method for calculating the Fréchet distance between each pair of the calculated second characteristic signals based on their characteristic distributions, and then determining whether the wind turbine blades are abnormal based on the calculated Fréchet distance, specifically includes:

[0048] The Fréchet distance between the characteristic distributions of any pairwise second characteristic signals can be calculated using the following formula:

[0049]

[0050] Where, μ i and μ j The n-dimensional vectors of the two second feature signals respectively, ∑ i and ∑ j Let be the characteristic covariance matrix of the two second characteristic signals, and Tr represent the sum of the elements on the diagonal of the matrix;

[0051] The difference between two second characteristic signals is measured by the calculated Fréchet distance. The smaller the calculated Fréchet distance, the closer the two second characteristic signals are. A threshold is set. If the difference between the Fréchet distances of the second characteristic signals corresponding to any two wind turbine blades is less than the threshold, then the wind turbine blades are not abnormal. If the difference between the Fréchet distances of the second characteristic signals corresponding to two wind turbine blades is greater than the threshold, then the wind turbine blades are abnormal.

[0052] On the other hand, this invention proposes a wind turbine blade anomaly detection system based on acoustic signature, comprising:

[0053] The data acquisition module is used to collect the raw audio data of the three blades of the wind turbine, and preprocess the data to convert the three raw audio data into three Mel spectrograms.

[0054] The voiceprint feature extraction module is used to establish a coarse extraction model based on a convolutional neural network. It takes a Mel spectrogram as input, performs a convolution operation on the Mel spectrogram, and extracts several first feature signals containing voiceprint features from the Mel spectrogram.

[0055] The blade acoustic signature extraction module is used to establish a blade acoustic signature capture model based on a long short-term memory neural network. It is used to receive each first feature signal, determine which first feature signal contains the sound of the wind turbine blade, and output the first feature signal containing the sound of the wind turbine blade as the second feature signal.

[0056] The noise reduction module is used to calculate the signal-to-noise ratio of the second feature signal and remove noise signals from the second feature signal based on the calculated signal-to-noise ratio.

[0057] The feature distribution calculation module is used to obtain three second feature signals after removing the noise signal, and calculate the feature distribution of each second feature signal respectively.

[0058] The anomaly detection module is used to calculate the Fréchet distance between each pair of the calculated second feature signals based on the feature distribution of each second feature signal, and to determine whether the wind turbine blades have an anomaly based on the calculated Fréchet distance.

[0059] In another aspect, the present invention also proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the wind turbine blade anomaly detection method based on acoustic print as described in any embodiment of the present invention.

[0060] In another aspect, the present invention also proposes a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the wind turbine blade anomaly detection method based on acoustic signature as described in any embodiment of the present invention.

[0061] The present invention has the following beneficial effects:

[0062] This invention discloses a method for detecting anomalies in wind turbine blades based on acoustic signatures. Addressing the noisy environment in which wind turbines operate, the method extracts a first feature signal containing acoustic signature characteristics using a coarse extraction model based on a convolutional neural network. Then, an LSTM neural network is used to capture a second feature signal containing the sound of the wind turbine blades within the first feature signal. The signal-to-noise ratio of the second feature signal is calculated, and noise is removed. Finally, anomaly detection is performed based on the distance between the feature distributions of the second feature signals of each wind turbine blade after noise removal. This significantly improves the accuracy of wind turbine blade anomaly detection. Attached Figure Description

[0063] Figure 1 This is a schematic diagram of the method flow according to an embodiment of the present invention;

[0064] Figure 2 This is an example diagram of wind turbine blade anomaly detection in an embodiment of the present invention;

[0065] Figure 3This is another example diagram of wind turbine blade anomaly detection in an embodiment of the present invention. Detailed Implementation

[0066] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0067] It should be understood that the step numbers used in the text are for ease of description only and are not intended to limit the order in which the steps are performed.

[0068] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0069] The terms “comprising” and “including” indicate the presence of the described feature, whole, step, operation, element and / or component, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or collections thereof.

[0070] The term “and / or” refers to any combination of one or more of the associated listed items, as well as all possible combinations, and includes these combinations.

[0071] Example 1:

[0072] See Figure 1 This embodiment proposes a method for detecting abnormalities in wind turbine blades based on acoustic signatures. It obtains acoustic signature data from the wind turbine blades and creates an anomaly monitoring algorithm to capture abnormal signs in the blades. The method specifically includes the following steps:

[0073] S100. Audio acquisition devices are installed on all three blades of the wind turbine. The audio acquisition devices mainly include a microphone and a sound detection unit, which are used to collect the raw audio data of the three blades of the wind turbine respectively and perform preprocessing to convert the three raw audio data into three Mel spectrograms respectively. In this embodiment, a neural network model is used for feature extraction in subsequent steps. However, if the raw audio data is directly input into the neural network model, the input feature dimension is too large, making feature extraction difficult. Therefore, the raw audio data is replaced with Mel spectrograms. Mel spectrograms are "snapshots" of sound waves. Because they are images, they are well-suited for use in CNN-based architectures that process images. A spectrogram is generated from a sound signal using the Fourier Transforms method, which breaks the sound signal down into its constituent frequencies and displays the amplitude of each frequency. Mel spectrograms shorten the duration of the sound signal into smaller time intervals, then apply Fourier Transforms to each interval to determine the frequencies contained within that interval. The Fourier Transforms from all time intervals are then combined into a single graph. Mel spectrograms plot the relationship between frequency (Y-axis) and time (X-axis), using different colors to represent the amplitude of each frequency; brighter colors indicate a stronger sound signal.

[0074] S200. Establish a coarse extraction model based on a convolutional neural network, which is used to input the Mel spectrogram, perform convolution operation on the Mel spectrogram, and extract several first feature signals containing voiceprint features from the Mel spectrogram.

[0075] S300. The sound of wind turbine blades can have repetitive mechanical acoustic signature features with a constant period. Unlike ambient noise in noisy environments, the repetitive mechanical acoustic signature features of wind turbine blades do not change much in the frequency band and appear continuously in the main frequency band. Based on this characteristic, this embodiment establishes a blade acoustic signature capture model based on a Long Short-Term Memory (LSTM) neural network to receive each first feature signal, determine which first feature signal contains the wind turbine blade sound, and output the first feature signal containing the wind turbine blade sound as the second feature signal. LSTM neural network is a special type of recurrent neural network. Recurrent neural networks are neural networks used to process sequential data. The most commonly used and powerful tool for time series models is the recurrent neural network (RNN). Compared to the independent characteristics of the calculation results of ordinary neural networks, the calculation result of each hidden layer of an RNN is related to the current input and the result of the previous hidden layer, making it very suitable for processing data with sequential changes. In this way, the calculation result of an RNN has the characteristic of remembering the results of previous steps. In a standard RNN, this repetitive module has only a very simple structure, such as a single tanh layer. The recurrent structure allows information to be passed from the current step to the next step. However, RNNs suffer from the vanishing gradient problem. This means that if the gap between the relevant information and the predicted location is small, the RNN can learn to use previous information. But as this gap increases, the RNN loses its ability to learn connections that are so far apart. The emergence of LSTM neural networks solved this long dependency problem.

[0076] Unlike standard RNNs where each unit has only one layer, LSTM has four layers that interact in a unique way. The key feature of LSTM is the addition of valve nodes to each layer outside the RNN structure. There are three types of valves: forget gates, input gates, and output gates. These valves can be opened or closed to determine whether the network's memory state (the state of the previous network) meets a threshold for inclusion in the current layer's computation. The valve node uses the sigmoid function to take the network's memory state as input; if the output meets the threshold, it multiplies the valve output by the current layer's result and uses it as the input for the next layer; otherwise, it forgets the output. The weights of each layer, including the valve nodes, are updated during each backpropagation training iteration. A more detailed calculation process for LSTM's decision-making is as follows:

[0077] The key to LSTM is the cell state. LSTM has the ability to remove or add information to the cell state through carefully designed structures called "gates." Gates are a method of selectively allowing information to pass through. They consist of a sigmoid layer and a pointwise multiplication operation. The sigmoid layer outputs a value between 0 and 1, describing how much of each quantity can pass through. 0 means "no quantity is allowed to pass," and 1 means "any quantity is allowed to pass." LSTM has three gates to protect and control the cell state.

[0078] The basic steps of an LSTM neural network are as follows:

[0079] (1) Decide to discard information

[0080] The first step in an LSTM network is deciding what information to discard from the cell state. This decision is made through a gate called the "forget gate." This gate reads ht-1 and xt and outputs a value between 0 and 1 to each number in the cell state Ct-1. 1 means "completely keep," and 0 means "completely discard."

[0081] f t =σ(W f ×[h t-1 ,x t ]+b f );

[0082] (2) Determine the updated information

[0083] The next step is to determine what new information is stored in the cell state. This involves two parts. First, the sigmoid layer, called the "input gate," determines what values ​​we will update.

[0084] Then, a tanh layer creates a new candidate value vector. Add it to the status.

[0085] i t =σ(W i ×[h t-1 ,x t ]+b i );

[0086]

[0087] (3) Update cell state

[0088] Update the old cell state, Ct-1 to Ct, and generate the state update based on the information calculated in the previous steps. Compare the old state with f. t Multiply, discard the information we determine needs to be discarded, and then add. These are the new candidate values.

[0089]

[0090] (4) Output information

[0091] First, a sigmoid layer is run to determine which part of the cell state will be output. Next, the cell state is processed by tanh (resulting in a value between -1 and 1) and multiplied by the output of the sigmoid gate. Finally, the network will output only the part of the information that we determined to output, based on the cell state.

[0092] o t =σ(W o ×[h t-1 ,x t ]+b o );

[0093] h t =o t ×tanh(C t );

[0094] In summary, the LSTM neural network is an improved recurrent neural network (RNN). It solves the "vanishing gradient" problem in RNN model training, can learn long-term and short-term dependencies in time series, and is currently the most successful RNN architecture.

[0095] S400. Calculate the signal-to-noise ratio of the second characteristic signal, and remove the noise signal from the second characteristic signal based on the calculated signal-to-noise ratio. Since the noise signal changes with time, the variance of the noise can be estimated as long as there is no sound from the fan blades. Then, the noise signal is removed from the second characteristic signal to improve the quality of the sound signal.

[0096] S500: Obtain the three second characteristic signals after removing the noise signal, and calculate the characteristic distribution of each second characteristic signal respectively;

[0097] S600: Based on the calculated feature distribution of each second feature signal, calculate the Fréchet distance between each pair of second feature signals, and determine whether the wind turbine blades are abnormal based on the calculated Fréchet distance. Based on the above steps S500 and S600, this embodiment obtains the feature distribution of three second feature signals respectively, measures the differences between each second feature signal through statistical methods, avoids the limitations of simply obtaining a single feature data for comparison, and finally quantifies the distribution differences between each second feature signal based on the Fréchet distance, which can quickly compare the differences between each second feature signal; thus, it is possible to quickly predict whether the wind turbine blades are abnormal based on the magnitude of the differences between the second feature signals corresponding to each wind turbine blade, and realize anomaly detection.

[0098] In a preferred embodiment of this invention, in step S100, the specific steps of acquiring the raw audio data corresponding to the three blades respectively, and preprocessing them to convert the three acquired raw audio data into three Mel spectrograms are as follows:

[0099] S101. For any original audio data, perform a Short-Time Fourier Transform (STFT). After subdividing the sound signal using a window of predetermined length, analyze the frequency of the sound signal and calculate the sound signal at a specified sampling frequency in the original audio data. The Short-Time Fourier Transform (STFT) is the most classic time-frequency domain analysis method. The principle of the STFT is to divide a long signal into frames, window it, perform a Fourier Transform (FFT) on each frame, and finally stack the results of each frame along another dimension (frequency domain) (rotated 90 degrees) to obtain a two-dimensional signal form similar to an image. These amplitudes are then mapped to a grayscale representation, where 0 represents black and 255 represents white; the larger the amplitude value, the darker the corresponding area.

[0100] S102. Obtain the power spectrum from the calculated sound signal, and then convert the power spectrum into a Mel spectrogram using a set of Mel filters based on the Mel-Frequency Cepstral Coefficients (MFCC) method.

[0101] In a preferred embodiment of this example, in step S200, the coarse extraction model includes:

[0102] The selection layer is used to select n adjacent voiceprint feature vectors from the input Mel spectrogram; in this embodiment, 8 voiceprint feature vectors are specifically selected.

[0103] The convolutional layer contains 32 1×1 convolutional operations and a ReLU activation function. Along the frequency axis, the convolutional layer consists of 32 1×1 filters used to perform convolution operations on the selected 8 speaker feature vectors. The specific formula is as follows:

[0104] y i,j =max(x i *w j ,0)

[0105] In the formula, x i It is the i-th voiceprint feature vector, i≤8, w j It is the j-th convolution operation with a kernel size of 1×1, where j≤32, y i,j It is the input x i and w j The first convolutional feature map extracted by performing convolution operations;

[0106] The feature fusion layer contains 64 convolutional operations with a kernel size of 5×1, residual connective blocks, and ReLU activation functions. It is used to perform convolution operations on the first convolutional feature map to obtain features, and then fuse the first convolutional feature map with the currently obtained features. The specific formula is as follows:

[0107] z i,l =max(max(y) i,j *f j,k ,0)*w k,l ,0)+y i,j

[0108] In the formula, f j,k Is with w j The corresponding k-th convolution operation with a kernel size of 5×1, w k,l To be with f j,k The corresponding l-th filter with a convolution kernel size of 5×1; z i,l The result of the calculation;

[0109] To make the network output match the input size, for z i,l Perform convolution operation, outputting the result with x. i The second convolutional feature map of the same size is as follows:

[0110] v i =z i,l *w m

[0111] In the formula, v i For the second convolutional feature map, w m For a convolution operation with a kernel size of 1×1, v i Size and x i same;

[0112] The feature enhancement layer includes a sigmoid activation function, which calculates the first feature signal based on the second convolutional feature map, as detailed below:

[0113]

[0114] In the formula, v i 'For the i-th voiceprint feature vector x i The corresponding first feature signal, e is a natural constant; through the feature improvement layer, the input value is standardized to the output value of 0 to 1, thereby improving the acoustic characteristics of the first feature signal.

[0115] In a preferred embodiment of this invention, in step S300, an attention module is inserted into the long short-term memory neural network in the blade acoustic signature capture model. This module is used to calculate the attention weights of the eight first feature signals that contain the sound of the wind turbine blades, thereby determining which first feature signal contains the sound of the wind turbine blades. The specific formula is as follows:

[0116]

[0117] In the formula, attention i Let a and W be the attention weights for the i-th first feature signal. x and W h For learning parameters, a T Let be the transpose of a, tanh() be the hyperbolic tangent function, and h be the internal state vector of the long short-term memory neural network.

[0118] A drawback of neural network architectures like LSTMs that process sequential data is that the initially stored information disappears over time, which is compensated for by attention algorithms. Attention algorithms use LSTMs to process sequential input data and consistently output a state vector h. Instead of referencing the vector at the same rate across all time points, attention is given at specific times to arrive at the final result.

[0119] In a preferred embodiment of this example, in step S400, the specific method for calculating the signal-to-noise ratio of the second feature signal and removing noise signals from the second feature signal based on the calculated signal-to-noise ratio is as follows:

[0120] S401. Obtain the second characteristic signal. In a noisy environment, since it is unknown whether the second characteristic signal is mixed with noise signals, it is necessary to estimate the signal-to-noise ratio (SNR). Calculate the prior and posterior SNR of the second characteristic signal according to the following formulas:

[0121]

[0122] ξ k =0.98*ξ k,old +0.02*max(γ k -1,0)

[0123] Where, γ k It is the posterior signal-to-noise ratio estimated from the k-th frequency component in the second characteristic signal. It is the k-th frequency component, λ d (k) is the power spectrum of the k-th frequency corresponding to the noise signal; ξ k ξ is the prior signal-to-noise ratio estimated from the k-th frequency component in the second characteristic signal. k,oldThis represents the previously estimated a priori signal-to-noise ratio, with the first a priori signal-to-noise ratio value initialized to 0.

[0124] S402. Calculate the spectral gain:

[0125]

[0126] Among them, gain k Let be the spectral gain of the k-th spectral component; the value of the spectral gain is between 0 and 1.

[0127] S403. Multiply the k-th frequency component in the second characteristic signal by the corresponding spectral gain to eliminate the noise signal, specifically:

[0128]

[0129] in, To remove the k-th frequency component of the noise signal;

[0130] Based on eliminating each frequency component of the noise signal, a second characteristic signal is obtained after removing the noise signal.

[0131] In a preferred embodiment of this example, in step S500, the method for obtaining the three second feature signals after removing the noise signal and calculating the feature distribution of each second feature signal is specifically as follows:

[0132] S501. For any second feature signal, feature extraction is performed through convolution operation to obtain m n-dimensional feature vectors Z. The mean of each dimension of the m feature vectors Z is calculated to obtain an n-dimensional vector μ. The n*n-order feature covariance matrix is ​​calculated through the m n-dimensional feature vectors.

[0133] For example, for a second feature signal, the first convolution operation is performed on the second feature signal to obtain the first n-dimensional feature vector. The second feature signal is convolved by a second convolution operation to obtain the second n-dimensional feature vector. Similarly, by performing a convolution operation on the second feature signal using the m-th convolution operation, the m-th n-dimensional feature vector is obtained. Calculate the mean of each dimension of the above m n-dimensional eigenvectors:

[0134] We obtain an n-dimensional vector μ = [y1, y2, y3, ..., y n ];

[0135] Continue by calculating the feature covariance matrix based on the above m n-dimensional eigenvectors:

[0136] Z xy=E[(xE[x])(yE[y])], where E[x] represents the expectation of variable x;

[0137]

[0138]

[0139]

[0140]

[0141] …

[0142] Similarly, the eigencovariance matrix of order n*n is calculated:

[0143]

[0144] S502. The above n-dimensional vector μ and feature covariance matrix are used as the feature distribution of the corresponding second feature signal.

[0145] In a preferred embodiment of this example, in step S600, the method for calculating the Fréchet distance between each pair of the calculated second feature signals based on their characteristic distributions, and for determining whether the wind turbine blades are abnormal based on the calculated Fréchet distance, specifically includes:

[0146] S601. Calculate the Fréchet distance between the characteristic distributions of any pair of second characteristic signals according to the following formula:

[0147]

[0148] Where, μ i and μ j The n-dimensional vectors of the two second feature signals respectively, ∑ i and ∑ j Let be the characteristic covariance matrix of the two second characteristic signals, and Tr represent the sum of the elements on the diagonal of the matrix;

[0149] S602. The difference between two second characteristic signals is measured based on the calculated Fréchet distance. The smaller the calculated Fréchet distance, the closer the two second characteristic signals are. A threshold is set. If the difference between the Fréchet distances of the second characteristic signals corresponding to any two wind turbine blades is less than the threshold, then the wind turbine blades are not abnormal. If the difference between the Fréchet distances of the second characteristic signals corresponding to two wind turbine blades is greater than the threshold, then the wind turbine blades are abnormal.

[0150] See details Figure 2 and Figure 3 First, set the threshold to 0.1, such as... Figure 2 As shown in the figure, the Mel-ray spectrograms of three wind turbine blades are blade A, blade B, and blade C. If, through the above steps, the Fréchet distance Fab = 0.87 between the second characteristic signals corresponding to blade A and blade B; the Fréchet distance Fac = 0.86 between the second characteristic signals corresponding to blade A and blade C; and the Fréchet distance Fbc = 0.88 between the second characteristic signals corresponding to blade B and blade C, then the maximum difference between each Fréchet distance is only 0.02, which is less than the threshold of 0.1. Therefore, the wind turbine blades have not experienced any abnormalities. Figure 3 As shown, if the calculated Fab = 0.76, Fac = 0.77, and Fbc = 0.87, then the difference between Fab and Fbc = 0.11 is greater than the threshold of 0.1, and the difference between Fac and Fbc = 0.1 is equal to the threshold of 0.1. Therefore, the wind turbine blades of the wind turbine have become abnormal.

[0151] Example 2:

[0152] This embodiment proposes a wind turbine blade anomaly detection system based on acoustic signature, including:

[0153] The data acquisition module is used to acquire the raw audio data of the three blades of the wind turbine respectively, and to preprocess the data to convert the three raw audio data into three Mel spectrograms respectively. This module is used to implement the function of step S100 in Embodiment 1, which will not be described in detail here.

[0154] The voiceprint feature extraction module is used to establish a coarse extraction model based on a convolutional neural network. It is used to input the Mel spectrogram, perform convolution operations on the Mel spectrogram, and extract several first feature signals containing voiceprint features from the Mel spectrogram. This module is used to implement the function of step S200 in Embodiment 1, which will not be described in detail here.

[0155] The blade acoustic signature extraction module is used to establish a blade acoustic signature capture model based on a long short-term memory neural network. It is used to receive each first feature signal, determine which first feature signal contains the sound of the wind turbine blade, and output the first feature signal containing the sound of the wind turbine blade as the second feature signal. This module is used to implement the function of step S300 in embodiment one, which will not be described in detail here.

[0156] The noise reduction module is used to calculate the signal-to-noise ratio of the second feature signal and remove noise signals from the second feature signal based on the calculated signal-to-noise ratio. This module is used to implement the function of step S400 in Embodiment 1, and will not be described in detail here.

[0157] The feature distribution calculation module is used to acquire three second feature signals after removing the noise signal, and calculate the feature distribution of each second feature signal respectively; this module is used to implement the function of step S500 in Embodiment 1, and will not be described in detail here;

[0158] The anomaly detection module is used to calculate the Fréchet distance between each pair of the calculated second feature signals based on the feature distribution of each second feature signal, and to determine whether the wind turbine blades have an anomaly based on the calculated Fréchet distance; this module is used to implement the function of step S600 in Embodiment 1, and will not be described in detail here.

[0159] Example 3:

[0160] This embodiment proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the wind turbine blade anomaly detection method based on acoustic print as described in any embodiment of the present invention.

[0161] Example 3:

[0162] This embodiment proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the wind turbine blade anomaly detection method based on acoustic print as described in any embodiment of the present invention.

[0163] In this application embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent the existence of A alone, A and B simultaneously, or B alone. A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of singular or plural items. For example, at least one of a, b, and c can represent: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, and c can be single or multiple.

[0164] Those skilled in the art will recognize that the units and algorithm steps described in the embodiments disclosed herein can be implemented using electronic hardware, computer software, or a combination of electronic hardware and software. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0165] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0166] In the several embodiments provided in this application, any function, if implemented as a software functional unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0167] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A method of fan blade anomaly detection based on voiceprint, characterized in that, The method comprises the following steps: Collecting original audio data of three blades of the fan respectively, and preprocessing, converting the collected three original audio data into three mel spectrograms respectively; A rough extraction model based on a convolutional neural network is established, which is used to input the mel spectrogram, and the mel spectrogram is subjected to convolution operation, and a plurality of first feature signals containing voiceprint features are extracted from the mel spectrogram; A blade voiceprint capturing model based on a long short-term memory neural network is established, which is used to receive each first feature signal, determine which first feature signal contains fan blade sound, and output the first feature signal containing fan blade sound as a second feature signal; The signal-to-noise ratio of the second feature signal is calculated, and the noise signal in the second feature signal is removed according to the calculated signal-to-noise ratio; Three second feature signals after removing the noise signal are obtained, and the feature distribution of each second feature signal is calculated respectively; According to the calculated feature distribution of each second feature signal, the Fréchet distance between each second feature signal is calculated, and whether the fan blade is abnormal is judged according to the calculated Fréchet distance; In the blade voiceprint capturing model, an attention module is inserted in the long short-term memory neural network, which is used to calculate the attention weight of the first feature signal containing the fan blade sound, and the specific formula is as follows: In the formula, attention i is the attention weight of the i-th first feature signal, a, W x and W h are learning parameters, a T is the transpose matrix of a, and tanh() is the hyperbolic tangent function. h is the internal state vector of the long short-term memory neural network. The specific method of calculating the signal-to-noise ratio of the second feature signal and removing the noise signal in the second feature signal according to the calculated signal-to-noise ratio is as follows: The second feature signal is obtained, and the prior signal-to-noise ratio and the posterior signal-to-noise ratio of the second feature signal are calculated according to the following formula: ξ k = 0.98 * ξ k,old + 0.02 * max(γ k - 1, 0) wherein γ k is the estimated posterior signal-to-noise ratio of the kth frequency component in the second feature signal, is the kth frequency component, λ d (k) is the kth frequency power spectrum of the corresponding noise signal; ξ k is the estimated prior signal-to-noise ratio of the kth frequency component in the second feature signal, ξ k,old denotes the prior signal-to-noise ratio estimated in the previous step, the value of the first prior signal-to-noise ratio is initialized to 0; The spectral gain is calculated: wherein gain k is the spectral gain of the kth spectral component; The kth frequency component in the second feature signal is multiplied by the corresponding spectral gain to eliminate the noise signal, and the specific method is as follows: wherein is the kth frequency component of the noise signal; Based on the frequency components of the noise signal, the second feature signal after removing the noise signal is obtained.

2. A method of fan blade anomaly detection based on voiceprint according to claim 1, characterized in that, The specific steps of obtaining the original audio data corresponding to the three blades respectively, and preprocessing, and converting the collected three original audio data into three mel spectrograms are as follows: For any original audio data, short-time Fourier transform is performed, the sound signal in the original audio data at a specified sampling frequency is calculated, and the power spectrum is obtained through the calculated sound signal, and then the power spectrum is converted into a mel spectrogram.

3. The method of claim 1, wherein the method further comprises: The rough extraction model comprises: A selection layer is used to select n adjacent voiceprint feature vectors from the input mel spectrogram; A convolution layer comprises a plurality of convolution operations and Relu activation functions, which is used to perform convolution operation on the selected n voiceprint feature vectors, and the specific formula is as follows: y i,j = max(x i *w j , 0) In the formula, x i is the i-th voiceprint feature vector, w j is the j-th convolution operation with a convolution kernel size of 1x1, y i,j is the input x i and w j perform a convolution operation to extract the first convolution feature map; A feature fusion layer comprises a plurality of convolution operations, residual connection blocks and Relu activation functions, which is used to perform convolution operation on the first convolution feature map to obtain features, and perform feature fusion on the first convolution feature map and the obtained features, and the specific formula is as follows: z i,l = max(max(y i,j * f j,k , 0) * w k,l , 0) + y i,j wherein f j,k is the kth convolution operation corresponding to w j and the convolution kernel size is 5x1, w k,l is the lth filter corresponding to f j,k and the convolution kernel size is 5x1; and z i,l is the operation result. For z i,l Convolution operation is performed, and the second convolution feature map with the same size as x i is output, specifically as follows: v i = z i,l * w m In the formula, v i is a second convolution feature map, w m is a convolution operation with a kernel size of 1x1, so that v i has the same size as x i . A feature improvement layer comprises a sigmoid activation function, which is used to calculate the first feature signal based on the second convolution feature map, and the specific formula is as follows: In the formula, v i ’ is the ith voiceprint feature vector x i The corresponding first feature signal e is a natural constant.

4. The method of claim 1, wherein the method further comprises: The method of obtaining three second feature signals after removing the noise signal and calculating the feature distribution of each second feature signal is as follows: For any second feature signal, feature extraction is performed through convolution operation to obtain m n-dimensional feature vectors Z, the mean value of each dimension of the m feature vectors Z is obtained to obtain an n-dimensional vector μ, and the n*n order feature covariance matrix is calculated through the m n-dimensional feature vectors, and the n-dimensional vector μ and the feature covariance matrix are taken as the feature distribution of the corresponding second feature signal.

5. A method of fan blade anomaly detection based on voiceprint according to claim 4, characterized in that, The method for calculating the Fréchet distance between each two second feature signals according to the calculated feature distribution of each second feature signal, and judging whether the fan blade is abnormal according to the calculated Fréchet distance is specifically: The Fréchet distance between the feature distributions of any two second feature signals is calculated according to the following formula: where μ i and μ j are n-dimensional vectors of the first and second characteristic signals, respectively, ∑ i and ∑ j are the characteristic covariance matrices of the first and second characteristic signals, respectively, and Tr denotes the sum of the elements on the diagonal of a matrix. According to the calculated Fréchet distance, the difference between the two second feature signals is measured, the smaller the calculated Fréchet distance is, the closer the two second feature signals are, a threshold value is set, if the difference between the Fréchet distances of the second feature signals corresponding to any two fan blades is less than the threshold value, the fan blade is normal, and if the difference between the Fréchet distances of the second feature signals corresponding to any two fan blades is greater than the threshold value, the fan blade is abnormal.

6. A voiceprint-based fan blade anomaly detection system, characterized in that, Comprise: A data acquisition module is configured to acquire original audio data of three blades of a fan respectively, and to preprocess the original audio data, and to convert the acquired three original audio data into three mel spectrograms respectively; A voiceprint feature extraction module is configured to establish a coarse extraction model based on a convolutional neural network, to input the mel spectrogram, to perform convolution operation on the mel spectrogram, and to extract a plurality of first feature signals containing voiceprint features from the mel spectrogram; A blade voiceprint extraction module is configured to establish a blade voiceprint capture model based on a long short-term memory neural network, to receive each first feature signal, to determine which first feature signal contains fan blade sound, and to output the first feature signal containing fan blade sound as a second feature signal; A denoising module is configured to calculate the signal-to-noise ratio of the second feature signal, and to remove noise signals in the second feature signal according to the calculated signal-to-noise ratio; A feature distribution calculation module is configured to obtain three second feature signals after removing noise signals, and to calculate the feature distribution of each second feature signal respectively; An abnormality detection module is configured to calculate the Fréchet distance between each two second feature signals according to the calculated feature distribution of each second feature signal, and to judge whether the fan blade is abnormal according to the calculated Fréchet distance. In the blade voiceprint capture model, an attention module is inserted in the long short-term memory neural network to calculate the attention weight of the first feature signal containing the fan blade sound, and the specific formula is as follows: In the formula, attention i is the attention weight of the i-th first feature signal, a, W x and W h are learning parameters, a T is the transpose matrix of a, and tanh() is the hyperbolic tangent function. h is the internal state vector of the long short-term memory neural network. The specific method for calculating the signal-to-noise ratio of the second feature signal and removing noise signals in the second feature signal according to the calculated signal-to-noise ratio is as follows: The prior signal-to-noise ratio and the posterior signal-to-noise ratio of the second feature signal are calculated according to the following formula: ξ k = 0.98 * ξ l,old + 0.02 * max(γ k - 1, 0) wherein γ k is the estimated posterior signal-to-noise ratio of the kth frequency component in the second feature signal, is the kth frequency component, λ d (k) is the kth frequency power spectrum of the corresponding noise signal; ξ k is the estimated prior signal-to-noise ratio of the kth frequency component in the second feature signal, ξ k,old denotes the prior signal-to-noise ratio estimated in the previous step, the value of the first prior signal-to-noise ratio is initialized to 0; The spectral gain is calculated: wherein gain k is the spectral gain of the kth spectral component; The kth frequency component in the second feature signal is multiplied by the corresponding spectral gain to eliminate noise signals, and the specific formula is as follows: wherein is the kth frequency component of the noise signal; The second characteristic signal after removing the noise signal is obtained based on eliminating each frequency component of the noise signal.

7. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the voiceprint-based fan blade anomaly detection method according to any one of claims 1-5 when executing the program.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the voiceprint-based fan blade anomaly detection method according to any one of claims 1-5.

Images