A classification-based method for detecting wind turbine blade audio faults

Abstract

The present invention provides a classification-based wind turbine blade audio fault detection method to solve the problems of high detection cost, difficult installation, complex structure, long deployment time and susceptibility to environmental influence in the existing wind turbine blade monitoring technology. The method comprises the steps of 1) obtaining an open source audio data set and a blade audio data set, wherein the blade audio data set comprises a training set and a test set; 2) extracting the Mel spectrum, Mel frequency cepstral coefficient and chromaticity feature of each audio data in the training set; 3) constructing an audio fault detection classification network model, sending the open source audio data set into the network model for pre-training, and then sending the Mel spectrum, Mel frequency cepstral coefficient and chromaticity feature of the audio data in the training set into the model for classification detection training, and using the test set to test the audio fault detection classification network model; 4) inputting the blade audio to be tested into the audio fault detection classification network model to obtain the classification detection result of the blade audio to be tested.

Description

Technical Field

[0001] The invention belongs to the field of wind power generation, and in particular relates to a classification-based wind turbine blade audio fault detection method. Background Art

[0002] With the demand for new energy development, wind energy has received much attention, and wind power generation has become one of the most effective ways to transform and use wind energy. The continuous development of large-scale equipment manufacturing and wind power technology has made wind turbines continue to develop in the direction of large-scale and marine, and the number of installations has increased year by year. Blades are an indispensable core component of wind turbine equipment. Monitoring them can greatly reduce the occurrence rate of failures and accidents, ensure the stable operation and safety of wind turbines, and thus avoid unnecessary economic losses. Therefore, how to monitor the operating status and fault category of blades is an urgent problem to be solved.

[0003] At present, the monitoring of wind turbine blades mainly includes two categories: visual monitoring and auditory monitoring.

[0004] As a mainstream method, visual monitoring can be divided into vibration detection technology, acoustic emission detection technology, fiber Bragg grating detection technology, machine vision detection technology and infrared thermal imaging technology. Among them, with the complexity of blade design and the development of giant blade length, the selection and installation of sensors, multi-sensor optimization fusion and wiring problems, and the accurate separation of blade natural frequency and vibration mode have become key factors restricting the development of vibration detection technology; acoustic emission detection technology is difficult to eliminate the interference of electromechanical noise, which makes it difficult to separate stress waves and clutter, and it is irreversible. If it is missed, it is difficult to achieve the monitoring effect; fiber Bragg grating detection technology has high detection cost and is not suitable for large-scale wind farm deployment and installation; the ambient light and background information of the wind farm will affect the detection effect and accuracy based on machine vision detection technology, and the existing detection method based on deep convolutional network is relatively complex, which consumes a lot of time and occupies a lot of hardware resources in actual engineering application deployment; infrared thermal imaging technology has high accuracy and sensitivity in temperature sensing and is easily affected by the external environment.

[0005] Auditory monitoring mainly includes audio detection technology. Audio detection technology is an important branch of modern computer hearing discipline. The detected audio signal has rich information. The non-contact audio signal has unique advantages. It avoids the difficulty of vibration signal data collection. At the same time, the use of audio fault detection technology can achieve the purpose of low-cost, low-consumption, high-efficiency and fast non-destructive monitoring of wind turbine blades. However, audio signals are easily affected by the environment, which in turn affects the accuracy of the detection results. Summary of the invention

[0006] In order to solve the technical problems of existing wind turbine blade monitoring technology, such as high detection cost, difficult installation, complex structure, long deployment time, and susceptibility to environmental influences that affect detection accuracy, the present invention provides a classification-based wind turbine blade audio fault detection method.

[0007] To achieve the above purpose, the technical solution provided by the present invention is:

[0008] A classification-based wind turbine blade audio fault detection method includes the following steps:

[0009] 1) Data Acquisition

[0010] 1.1) Obtain open source audio datasets;

[0011] 1.2) Classify and organize the audio data of the leaves to obtain a leaf audio data set, which includes a training set and a test set;

[0012] 2) Feature extraction of training set audio data

[0013] Extract the Mel spectrum, Mel frequency cepstral coefficients and chromaticity features of each audio data in the training set in step 1.2);

[0014] 3) Establishing audio fault detection classification network model

[0015] 3.1) Constructing an audio fault detection and classification network model, and sending the open source audio data set in step 1.1) into the audio fault detection and classification network model for pre-training to obtain a basic pre-training model;

[0016] 3.2) The Mel spectrum, Mel frequency cepstral coefficient and chromaticity feature of each audio data in the training set of step 2) are sent to the basic pre-training model of step 3.1) for classification detection training to obtain a trained audio fault detection classification network model;

[0017] 3.3) extracting the Mel spectrum, Mel frequency cepstral coefficient and chromaticity features of each audio data in the test set of step 1.2), and sending them to the audio fault detection classification network model trained in step 3.2) for testing, and statistically analyzing the test results;

[0018] If the accuracy of the test result is higher than the given threshold, the audio fault detection classification network model is established;

[0019] If the accuracy of the test result is lower than the given threshold, the condition parameters of the classification network model are adjusted until the statistical test result meets the requirements and the audio fault detection classification network model is established;

[0020] 4) Detection of the audio frequency of the blade to be tested

[0021] The audio of the blade to be tested is input into the audio fault detection classification network model established in step 3.3) to obtain the classification detection result of the audio of the blade to be tested.

[0022] Further, in step 1.2), the leaf audio dataset also includes a validation set;

[0023] Between step 3) and step 4), step A) verifying the network model is also included: extracting the Mel spectrum, Mel frequency cepstral coefficient and chromaticity features of each audio data in the verification set in step 1.2), and sending them to the audio fault detection classification network model established in step 3.3) for verification, so as to obtain the fault detection results of the audio data in the verification set;

[0024] At the same time, the corresponding Mel spectrum graph is drawn according to the Mel spectrum of each audio data in the validation set, and the actual classification result of each audio data is obtained according to the Mel spectrum graph, so as to obtain the audio data classification result under the Mel spectrum graph;

[0025] The fault detection results of the validation set audio data are compared with the audio data classification results obtained by the Mel-spectrogram to verify the effectiveness of the audio fault detection and classification network model.

[0026] Furthermore, in step 2) and step 3.3), the Mel spectrum features of the audio data are extracted as follows:

[0027] The audio data is transformed into a Mel spectrum Mel(f) through a Mel-scale filter bank to obtain the Mel spectrum features of the audio data;

[0028]

[0029] Wherein, f represents the frequency of the audio data.

[0030] Furthermore, in step 2) and step 3.3), the Mel-frequency cepstral coefficient features of the audio data are extracted as follows:

[0031] a) The audio data is pre-emphasized through a high-pass filter, where the high-pass filter calculation formula is as follows:

[0032] H(z)=1-μz -1

[0033] Where: μ is the adjustment parameter, and the value range of μ is 0.9-1.0; z is the frequency of the audio data in the z domain;

[0034] b) processing the pre-emphasized audio data in different regions according to the set sampling points and the set sampling frequency, and recording each region as a frame;

[0035] c) Multiply each frame of audio signal by the Hamming window, and then obtain the energy distribution on the spectrum through fast Fourier transform FFT. The energy spectrum of audio data is expressed as follows:

[0036]

[0037] Where: x(n) is the input audio signal, N is the number of Fourier transform points, j is the complex unit, 0≤k≤N;

[0038] d) The energy spectrum of the audio data is processed by a triangular filter bank to obtain the logarithmic energy s(m):

[0039]

[0040] Where, 0≤m≤M, M is the number of filters in the triangular filter bank;

[0041] H m (k) is the triangular filter frequency of the triangular filter bank, and the calculation formula is as follows:

[0042]

[0043] e) Perform discrete cosine transform on the digital energy of the audio data to obtain the Mel-frequency cepstrum coefficient C(n):

[0044]

[0045] Wherein: L is the order of Mel frequency cepstrum coefficients, which is 12-16, M is the number of triangular filters in the triangular filter bank, n=1, 2, ..., L.

[0046] Further, it is characterized in that step 3.1) is specifically:

[0047] 3.1.1) The open source audio dataset is fed into three convolution-BN-pooling structures with the same structure through the input layer of the audio fault detection and classification network model to extract audio features in parallel;

[0048] 3.1.2) The audio features output by the three convolution-BN-pooling structures are added, and the regularization effect is achieved through Dropout. The Dense fully connected layer is then used to perform global processing of local information. Finally, the fully connected layer Dense is used to reduce the output channels to the required number of categories to achieve audio data classification.

[0049] Furthermore, step 3.2) is specifically as follows: take the average, maximum and minimum values ​​of the Mel spectrum m, Mel frequency cepstral coefficient n and chrominance l features of each audio data in the training set of step 2), and use the splicing method to obtain the features of (m+n+l, 3), and then send them into the basic pre-training model of step 3.1) for classification detection training to obtain a trained audio fault detection classification network model.

[0050] Further, step 1.1) is specifically as follows: download a standard speech classification task dataset, which contains 10 types of speech, namely, air conditioner, car horn, children playing, dog barking, drilling, engine idling, gun shooting, handheld rock drill, police siren, and street music, extract the two largest types of data and mark them as normal data and abnormal data, or divide the 10 types of data into two major types of data, respectively marked as normal data and abnormal data, to obtain an open source audio dataset;

[0051] In step 1.2), the blade audio dataset is a blade audio dataset containing two categories: normal and fault.

[0052] Further, step 1.1) can also be specifically as follows: download a standard speech classification task dataset, which contains 10 kinds of speech, namely, air conditioner, car horn, children playing, dog barking, drilling, engine idling, gun shooting, handheld rock drill, police siren, and street music, extract the M types of data with the largest number and mark them as M types of audio category data with identification, or divide the 10 types of data into M types of audio category data with identification, to obtain an open source audio dataset, wherein M is an integer greater than 2 and less than or equal to 10;

[0053] In step 1.2), the leaf audio dataset is a leaf audio dataset containing M types of audio data.

[0054] Compared with the prior art, the advantages of the present invention are:

[0055] 1. The data acquisition in the method of the present invention includes obtaining an open source audio data set, and pre-training the classification network model through the open source audio data set. Without the need for a massive blade audio database, an audio fault detection classification network model with higher accuracy and faster speed on the verification set and the test set can be obtained.

[0056] 2. The method of the present invention also includes drawing a Mel spectrum graph through the Mel spectrum of the audio data, and obtaining the audio data classification result through the Mel spectrum graph, and comparing it with the fault detection result of the corresponding audio data output by the audio fault detection classification network model to verify the effectiveness of the audio fault detection classification network model.

[0057] 3. Both the open source audio dataset and the leaf audio dataset of the present invention can realize binary classification of audio data, and thus realize the detection of normal and abnormal leaves.

[0058] 4. The open source audio data set and blade audio data set of the present invention can both realize multi-classification of audio data. The audio fault detection classification network model shows good generalization ability. When the types of audio faults increase, more types of classification tasks can be implemented according to the data type, thereby realizing detection and early warning of significant abnormalities such as blade breakage, cracking, and lightning strikes. It can greatly improve the real-time and accuracy of wind farm unit fault detection, ensure the reliable operation of the unit, and provide a theoretical basis and technical support for wind farm unit blade fault analysis and diagnosis, so as to realize the low-cost, efficient and rapid deployment of wind farm blade intelligent audio detection system, and finally achieve intelligent and efficient wind turbine blade monitoring effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 It is a flow chart of the wind turbine blade audio fault detection method based on classification of the present invention;

[0060] Figure 2 It is a structural schematic diagram of a classification network model in a classification-based wind turbine blade audio fault detection method of the present invention;

[0061] Figure 3 It is a flow chart of Mel cepstral coefficient extraction in the classification-based wind turbine blade audio fault detection method of the present invention;

[0062] Figure 4 is a Mel-spectrogram of normal audio data in Embodiment 2 of the present invention, wherein a is a spectrum diagram of the original audio signal, and b is a Mel-spectrogram;

[0063] Figure 5 is a Mel spectrum diagram of the lightning strike audio data in the second embodiment of the present invention, wherein a is the spectrum diagram of the original audio signal, and b is the Mel spectrum diagram;

[0064] Figure 6 It is a Mel-spectrogram of the pigeon tail whistle audio data in the second embodiment of the present invention, wherein a is the original audio signal spectrum diagram and b is the Mel-spectrogram. DETAILED DESCRIPTION

[0065] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0066] Based on deep learning and computer auditory technology, the present invention takes the audio data of blades as the object and proposes a fast and accurate method for detecting the fault type of wind turbine blades, thereby improving the real-time and accuracy of wind farm unit fault detection, reducing the maintenance and overhaul costs of wind turbines, ensuring the reliable operation of wind turbines, and providing a theoretical basis and technical support for the analysis and diagnosis of wind farm unit blade faults, so as to realize the low-cost, efficient and rapid deployment of the wind farm blade intelligent audio detection system.

[0067] The present invention provides a classification-based wind turbine blade audio fault detection method, which first uses an open source data set to train a pre-trained model, and then uses the pre-trained model to train on the blade audio data set to obtain a model for audio fault monitoring. Figure 1 As shown, it is a flow chart of the wind turbine blade audio fault detection method based on classification of the present invention. The method of the present invention is described in detail below. The specific steps are as follows:

[0068] 1) Data Acquisition

[0069] 1.1) Obtain open source audio datasets;

[0070] 1.2) Classify and organize the audio data of the leaves to obtain a leaf audio data set, which includes a training set, a test set, and a validation set. The ratio of the training set, the validation set, and the test set is 7:2:1;

[0071] 2) Feature extraction of training set audio data

[0072] Extract the Mel spectrum, Mel frequency cepstral coefficients and chromaticity features of each audio data in the training set in step 1.2);

[0073] 3) Establishing audio fault detection classification network model

[0074] 3.1) The open source audio dataset in step 1.1) is fed into the classification network model for pre-training to obtain a basic pre-training model;

[0075] The classification network model structure of the present invention is as follows Figure 2 As shown in the figure, the input layer is sent to three convolution blocks with the same structure, so that audio features can be extracted in parallel during the processing, which speeds up the feature extraction rate; the output features of the convolution blocks are fused to increase the audio feature information and improve the classification accuracy; two parallel maximum poolings are performed in a single convolution block, and the pooled features are fused, which reduces the operating parameters while avoiding the problem of information loss caused by the pooling operation. The network structure is simple and has good accuracy and rate advantages for wind turbine blade audio classification tasks.

[0076] 3.2) The Mel spectrum, Mel frequency cepstral coefficient and chromaticity feature of each audio data in the training set of step 2) are sent to the basic pre-training model of step 3.1) for classification detection training to obtain a trained audio fault detection classification network model;

[0077] 3.3) extracting the Mel spectrum, Mel frequency cepstral coefficient and chromaticity features of each audio data in the test set of step 1.2), and sending them to the audio fault detection classification network model of step 3.2) for testing, and statistically analyzing the test results;

[0078] If the accuracy of the test result is higher than the given threshold, the audio fault detection classification network model is established;

[0079] If the accuracy of the test result is lower than the given threshold, the conditional parameters of the classification network model are adjusted until the statistical test result meets the requirements;

[0080] 3) Verify the network model

[0081] Extract the Mel spectrum, Mel frequency cepstral coefficient and chromaticity features of each audio data in the verification set in step 1.2), and send them to the audio fault detection classification network model in step 3.3) for verification to obtain the fault detection results of the audio data in the verification set;

[0082] At the same time, the corresponding Mel spectrum graph is drawn according to the Mel spectrum of each audio data in the verification set, and then the actual classification result of each audio data is obtained according to the Mel spectrum graph, so as to obtain the audio data classification result obtained by the Mel spectrum graph in the verification set;

[0083] Compare the fault detection results of the validation set audio data with the audio data classification results obtained from the Mel-spectrogram to verify the effectiveness of the audio fault detection classification network model;

[0084] 4) Detection of the audio frequency of the blade to be tested

[0085] The audio of the blade to be tested is input into the audio fault detection classification network model in step 3.2) to obtain the fault detection result of the audio of the blade to be tested.

[0086] The method of the present invention has the following characteristics:

[0087] (1) Without the need for massive amounts of blade audio data, an audio fault detection classification network model with high accuracy and high speed can be obtained on the validation set and test set;

[0088] (2) A classification network model for wind farm blade audio detection is constructed, and the binary classification and multi-classification tasks of audio data are realized through the model. The audio fault detection classification network model of the present invention shows good generalization ability. When the types of audio faults increase, more types of classification tasks can be realized according to the data type;

[0089] (3) The present invention combines the Mel spectrum of audio features with the audio fault detection classification network model detection method. By statistically analyzing the Mel spectrum graphs of different types of audio data, the model detection results are assisted in the analysis to verify the accuracy of the audio fault detection classification network model of the present invention. The Mel spectrum and the corresponding classified audio detection results can be statistically analyzed, laying a theoretical and research foundation for intuitively reflecting the blade status in the audio through audio features.

[0090] (4) The present invention establishes an audio fault detection and classification network model based on blade audio data to detect and warn of significant abnormalities such as blade breakage, cracking, and lightning strikes. This can greatly improve the real-time and accuracy of wind farm unit fault detection, ensure the reliable, efficient and safe operation of the unit, and achieve intelligent and efficient wind turbine blade monitoring effects, which has important practical significance and application value.

[0091] (5) The method of the present invention extracts and processes audio data features to detect and distinguish blade fault types, realize intelligent monitoring of blades of each unit in the wind farm, and ensure the operating status and safety of blades of the entire wind farm. The method of the present invention has a faster processing rate and detection accuracy, and shows good advantages in various aspects of performance. It has a broad application space and has the potential to expand market share.

[0092] (6) The method of the present invention not only effectively balances the accuracy and timeliness of audio fault classification, thereby reducing wind turbine unit failures caused by blade failures and ensuring reliable operation of the unit; but also multi-classification audio-based fault detection does not require the construction of a large number of blade audio fault databases, which greatly reduces costs and has good scalability for current large-scale engineering equipment operation fault detection systems. This method can be applied to non-destructive testing of a variety of engineering equipment.

[0093] The present invention collects the audio data of the blades at regular intervals through a sound pickup device installed at the bottom of the tower, analyzes and processes the blade audio data, and establishes a blade audio fault detection classification network model based on multiple classifications. If there are fewer valid audio data of blade failure, an early warning can be given for significant abnormalities such as blade breakage, cracking, and lightning strikes, without judging the type of blade abnormality, that is, only the input audio data is detected, and the audio data is output as normal (0) / abnormal (1), thereby realizing binary abnormality detection; if there are more valid audio data of blade failure, the data set can be divided according to the specific abnormal data type, and the model can be retrained to obtain multiple types of abnormal fault detection results, thereby realizing multi-classification abnormality detection.

[0094] Embodiment 1

[0095] Taking the blade audio detection of a wind turbine as an example, this embodiment provides a wind turbine blade audio fault detection method based on binary classification, comprising the following steps:

[0096] Step 1: Data acquisition and organization

[0097] Step 1.1: Obtain open source audio datasets and download the standard speech classification task dataset. The dataset contains 10 types of speech, namely air conditioners, car horns, children playing, dogs barking, drilling, engine idling, gun shooting, handheld rock drills, police sirens, and street music. Each recording is about 4 seconds long and is used for training the pre-training model. The two largest types of data are extracted to divide them into normal data and abnormal data, or the 10 types of data are divided into two major types of data and labeled as normal data and abnormal data respectively.

[0098] Step 1.2: Collection and organization of wind farm blade audio data sets. The sound pickup equipment installed at the bottom of the tower collects blade audio data at regular intervals. The collected blade audio materials are organized to obtain blade audio data sets of two categories, normal and faulty. Each audio segment is about 30 seconds long. There are 3896 normal blade audio segments and 2514 faulty blade audio segments in the currently organized data set. The blade audio data set is divided into training set, test set, and validation set.

[0099] Step 2: Feature extraction of training set audio data

[0100] Step 2.1: Get Mel spectrum features

[0101] The spectrogram is often a large image. In order to obtain the sound features of appropriate size, each audio data in the training set is often transformed into a Mel spectrum Mel(f) through a Mel-scaled filter bank to obtain the Mel spectrum features of each audio data. Mel spectrum has a long-standing reputation in artificial features. In this method, the m-dimensional Mel spectrum features are extracted. The perceptual frequency domain formula in Mel is shown in (1), and the actual speech frequency formula in Hertz (Hz) is shown in (2). In the Mel frequency domain, the perception of pitch is a linear relationship, as follows:

[0102]

[0103] f=700(10 Mel(f) / 2595 -1) (2)

[0104] Step 2.2: Get Mel frequency cepstral coefficient features

[0105] The cepstral coefficients obtained from the mel-frequency spectrum are called mel-frequency cepstral coefficients (MFCC). The method of this embodiment extracts n-dimensional mel-frequency cepstral coefficient features. The specific speech feature parameter MFCC extraction process is as follows: Figure 3 In order to enhance the high frequency part and make the spectrum of the signal flat, pre-emphasis, framing and windowing pre-processing are performed. Pre-emphasis processing is actually passing the speech signal through a high-pass filter, framing is to process the audio in different areas through certain sampling points and sampling frequencies, and windowing is to multiply each frame by a Hamming window.

[0106] 2.2.1) Each audio data is pre-emphasized through a high-pass filter. The high-pass filter calculation formula is as follows:

[0107] H(z)=1-μz -1 (3)

[0108] Wherein: the value of μ is between 0.9 and 1.0, and is 0.97 in this embodiment; z is the frequency of the audio data in the z domain;

[0109] 2.2.2) Each audio data after pre-emphasis processing is processed in different regions according to the set sampling point and the set sampling frequency, and each region is recorded as a frame;

[0110] 2.2.3) Multiply the audio signal of each frame by the Hamming window to increase the continuity of the left and right ends of the frame. For example, the signal after framing is S(n), n = 0, 1, ..., N-1, N is the frame size, and after multiplying by the Hamming window, S'(n) = S(n) × W(n), W(n) is in the form of formula (4), where a is 0.46:

[0111]

[0112] After the audio signal of each frame is multiplied by the Hamming window, the energy distribution on the spectrum is obtained by fast Fourier transform FFT. The spectrum of each frame after framing and windowing is obtained by fast Fourier transform, and the spectrum of the audio signal is squared to obtain the power spectrum of the speech signal. The FFT of the audio signal is:

[0113]

[0114] Where x(n) is the input speech signal, N is the number of Fourier transform points, j is the complex unit, 0≤k≤N;

[0115] 2.2.4) The energy spectrum of each audio data is processed by a set of Mel-scale triangular filter banks, and a filter bank with M filters is defined (the number of filters is close to the number of critical bands). The filter used is a triangular filter, and its frequency response is defined as:

[0116]

[0117] Where: The spectrum is smoothed by Mel filtering, and the effect of harmonics is eliminated, highlighting the resonance peak of the original speech. The logarithmic energy output by each triangular filter group is:

[0118]

[0119] Then the Mel frequency cepstrum (MFCC) coefficients are obtained through discrete cosine transform (DCT):

[0120]

[0121] The above is the L-order parameter, where L refers to the order of MFCC coefficients, usually 12-16, and M is the number of triangular filters. The inverse spectrum is the spectrum obtained by performing Fourier transform of the audio signal after logarithmic operation and then inverse Fourier transform.

[0122] Step 2.3: Get chromaticity information

[0123] Chroma is an interesting and powerful representation of audio, which projects the entire spectrum into l intervals. This method extracts l-dimensional chroma features.

[0124] Step 3: Build an audio fault detection classification network model

[0125] Step 3.1: Get a basic pre-trained model

[0126] Use the open source audio data set divided in step 1.1 to train and learn the binary classification model to obtain a binary classification basic pre-training model; specifically, the binary classification network structure is as follows Figure 2 As shown in the figure, for the input scale of 180 and channel 3 data, it is sent to the convolution-BN-pooling structure of three identical network structure blocks, and the speech signal features are enhanced by three identical structure blocks. Then, the features of the three output 256 channels are added to generate 768 channel features. At the same time, Dropout is used to achieve the regularization effect, and then the Dense fully connected layer is used to perform global processing of local information. Among them, BN is used to effectively alleviate the occurrence of overfitting and speed up the network training speed. Finally, the fully connected layer Dense is used to reduce the output channels to 2, which is the number of categories for classification, 2 categories (normal data, abnormal data);

[0127] Step 3.2: Obtaining the Audio Fault Detection Classification Network Model

[0128] Based on the binary classification basic pre-training model, the Mel spectrum m, Mel frequency cepstral coefficient n, and chrominance l features extracted from each audio data in step 2 are averaged, maximum, and minimum respectively, and the feature (m+n+l, 3) is obtained by splicing, and the wind farm blade audio data set in step 1.2 is input to train the binary classification basic pre-training model to obtain the final binary classification audio fault detection classification network model (normal or faulty);

[0129] Step 3.3: Test the Audio Fault Detection Classification Network Model

[0130] According to the method of step 2), the Mel spectrum, Mel frequency cepstral coefficient and chromaticity feature of each audio data in the test set 1.2) (the audio data of the test set has the marked normal / abnormal audio data) are extracted, and sent to the binary classification audio fault detection classification network model of step 3.2) for testing, and the test results are counted; the loss, accuracy and rate (10 seconds audio data) of the test binary classification audio fault detection classification network model are shown in Table 1;

[0131] Table 1 Binary classification model test performance table

[0132]

[0133] In Table 1, the accuracy of the test results is higher than the given threshold, and the two-class audio fault detection classification network model is established; and it can be seen that the classification accuracy is high and the detection rate is fast;

[0134] 4) Verify the network model

[0135] According to the method of step 2), the Mel spectrum, Mel frequency cepstral coefficient and chromaticity features of each audio data in the verification set of 1.2) are extracted, and sent to the binary audio fault detection classification network model of step 3.2) for verification to obtain the fault detection results of the audio data in the verification set;

[0136] At the same time, a corresponding Mel spectrum graph is drawn according to the Mel spectrum of each test audio data in the validation set, and the actual classification result of the audio data is obtained according to the Mel spectrum graph, so as to obtain the audio data classification result obtained by the Mel spectrum graph;

[0137] Compare the fault detection results of the validation set audio data with the audio data classification results obtained by the Mel spectrum graph to verify the effectiveness of the audio fault detection classification network model. The results are fed back to the blade status monitoring platform for audio fault analysis. After multiple statistical analyses, the statistical results of the performance indicators of the binary audio fault detection classification network model are obtained, as shown in Table 2:

[0138] Table 2 Evaluation index table of binary classification model,

[0139]

[0140] 5) Detection of the audio frequency of the blade to be tested

[0141] The audio of the blade to be tested is input into the binary audio fault detection classification network model in step 3.2) to obtain the fault detection result of the audio of the blade to be tested, and the detection result is normal or faulty.

[0142] Embodiment 2

[0143] Taking the blade audio detection of a certain wind turbine as an example, this embodiment provides a wind turbine blade audio fault detection method based on multi-classification, including the following steps:

[0144] Step 1: Data acquisition and organization

[0145] Step 1.1: Obtain open source audio data sets, download the standard speech classification task data set, which contains 10 types of speech, namely air conditioners, car horns, children playing, dogs barking, drilling, engine idling, gun shooting, handheld rock drills, police sirens, and street music. Each recording is about 4 seconds long and is used for pre-training model training. The M types of data with the largest number are extracted and divided into audio category data with identification, or the 10 types of data are divided into M categories of data, and they are labeled and distinguished. Among them, M is an integer greater than 2, and significant abnormalities such as blade breakage, cracking, and lightning strikes are detected and warned;

[0146] Step 1.2: Collection and organization of wind farm blade audio data sets. The sound pickup equipment installed at the bottom of the tower collects blade audio data at regular intervals. The collected blade audio materials are organized to obtain blade audio data sets of M categories, including fracture, cracking, and lightning strikes. The currently organized data set contains 4496 normal blade audio segments, 1514 fractured segments, 1452 lightning struck segments, and 1221 cracked segments, which are summarized as M types of audio data. The blade audio data set is divided into training set, test set, and validation set.

[0147] Step 2: Feature extraction of training set audio data

[0148] The steps are the same as those in Example 1.

[0149] Step 3: Build an audio fault detection classification network model

[0150] Step 3.1: Get a basic pre-trained model

[0151] Use the open source audio data set divided in step 1.1 to train and learn a multi-classification model to obtain a multi-classification basic pre-training model. Specifically, the network structure sends the data of input scale 180 and channel 3 into the convolution-BN-pooling structure of three identical network structure blocks, and enhances the speech signal features through three identical structure blocks. Then, the features of the three output 256 channels are added to generate 768 channel features. At the same time, Dropout is used to achieve the regularization effect, and then the Dense fully connected layer is used to perform global processing of local information. BN is used to effectively alleviate the occurrence of overfitting and speed up the network training speed. Finally, the fully connected layer Dense is used to reduce the output channels to the number of categories M, which is also the number of categories for classification, M categories (fracture, cracking, lightning strike, etc.);

[0152] Step 3.2: Obtaining the Audio Fault Detection Classification Network Model

[0153] Based on the multi-classification basic pre-training model, the Mel spectrum m, Mel frequency cepstrum coefficient n, and chrominance l features extracted from each audio data in step 2 are averaged, maximum, and minimum respectively, and the feature (m+n+l, 3) is obtained by splicing. The wind farm blade audio data set in step 1.2 is used to train the multi-classification basic pre-training model to obtain the final multi-classification audio fault detection classification network model (normal, fracture, cracking, lightning strike, etc.)

[0154] Step 3.3: Test and obtain the audio fault detection classification network model

[0155] According to the method of step 2), the Mel spectrum, Mel frequency cepstrum coefficient and chromaticity characteristics of each audio data in the test set 1.2) (the audio data of the test set has audio data of identification types such as normal, broken, cracked, and lightning strike) are extracted, and sent to the multi-classification audio fault detection classification network model in step 3.2) for testing, and the test results are statistically analyzed; the loss, accuracy and rate (10 seconds of audio data) of the test multi-classification audio fault detection classification network model are shown in Table 3;

[0156] Table 3 Multi-classification model test performance table

[0157]

[0158]

[0159] In Table 3, the accuracy of the test results is higher than the given threshold, and the multi-classification audio fault detection classification network model is established. It can be seen that the classification accuracy is high and the detection rate is fast.

[0160] 4) Verify the network model

[0161] According to the method of step 2), the Mel spectrum, Mel frequency cepstral coefficient and chromaticity feature of each audio data in the verification set of 1.2) are extracted, and sent to the audio fault detection classification network model of step 3.2) for verification to obtain the fault detection result of the audio data in the verification set;

[0162] At the same time, the corresponding Mel spectrum graph is drawn according to the Mel spectrum of each test audio data in the validation set, such as Figures 4 to 6 As shown, the mel-spectrogram of normal, lightning strike, and pigeon tail whistle audio data; the actual classification result of the audio data is obtained according to the mel-spectrogram, and the classification result of the audio data obtained by the mel-spectrogram is obtained;

[0163] Compare the fault detection results of the validation set audio data with the audio data classification results obtained by the Mel spectrum graph to verify the effectiveness of the audio fault detection classification network model. The results are fed back to the blade status monitoring platform for audio fault analysis. After multiple statistical analyses, the statistical results of the performance indicators of the multi-classification audio fault detection classification network model are obtained, as shown in Table 4:

[0164] Table 4 Statistics of multi-classification model evaluation indicators

[0165]

[0166] 5) Detection of the audio frequency of the blade to be tested

[0167] The audio of the blade to be tested is input into the multi-classification audio fault detection classification network model in step 3.2) to obtain the fault detection result of the audio of the blade to be tested, and the detection result is normal or a fault type such as breakage, cracking, lightning strike, etc.

[0168] The above is only a description of the preferred implementation mode of the present invention, and the technical solution of the present invention is not limited thereto. Any modification made by those skilled in the art on the basis of the main technical concept of the present invention belongs to the technical scope to be protected by the present invention.

Claims

1. A classification-based wind turbine blade audio fault detection method, characterized in that: The following steps are involved: 1) Data Acquisition 1.1) Obtain open source audio datasets; 1.2) Classify and organize the audio data of the leaves to obtain a leaf audio data set, which includes a training set and a test set; 2) Feature extraction of training set audio data Extract the Mel spectrum, Mel frequency cepstral coefficients and chromaticity features of each audio data in the training set in step 1.2); 3) Establishing audio fault detection classification network model 3.1) Construct an audio fault detection classification network model, and send the open source audio data set in step 1.1) into the audio fault detection classification network model for pre-training to obtain a basic pre-training model; specifically: 3.1.1) The open source audio dataset is fed into three convolution-BN-pooling structures with the same structure through the input layer of the audio fault detection and classification network model to extract audio features in parallel; 3.1.2) The audio features output by the three convolution-BN-pooling structures are added, and the regularization effect is achieved through Dropout. The Dense fully connected layer is then used to perform global processing of local information. Finally, the fully connected layer Dense is used to reduce the output channels to the required number of categories to achieve audio data classification; 3.2) Take the average, maximum and minimum values ​​of the Mel spectrum m, Mel frequency cepstral coefficient n and chrominance l features of each audio data in the training set of step 2), and use the splicing method to obtain the feature (m+n+l, 3), and then send it to the basic pre-training model in step 3.1) for classification detection training to obtain the trained audio fault detection classification network model; 3.3) extracting the Mel spectrum, Mel frequency cepstral coefficient and chromaticity features of each audio data in the test set of step 1.2), and sending them to the audio fault detection classification network model trained in step 3.2) for testing, and statistically analyzing the test results; If the accuracy of the test result is higher than the given threshold, the audio fault detection classification network model is established; If the accuracy of the test result is lower than the given threshold, the condition parameters of the classification network model are adjusted until the statistical test result meets the requirements and the audio fault detection classification network model is established; 4) Detection of the audio frequency of the blade to be tested The audio of the blade to be tested is input into the audio fault detection classification network model established in step 3.3) to obtain the classification detection result of the audio of the blade to be tested.

2. The classification-based wind turbine blade audio fault detection method according to claim 1 is characterized in that: In step 1.2), the leaf audio dataset also includes a validation set; Between step 3) and step 4), step A) verifying the network model is also included: extracting the Mel spectrum, Mel frequency cepstral coefficient and chromaticity features of each audio data in the verification set in step 1.2), and sending them to the audio fault detection classification network model established in step 3.3) for verification, so as to obtain the fault detection results of the audio data in the verification set; At the same time, the corresponding Mel spectrum graph is drawn according to the Mel spectrum of each audio data in the validation set, and the actual classification result of each audio data is obtained according to the Mel spectrum graph, so as to obtain the audio data classification result under the Mel spectrum graph; The fault detection results of the validation set audio data are compared with the audio data classification results obtained by the Mel-spectrogram to verify the effectiveness of the audio fault detection and classification network model.

3. The classification-based wind turbine blade audio fault detection method according to claim 2 is characterized in that: In step 2) and step 3.3), the Mel spectrum features of the audio data are extracted as follows: The audio data is transformed into a Mel spectrum Mel(f) through a Mel-scale filter bank to obtain the Mel spectrum features of the audio data; Wherein, f represents the frequency of the audio data.

4. The classification-based wind turbine blade audio fault detection method according to claim 1 is characterized in that: Step 1.1) is specifically: Download the standard speech classification task dataset, which contains 10 types of speech, namely air conditioner, car horn, children playing, dog barking, drilling, engine idling, gun shooting, handheld rock drill, police siren, and street music. Extract the two largest types of data and label them as normal data and abnormal data, or divide the 10 types of data into two major types of data and label them as normal data and abnormal data respectively to obtain an open source audio dataset; In step 1.2), the blade audio dataset is a blade audio dataset containing two categories: normal and fault.

5. The classification-based wind turbine blade audio fault detection method according to claim 1, characterized in that: Step 1.1) is specifically: Download the standard speech classification task dataset, which contains 10 kinds of speech, namely air conditioner, car horn, children playing, dog barking, drilling, engine idling, gun shooting, handheld rock drill, police siren, and street music. Extract the M types of data with the largest number and mark them as M types of audio category data with identification, or divide the 10 types of data into M types of audio category data with identification to obtain an open source audio dataset, where M is an integer greater than 2 and less than or equal to 10; In step 1.2), the leaf audio dataset is a leaf audio dataset containing M types of audio data.

Images