Wind turbine generator gearbox fault discrimination method and device based on multi-data fusion

Abstract

The invention provides a wind turbine generator gearbox fault discrimination method and device based on multi-data fusion. The corresponding method comprises the following steps: converting acoustic data of a wind turbine generator gearbox into a two-dimensional image; analyzing the infrared thermal imaging data of the wind turbine generator gearbox into three-channel data; extracting feature data of the two-dimensional image and the three-channel data according to a pre-generated convolutional neural network; and according to the feature data and a pre-generated Transform model, judging the fault of the gear box of the wind turbine generator. According to the invention, a multi-modal information fusion method is used to combine with mutually associated fault features in multiple sensor signals, features with correlation are learned from the perspective of different modals, fault feature information is enriched, the accuracy and reliability of the fault diagnosis method are enhanced, and meanwhile, the characteristics of multiple sensors are combined, so that the fault diagnosis accuracy is improved. The failure problem of a certain sensor in an extreme environment can be prevented, and the robustness and stability of the wind turbine generator gearbox fault diagnosis method are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of wind turbine gearbox fault diagnosis, and in particular to a wind turbine gearbox fault diagnosis method and device based on multi-data fusion. Background Art

[0002] At present, with the large-scale and complex wind turbine equipment, fault diagnosis methods require richer and more comprehensive data to analyze the operating status and health of the equipment. However, in the face of complex diagnostic objects, a single sensor can no longer accurately reflect the operating information of wind turbine equipment, which leads to the inability of fault diagnosis methods to accurately identify faults through sensor data. Secondly, affected by the actual engineering environment, interference from different factors will also affect different sensors, resulting in serious signal noise, affecting the identification and extraction of effective fault information. Summary of the invention

[0003] The present invention provides a wind turbine gearbox fault diagnosis method and device based on multi-data fusion, which use a multimodal information fusion method to combine the interrelated fault features in multiple sensor signals, learn relevant features from different modal perspectives, enrich fault feature information, and enhance the accuracy and reliability of the fault diagnosis method. At the same time, combined with the characteristics of multiple sensors, it can prevent the failure of certain sensors in extreme environments and improve the robustness and stability of the wind turbine gearbox fault diagnosis method.

[0004] In order to achieve the above objectives, in the first aspect, this party provides a wind turbine gearbox fault identification method based on multi-data fusion, comprising:

[0005] Converting the acoustic data of the wind turbine gearbox into a two-dimensional image;

[0006] Parsing the infrared thermal imaging data of the wind turbine gearbox into three-channel data;

[0007] Extracting feature data of the two-dimensional image and the three-channel data according to a pre-generated convolutional neural network;

[0008] The wind turbine gearbox fault is determined based on the characteristic data and the pre-generated Transformer model.

[0009] In some embodiments of the present invention, converting the acoustic data of the wind turbine gearbox into a two-dimensional image includes:

[0010] Extracting frequency spectrum data of the acoustic data;

[0011] The spectrum data is converted into the two-dimensional image through a Markov transfer field; wherein the acoustic data is one-dimensional data, and the category of the Markov transfer field is non-hyperparameter.

[0012] In some embodiments of the present invention, a method for determining a wind turbine gearbox fault based on multi-data fusion further includes: the two-dimensional image is a Markov transfer field matrix;

[0013] A Markov transition field image is generated according to the Markov transition field matrix.

[0014] In some embodiments of the present invention, extracting feature data of the two-dimensional image and the three-channel data according to a pre-generated convolutional neural network includes:

[0015] Performing data-level fusion on the Markov transition field image and the three-channel data to generate four-channel fused data;

[0016] Inputting the four-channel fused data into the convolutional neural network to extract high-order image features of the two-dimensional image and high-order image features of the three-channel data;

[0017] The high-order image features of the two-dimensional image and the high-order image features of the three-channel data are fused at a feature level to generate the feature data.

[0018] In some embodiments of the present invention, the convolutional neural network includes: an input layer, a convolution layer, a pooling layer and a flattening layer; wherein the number of convolution layers is 3, and the number of pooling kernels of the pooling layer is equal to the number of convolution kernels of the convolution layer.

[0019] In a second aspect, the present invention provides a wind turbine gearbox fault identification device based on multi-data fusion, the device comprising:

[0020] An acoustic data conversion module, used for converting the acoustic data of the wind turbine gearbox into a two-dimensional image;

[0021] An infrared thermal imaging data parsing module, used for parsing the infrared thermal imaging data of the wind turbine gearbox into three-channel data;

[0022] A feature data extraction module, used to extract feature data of the two-dimensional image and the three-channel data according to a pre-generated convolutional neural network;

[0023] A fault identification module is used to identify the gearbox fault of the wind turbine generator set according to the characteristic data and a pre-generated Transformer model.

[0024] In some embodiments of the present invention, the acoustic data conversion module includes:

[0025] A spectrum data extraction unit, used for extracting spectrum data of the acoustic data;

[0026] A two-dimensional data generating unit is used to convert the spectrum data into the two-dimensional image through a Markov transfer field; wherein the acoustic data is one-dimensional data, and the category of the Markov transfer field is non-hyperparameter.

[0027] In some embodiments of the present invention, a wind turbine gearbox fault identification device based on multi-data fusion further includes: the two-dimensional image is a Markov transfer field matrix;

[0028] The Markov transition field image generation module is used to generate a Markov transition field image according to the Markov transition field matrix.

[0029] In some embodiments of the present invention, the feature data extraction module includes:

[0030] A fused data generating unit, used for performing data-level fusion on the Markov transition field image and the three-channel data to generate four-channel fused data;

[0031] A high-order image feature extraction unit, used for inputting the fused data of the four channels into the convolutional neural network to extract high-order image features of the two-dimensional image and high-order image features of the three-channel data;

[0032] The feature data extraction unit is used to perform feature-level fusion on the high-order image features of the two-dimensional image and the high-order image features of the three-channel data to generate the feature data.

[0033] In some embodiments of the present invention, the convolutional neural network includes: an input layer, a convolution layer, a pooling layer and a flattening layer; wherein the number of convolution layers is 3, and the number of pooling kernels of the pooling layer is equal to the number of convolution kernels of the convolution layer.

[0034] In a third aspect, the present invention provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of a method for determining a wind turbine gearbox fault based on multi-data fusion.

[0035] In a fourth aspect, the present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the processor implements the steps of a method for determining a wind turbine gearbox fault based on multi-data fusion.

[0036] In a fifth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a method for determining a wind turbine gearbox fault based on multi-data fusion.

[0037] From the above description, it can be seen that an embodiment of the present invention provides a method and device for determining a wind turbine gearbox fault based on multi-data fusion, and the corresponding method includes: first, converting the acoustic data of the wind turbine gearbox into a two-dimensional image; then, parsing the infrared thermal imaging data of the wind turbine gearbox into three-channel data; extracting feature data of the two-dimensional image and the three-channel data according to a pre-generated convolutional neural network; finally, determining the wind turbine gearbox fault according to the feature data and the pre-generated Transformer model.

[0038] The corresponding device includes: an acoustic data conversion module, which is used to convert the acoustic data of the wind turbine gearbox into a two-dimensional image; an infrared thermal imaging data parsing module, which is used to parse the infrared thermal imaging data of the wind turbine gearbox into three-channel data; a feature data extraction module, which is used to extract feature data of the two-dimensional image and the three-channel data according to a pre-generated convolutional neural network; and a fault discrimination module, which is used to discriminate the wind turbine gearbox fault according to the feature data and the pre-generated Transformer model.

[0039] In summary, the present invention uses a multimodal information fusion method to combine the interrelated fault features in various sensor signals, learns related features from different modal perspectives, enriches fault feature information, and enhances the accuracy and reliability of the fault diagnosis method. At the same time, combined with the characteristics of various sensors, it can prevent the failure of certain sensors in extreme environments and improve the robustness and stability of the fault diagnosis method. Therefore, the fault diagnosis method based on multimodal information fusion has become an effective method for solving practical engineering problems in the field of fault diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0041] Figure 1 A schematic diagram of a wind turbine gearbox fault diagnosis method based on multi-data fusion provided in an embodiment of the present invention Figure 1 ;

[0042] Figure 2 A schematic diagram of a flow chart of step 100 provided in an embodiment of the present invention;

[0043] Figure 3 A schematic diagram of a wind turbine gearbox fault diagnosis method based on multi-data fusion provided in an embodiment of the present invention Figure 2 ;

[0044] Figure 4 A flowchart of step 300 provided in an embodiment of the present invention;

[0045] Figure 5 A schematic flow chart of a method for determining a wind turbine gearbox fault based on multi-data fusion provided in a specific application example of the present invention;

[0046] Figure 6 A mind map of a wind turbine gearbox fault identification method based on multi-data fusion provided in a specific application example of the present invention;

[0047] Figure 7 This is a flow chart of step S1 provided in a specific application example of the present invention;

[0048] Figure 8 This is a flow chart of step S3 provided in a specific application example of the present invention;

[0049] Fig. 9 A schematic diagram of a wind turbine gearbox fault identification device based on multi-data fusion provided in an embodiment of the present invention Figure 1 ;

[0050] Fig.10 Schematic diagram of the structure of the acoustic data conversion module 10 provided in an embodiment of the present invention;

[0051] Fig.11 A schematic diagram of a wind turbine gearbox fault identification device based on multi-data fusion provided in an embodiment of the present invention Figure 2 ;

[0052] Fig.12 It is a structural schematic diagram of a feature data extraction module 30 provided in an embodiment of the present invention;

[0053] Fig.13 It is a schematic diagram of the structure of an electronic device in an embodiment of the present invention. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0056] It should be noted that the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or apparatuses.

[0057] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0058] Existing wind turbine gearbox fault diagnosis methods mostly use separate data preprocessing methods for different modal data, such as separately acquiring and processing infrared data, vibration signal data, and SCADA data. Although this approach technically simplifies the data processing process and reduces the complexity of the system, it actually ignores the potential correlation between different data sources, which may lead to an incomplete understanding of the overall system status, thereby affecting the final fault diagnosis and the formulation of preventive measures. For example, infrared thermal imaging data is more sensitive to non-structural damage such as overheating caused by decreased lubricant viscosity, while acoustic data is more sensitive to structural damage such as gear tooth breakage. It is difficult to obtain complete typical fault information with a single infrared thermal imaging data or acoustic data.

[0059] Based on the above reasons, the embodiment of the present invention provides a specific implementation of a wind turbine gearbox fault identification method based on multi-data fusion, see Figure 1 , the method specifically includes the following contents:

[0060] Step 100: converting the acoustic data of the wind turbine gearbox into a two-dimensional image;

[0061] Step 200: parsing the infrared thermal imaging data of the wind turbine gearbox into three-channel data;

[0062] Step 300: extracting feature data of the two-dimensional image and the three-channel data according to a pre-generated convolutional neural network;

[0063] Step 400: Identify the wind turbine gearbox fault according to the characteristic data and the pre-generated Transformer model.

[0064] From the above description, it can be seen that an embodiment of the present invention provides a wind turbine gearbox fault identification method based on multi-data fusion, including: first, converting the acoustic data of the wind turbine gearbox into a two-dimensional image; then, parsing the infrared thermal imaging data of the wind turbine gearbox into three-channel data; extracting feature data of the two-dimensional image and the three-channel data according to a pre-generated convolutional neural network; finally, identifying the wind turbine gearbox fault according to the feature data and the pre-generated Transformer model.

[0065] In view of the lack of effective modal characterization methods and effective feature fusion mechanisms in existing multi-sensor fusion methods, the present invention proposes a wind turbine gearbox fault diagnosis method based on the complementary fusion of infrared thermal imaging and acoustic data. The one-dimensional acoustic data is converted into image data using the Markov transfer field in the frequency domain, and the infrared thermal imaging data is decomposed into RGB three channels. The above four-channel data is used as the input of the feature extraction network. While performing feature preprocessing, the differences between different signal modes are reduced. The fusion of heterogeneous data is used to provide complementary information, thereby enhancing the accuracy of fault diagnosis.

[0066] For step 100, acoustic data is more sensitive to structural damage such as broken gear teeth of the wind turbine gearbox. Specifically, step 100 may include the following steps: First, preprocess the acoustic data to ensure the quality and stability of the signal. Including denoising, normalization and intercepting a specific time period. Select a window function (such as Hamming window, Hanning window, etc.) and a window length. Slide the window function to different positions of the signal and take out the signal fragments one by one. Apply Fourier transform to each signal fragment to obtain the spectrum of the fragment. Combine the spectra of all fragments to obtain a two-dimensional time-frequency image, which is the spectrogram. The horizontal axis of the spectrogram is time, the vertical axis is frequency, and the color or intensity represents the amplitude or energy of the signal at a specific time and frequency. Convert the calculated spectrogram into an image format, and use color mapping to represent different amplitudes or energy values. For example, a grayscale image or a pseudo-color image can be used to enhance the visualization effect.

[0067] For step 200, infrared thermal imaging data is more sensitive to non-structural damage such as overheating caused by decreased viscosity of the lubricating oil in the gearbox of the wind turbine. Step 200 specifically includes: Ian, read the infrared thermal imaging data. These data are usually stored in matrix form, and each pixel value represents temperature information. Some standard libraries such as OpenCV, NumPy, etc. can be used to read and process these data. In order to map the temperature data to the RGB color space, the temperature data needs to be normalized. Preferably, the temperature value range will be mapped to a pixel value range of 0-255. Then, the normalized temperature data is converted into an RGB image using color mapping. The final RGB image can be saved as an image file or displayed directly.

[0068] For step 300, the convolutional neural network can automatically extract meaningful features from the original image and combine these features layer by layer to form higher-level abstract features. Through this process, the convolutional neural network can learn low-level features such as texture, shape, and edge in the image, and gradually extract higher-level features to provide powerful input for subsequent fault classification. Specifically, the convolutional neural network consists of the following main components:

[0069] Convolution layer: Use convolution kernel to perform convolution operation on the input image to extract local features. Activation function layer: Apply nonlinear activation function, such as ReLU function, to the convolution result. Pooling layer: Downsample the feature map to reduce the dimension while retaining important features. Fully connected layer: Concatenate the features extracted by the previous layers and input them into the fully connected neural network for classification or regression.

[0070] By repeating the above convolution-activation-pooling process, the convolutional neural network can gradually extract higher-level features from the underlying edge and texture features, and finally output a highly abstract feature vector to provide powerful input for subsequent tasks.

[0071] For step 400, the Transformer model divides the feature data into patches of fixed size, and flattens the patches as input and embeds them into the Transformer model for processing to determine the fault category of the wind turbine gearbox.

[0072] In some embodiments of the present invention, see Figure 2 , step 100 comprises:

[0073] Step 101: extracting frequency spectrum data of the acoustic data;

[0074] Step 102: converting the spectrum data into the two-dimensional image through a Markov transfer field; wherein the acoustic data is one-dimensional data, and the category of the Markov transfer field is non-hyperparameter.

[0075] Specifically, the non-hyperparameter Markov transition field (MTF) is used to convert the spectrum encoding of the one-dimensional acoustic data into a two-dimensional image, and the statistics of the frequency amplitude space state transition probability are calculated to obtain the spectrum difference. The original acoustic data Sound is converted into a spectrum x', and the spectrum x' is normalized to obtain X'. The amplitudes of X' are arranged from small to large to obtain an ascending set X(n) = {x1, x2, ..., x N}. The distribution of the sequence x value is divided into Q discrete distribution quantile units, and the quantile q is used j (q j ∈[1,Q]) to mark the discrete quantile units. According to the division of each unit, each time domain value x i Mapped to the corresponding quantile unit q j superior.

[0076]

[0077] Where P i,j is the multi-step transition probability, indicating that it is currently located at q j The elements in the quantile region transition to q in multiple steps at the next moment. i The probability of the quantile region.

[0078] By combining the time domain signal amplitude with Q discrete distribution quantile units obtained after discretization, the continuous time series is converted into a quantile sequence represented by quantiles. The specific transition probability is obtained by counting the changes in the quantile sequence at each moment. Arrange all the transition probabilities of the Markov chain along the transition rule and construct the Q*Q Markov transition probability matrix T:

[0079]

[0080] Construct the Markov transition field matrix M:

[0081]

[0082] Where p i,j Indicates q i ,q j The transition probabilities of the corresponding quantile relations on the matrix T, the elements on the diagonal are the corresponding self-transition probabilities.

[0083] In some embodiments of the present invention, the two-dimensional image is a Markov transition field matrix; in addition, see Figure 3 , a wind turbine gearbox fault identification method based on multi-data fusion, further comprising:

[0084] Step 500: Generate a Markov transition field image according to the Markov transition field matrix.

[0085] In some embodiments of the present invention, for step 200, the infrared thermal imaging data is decomposed into three-channel data. Infrared thermal imaging is more sensitive to changes in thermal characteristics caused by faults. In order to retain the original fault information, the infrared thermal imaging image data is decomposed into RGB three channels.

[0086] P=[P R ,P G ,P B ]

[0087] Where P R ,P G ,P B They represent the red, green and blue color components of infrared thermal imaging data respectively.

[0088] In some embodiments of the present invention, see Figure 4 , step 300 comprises:

[0089] Step 301: performing data-level fusion on the Markov transition field image and the three-channel data to generate four-channel fused data;

[0090] Specifically, based on the three-channel RGB image and the frequency domain Markov transfer field image, data level fusion is performed to obtain the four-channel fused data D for subsequent network training. fusion :

[0091] D fusion =[P r ,P g ,P b ,M]

[0092] Step 302: Inputting the four-channel fused data into the convolutional neural network to extract high-order image features of the two-dimensional image and high-order image features of the three-channel data;

[0093] Step 303: performing feature-level fusion on the high-order image features of the two-dimensional image and the high-order image features of the three-channel data to generate the feature data.

[0094] In step 302 and step 303, a convolutional neural network is used to extract features of the four-channel image data. The feature extraction module consists of an input layer, a convolutional layer (Conv2d layer), a pooling layer (Pooling layer), and a flattening layer (Flatten layer). The network structure is deepened by repeatedly stacking the Conv2d layer and the Pooling layer to increase the complexity of the features.

[0095] The four-channel image data is converted into a two-dimensional matrix and input into the Conv2d layer. The Conv2d layer uses a certain size and number of convolution kernels to gradually perform convolution operations on the matrix. The convolution calculation formula is as follows:

[0096]

[0097] In the formula, is the jth feature of the lth convolution operation, M j The number of features of the l-th layer convolution operation, k is the trainable convolution kernel, b is the bias, f(·) is the activation function, and * is the convolution operation.

[0098] The Pooling layer is set after the Conv2d layer, and the output features of the Conv2d layer are downsampled. The feature information is integrated and reduced in dimension through the pooling kernel. The number of pooling kernels should be equal to the number of convolution kernels. The maximum pooling method is used for the pooling operation, that is, the maximum value of the pooling kernel area is taken:

[0099]

[0100] In the formula, y j is the output value of the pooling operation, is the pooled input feature of size n*n, u(n*n) is the pooling kernel of size n*n, and the maximum value in its area is taken.

[0101] The Flatten layer is used to "flatten" the input, that is, to convert the multi-dimensional input into one dimension, for the transition between the convolutional neural network feature extraction module and the feature fusion module. The image data of each channel is input into the feature extraction module structure respectively, and the hidden layers are set to three Conv2d layers, three Pooling layers and Flatten layers.

[0102] The high-order image features X of each modality image data are obtained through the above feature extraction module. i ,Y i …, use the Concatenate layer to perform feature-level fusion, the formula is as follows:

[0103]

[0104] X i and Y i represents the high-order feature map on the i-th channel of the two modalities, K i , K i+c represents the convolution kernel corresponding to the i-th channel feature map, K i , K i+c It is an m*n feature block.

[0105] In some embodiments of the present invention, the convolutional neural network includes: an input layer, a convolution layer, a pooling layer and a flattening layer; wherein the number of convolution layers is 3, and the number of pooling kernels of the pooling layer is equal to the number of convolution kernels of the convolution layer.

[0106] In some embodiments of the present invention, step 400 includes: using a visual Transformer module to determine the fault type. First, the feature block Z of size m*n is embedded in the Position Embedding layer. concatenate Divide into H feature blocks z of size p*p; for H p 2 The small feature block z of D dimensions is linearly transformed so that the original feature block is converted into N D-dimensional feature vectors.

[0107]

[0108] The Transformer layer simultaneously accepts N D-dimensional feature vectors and the position information of each feature block. The formula can be expressed as:

[0109]

[0110] G′ L =MHA(LN(G L-1 ))+G L-1

[0111] G L =MLP(LN(G′ L ))+G′ L

[0112] in G′ L , represents the input features, intermediate layer features, and output features of the Transformer layer, z cls is the class label vector, z 1 E represents a linear transformation of the first small feature block, MHA represents a multi-head attention mechanism, and MLP represents a feedforward neural network, which is used to connect the Sofmax classifier. MHA is a multiple independent self-attention operation. Each feature block is obtained by linear transformation. and The three embedded matrices are used to get the output of a single attention head:

[0113]

[0114] Finally, the outputs of multiple heads are spliced ​​by Contact operation to obtain the outputs of multiple heads, and the fault type judgment results are output:

[0115] MHA(Q,K,V)=Contact(head1,head2,…head n )

[0116] Among them, head n represents the nth self-attention operation.

[0117] From the above description, it can be seen that the embodiment of the present invention proposes a wind turbine gearbox fault discrimination method based on the complementary fusion of infrared thermal imaging and acoustic data. The method includes: first, converting a one-dimensional acoustic sequence into spectral data, and then converting the spectral data into a two-dimensional image using a Markov transfer field, and performing a three-channel decomposition of red, green and blue on the infrared thermal imaging; second, inputting the four-channel image data into a convolutional neural network, while reducing the difference of multi-sensor signals, adaptively extracting features from the image, and performing feature fusion; finally, inputting the feature image after weighted fusion into a fault type discrimination model based on visual Transfomer to judge the gearbox fault type. Compared with the traditional method, the present invention first converts the acoustic data into a two-dimensional data image, solves the problem of difficulty in information fusion caused by the different modalities of infrared thermal imaging data and acoustic data, and then uses a convolutional neural network to extract data features, and finally uses visual Transfomer to discriminate the fault type, thereby improving the utilization rate of fault features, and providing an effective solution for efficient and accurate state monitoring and fault diagnosis of wind turbine gearboxes.

[0118] To further illustrate the present invention, the present invention provides a specific application example of a wind turbine gearbox fault identification method based on multi-data fusion, see Figure 5 as well as Figure 6 , specifically including the following steps.

[0119] The purpose of the present invention is to overcome the problems that the multi-source heterogeneous data collected by existing sensors is difficult to use effectively, it is difficult to obtain accurate fault information using only single sensor data, and the multi-modal heterogeneous data lacks a modal fusion mechanism, which leads to low wind turbine gearbox fault type diagnosis accuracy.

[0120] It is understandable that the various structural and non-structural fault characteristics of wind turbine gearboxes include non-structural overheating damage caused by problems such as decreased or missing lubricating oil viscosity, and structural damage caused by problems such as gear tooth breakage. Infrared thermal imaging data is more sensitive to non-structural overheating damage, while acoustic data is more sensitive to structural damage such as gear tooth breakage. The two heterogeneous data are fused and then input into a fault type discrimination model based on convolutional neural networks and visual Transfomer to discriminate the internal fault type of the wind turbine gearbox. Based on this, the present invention provides a specific application example of a wind turbine gearbox fault discrimination method based on multi-data fusion, comprising the following steps:

[0121] S1: Preprocess the infrared thermal imaging and acoustic heterogeneous data of the wind turbine gearbox.

[0122] Specifically, see Figure 7 , decompose the infrared thermal imaging data into three-channel data, and use the non-hyperparameter Markov transfer field (MTF) calculation method to convert the spectrum encoding of the one-dimensional acoustic data into a two-dimensional image. Further, the acoustic data and infrared thermal imaging data are processed separately in the data fusion module. Decompose the infrared thermal imaging data into three-channel RGB images, calculate the spectrum of the acoustic data and encode it into an SMTF image. Based on the three-channel RGB image and the SMTF image, data-level fusion is performed to obtain four-channel fused data, which is used for subsequent convolutional neural network training.

[0123] S2: Feature extraction of four-channel image data through convolutional neural network.

[0124] The four-channel image data is convolved, pooled, and flattened, and the network structure is deepened through three convolutional layers and a maximum pooling layer to increase the complexity of the features.

[0125] S3: Determine the type of wind turbine gearbox fault through the Transfomer module.

[0126] Specifically, see Figure 8 , the feature image data processed by the convolutional neural network is input into the visual Transfomer module, the feature image is converted into a serialized representation that can be input into the Transformer encoder, and the Transformer encoder is used to capture global information and learn feature representations. Finally, the output of the model will be sent to an output layer containing a multi-layer perceptron (MLP) to obtain the optimal training model parameters and identify the gearbox fault type.

[0127] From the above description, it can be seen that the specific application example of the present invention proposes a method for distinguishing the fault of a wind turbine gearbox based on the complementary fusion of infrared thermal imaging and acoustic data. The method comprises: firstly, converting a one-dimensional acoustic sequence into spectral data, and then converting the spectral data into a two-dimensional image using a Markov transfer field, and performing a three-channel decomposition of the infrared thermal imaging into red, green and blue; secondly, inputting the four-channel image data into a convolutional neural network, and performing adaptive feature extraction and feature fusion on the image while reducing the difference in multi-sensor signals; finally, inputting the feature image after weighted fusion into a fault type discrimination model based on visual Transfomer to judge the type of gearbox fault. Compared with the traditional method, the present invention first converts the acoustic data into a two-dimensional data image, solves the problem of difficulty in information fusion caused by the different modalities of infrared thermal imaging data and acoustic data, and then uses a convolutional neural network to extract data features, and finally uses a visual Transfomer to distinguish the fault type, thereby improving the utilization rate of fault features, and providing an effective solution for efficient and accurate state monitoring and fault diagnosis of wind turbine gearboxes. Compared with the prior art, the present invention has the following advantages:

[0128] (1) The acoustic data is preprocessed based on the frequency domain Markov transfer field, and the one-dimensional acoustic data is converted into spectral data. This can more sensitively capture the changes in signal frequency and convert the spectral information into image data, thus solving the problem of difficulty in fusion caused by the different modalities of infrared thermal imaging and acoustic data.

[0129] (2) Infrared thermal imaging data and acoustic data are used together as the data input part of the fault discrimination model. The data-level fusion method is used to minimize the information loss of the original infrared thermal imaging and acoustic data, and to achieve full and efficient use of different modal information, thereby improving the problem that infrared thermal imaging data and acoustic data are only sensitive to non-structural and structural damage, respectively, making it difficult to obtain accurate fault information.

[0130] (3) Use convolutional neural networks and visual transformers to identify fault types. Use convolutional neural networks to automatically learn spatial hierarchical features through multi-layer convolution to extract local features of the image. Use visual transformers to perform global modeling through a self-attention mechanism. When processing images, the relationship between various regions in the image is considered at the same time, which has better global perception capabilities. By combining the advantages of these two methods, the accuracy of the model in identifying gearbox fault types can be improved while maintaining computational efficiency.

[0131] Based on the same inventive concept, the embodiments of the present application also provide a wind turbine gearbox fault identification device based on multi-data fusion, which can be used to implement the method described in the above embodiments, such as the following embodiments. Since the principle of solving the problem by the wind turbine gearbox fault identification device based on multi-data fusion is similar to the wind turbine gearbox fault identification method based on multi-data fusion, the implementation of the wind turbine gearbox fault identification device based on multi-data fusion can refer to the implementation of the wind turbine gearbox fault identification method based on multi-data fusion, and the repeated parts will not be repeated. As used below, the term "unit" or "module" can be a combination of software and / or hardware that implements a predetermined function. Although the system described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0132] The embodiment of the present invention provides a specific implementation of a wind turbine gearbox fault determination device based on multi-data fusion that can implement a wind turbine gearbox fault determination method based on multi-data fusion, see Fig. 9 , a wind turbine gearbox fault identification device based on multi-data fusion specifically includes the following contents:

[0133] An acoustic data conversion module 10, used to convert the acoustic data of the wind turbine gearbox into a two-dimensional image;

[0134] The infrared thermal imaging data parsing module 20 is used to parse the infrared thermal imaging data of the wind turbine gearbox into three-channel data;

[0135] A feature data extraction module 30, configured to extract feature data of the two-dimensional image and the three-channel data according to a pre-generated convolutional neural network;

[0136] The fault identification module 40 is used to identify the gearbox fault of the wind turbine generator set according to the characteristic data and the pre-generated Transformer model.

[0137] In some embodiments of the present invention, see Fig.10 , the acoustic data conversion module 10 includes:

[0138] A spectrum data extraction unit 10a, used to extract spectrum data of the acoustic data;

[0139] The two-dimensional data generating unit 10b is used to convert the spectrum data into the two-dimensional image through a Markov transfer field; wherein the acoustic data is one-dimensional data, and the category of the Markov transfer field is non-hyperparameter.

[0140] In some embodiments of the present invention, see Fig.11, a wind turbine gearbox fault identification device based on multi-data fusion, further comprising: the two-dimensional image is a Markov transfer field matrix;

[0141] The Markov transition field image generation module 50 is used to generate a Markov transition field image according to the Markov transition field matrix.

[0142] In some embodiments of the present invention, see Fig.12 , the feature data extraction module 30 includes:

[0143] A fused data generating unit 30a, configured to perform data-level fusion on the Markov transition field image and the three-channel data to generate four-channel fused data;

[0144] A high-order image feature extraction unit 30b, used for inputting the fused data of the four channels into the convolutional neural network to extract the high-order image features of the two-dimensional image and the high-order image features of the three-channel data;

[0145] The feature data extraction unit 30c is used to perform feature-level fusion on the high-order image features of the two-dimensional image and the high-order image features of the three-channel data to generate the feature data.

[0146] In some embodiments of the present invention, the convolutional neural network includes: an input layer, a convolution layer, a pooling layer and a flattening layer; wherein the number of convolution layers is 3, and the number of pooling kernels of the pooling layer is equal to the number of convolution kernels of the convolution layer.

[0147] From the above description, it can be seen that an embodiment of the present invention provides a wind turbine gearbox fault identification device based on multi-data fusion, including: an acoustic data conversion module, used to convert the acoustic data of the wind turbine gearbox into a two-dimensional image; an infrared thermal imaging data analysis module, used to parse the infrared thermal imaging data of the wind turbine gearbox into three-channel data; a feature data extraction module, used to extract feature data of the two-dimensional image and the three-channel data according to a pre-generated convolutional neural network; a fault identification module, used to identify the wind turbine gearbox fault according to the feature data and the pre-generated Transformer model.

[0148] In summary, the present invention uses a multimodal information fusion method to combine the interrelated fault features in multiple sensor signals, learns relevant features from different modal perspectives, enriches fault feature information, and enhances the accuracy and reliability of the fault diagnosis method. At the same time, combined with the characteristics of multiple sensors, it can prevent the failure of certain sensors in extreme environments and improve the robustness and stability of the fault diagnosis method.

[0149] The embodiment of the present application also provides a specific implementation of an electronic device capable of implementing all steps in the wind turbine gearbox fault identification method based on multi-data fusion in the above embodiment, see Fig.13 , electronic equipment specifically includes the following:

[0150] Processor (processor) 1201, memory (memory) 1202, communication interface (CommunicationsInterface) 1203 and bus 1204;

[0151] The processor 1201 , the memory 1202 , and the communication interface 1203 communicate with each other via the bus 1204 ; the communication interface 1203 is used to realize information transmission between relevant devices such as the server-side device, the power measurement device, and the user-side device.

[0152] The processor 1201 is used to call the computer program in the memory 1202. When the processor executes the computer program, all the steps in the wind turbine gearbox fault identification method based on multi-data fusion in the above embodiment are implemented. For example, when the processor executes the computer program, the following steps are implemented:

[0153] Step 100: converting the acoustic data of the wind turbine gearbox into a two-dimensional image;

[0154] Step 200: parsing the infrared thermal imaging data of the wind turbine gearbox into three-channel data;

[0155] Step 300: extracting feature data of the two-dimensional image and the three-channel data according to a pre-generated convolutional neural network;

[0156] Step 400: Identify the wind turbine gearbox fault according to the characteristic data and the pre-generated Transformer model.

[0157] The embodiment of the present application also provides a computer-readable storage medium capable of implementing all the steps of the wind turbine gearbox fault determination method based on multi-data fusion in the above embodiment. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, all the steps of the wind turbine gearbox fault determination method based on multi-data fusion in the above embodiment are implemented. For example, when the processor executes the computer program, the following steps are implemented:

[0158] Step 100: converting the acoustic data of the wind turbine gearbox into a two-dimensional image;

[0159] Step 200: parsing the infrared thermal imaging data of the wind turbine gearbox into three-channel data;

[0160] Step 300: extracting feature data of the two-dimensional image and the three-channel data according to a pre-generated convolutional neural network;

[0161] Step 400: Identify the wind turbine gearbox fault according to the characteristic data and the pre-generated Transformer model.

[0162] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the hardware + program embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0163] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0164] Although the present application provides method operation steps such as embodiments or flow charts, more or fewer operation steps may be included based on conventional or non-creative labor. The order of steps listed in the embodiments is only one way of executing the order of many steps and does not represent the only execution order. When the actual device or client product is executed, it can be executed in the order of the method shown in the embodiments or the drawings or in parallel (for example, in a parallel processor or multi-threaded processing environment).

[0165] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0166] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0167] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0168] The present invention uses specific embodiments to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A wind turbine gearbox fault identification method based on multi-data fusion, characterized in that: include: Converting the acoustic data of the wind turbine gearbox into a two-dimensional image; Parsing the infrared thermal imaging data of the wind turbine gearbox into three-channel data; Extracting feature data of the two-dimensional image and the three-channel data according to a pre-generated convolutional neural network; The wind turbine gearbox fault is determined based on the characteristic data and the pre-generated Transformer model.

2. The method for determining a wind turbine gearbox fault according to claim 1, characterized in that: Converting the acoustic data of the wind turbine gearbox into a two-dimensional image includes: Extracting frequency spectrum data of the acoustic data; The spectrum data is converted into the two-dimensional image through a Markov transfer field; wherein the acoustic data is one-dimensional data, and the category of the Markov transfer field is non-hyperparameter.

3. The method for determining a wind turbine gearbox fault according to claim 2, characterized in that: Also includes: The two-dimensional image is a Markov transition field matrix; A Markov transition field image is generated according to the Markov transition field matrix.

4. The method for determining a wind turbine gearbox fault according to claim 3, characterized in that: The extracting feature data of the two-dimensional image and the three-channel data according to the pre-generated convolutional neural network includes: Performing data-level fusion on the Markov transition field image and the three-channel data to generate four-channel fused data; Inputting the four-channel fused data into the convolutional neural network to extract high-order image features of the two-dimensional image and high-order image features of the three-channel data; The high-order image features of the two-dimensional image and the high-order image features of the three-channel data are fused at a feature level to generate the feature data.

5. The method for determining a wind turbine gearbox fault according to any one of claims 1 to 4, characterized in that: The convolutional neural network includes: an input layer, a convolution layer, a pooling layer and a flattening layer; the number of convolution layers is 3, and the number of pooling cores of the pooling layer is equal to the number of convolution cores of the convolution layer.

6. A wind turbine gearbox fault identification device based on multi-data fusion, characterized in that: include: An acoustic data conversion module, used for converting the acoustic data of the wind turbine gearbox into a two-dimensional image; An infrared thermal imaging data parsing module, used for parsing the infrared thermal imaging data of the wind turbine gearbox into three-channel data; A feature data extraction module, used to extract feature data of the two-dimensional image and the three-channel data according to a pre-generated convolutional neural network; A fault identification module is used to identify the gearbox fault of the wind turbine generator set according to the characteristic data and a pre-generated Transformer model.

7. The wind turbine gearbox fault identification device according to claim 6, characterized in that: The acoustic data conversion module includes: A spectrum data extraction unit, used for extracting spectrum data of the acoustic data; A two-dimensional data generating unit is used to convert the spectrum data into the two-dimensional image through a Markov transfer field; wherein the acoustic data is one-dimensional data, and the category of the Markov transfer field is non-hyperparameter.

8. The wind turbine gearbox fault identification device according to claim 7, characterized in that: Also includes: The two-dimensional image is a Markov transition field matrix; The Markov transition field image generation module is used to generate a Markov transition field image according to the Markov transition field matrix.

9. The wind turbine gearbox fault identification device according to claim 8, characterized in that: The feature data extraction module comprises: A fused data generating unit, used for performing data-level fusion on the Markov transition field image and the three-channel data to generate four-channel fused data; A high-order image feature extraction unit, used for inputting the fused data of the four channels into the convolutional neural network to extract high-order image features of the two-dimensional image and high-order image features of the three-channel data; The feature data extraction unit is used to perform feature-level fusion on the high-order image features of the two-dimensional image and the high-order image features of the three-channel data to generate the feature data.

10. The wind turbine gearbox fault identification device according to any one of claims 6 to 9, characterized in that: The convolutional neural network includes: an input layer, a convolution layer, a pooling layer and a flattening layer; the number of convolution layers is 3, and the number of pooling cores of the pooling layer is equal to the number of convolution cores of the convolution layer.

11. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the steps of the wind turbine gearbox fault identification method based on multi-data fusion as described in any one of claims 1 to 5 are implemented.

12. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the wind turbine gearbox fault diagnosis method based on multi-data fusion as described in any one of claims 1 to 5 are implemented.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the wind turbine gearbox fault identification method based on multi-data fusion as described in any one of claims 1 to 5 are implemented.

Images