Blade damage classification method and device based on fan blade audio signals

Abstract

The invention relates to an intelligent detection technology, and discloses a fan blade audio signal-based blade damage classification method, which comprises the following steps of: acquiring a fan audio signal, converting the fan audio signal into a first signal image by using a first image conversion model, and converting the fan audio signal into a second signal image by using a second image conversion model; fusing the first signal image and the second signal image to obtain a signal fusion image, extracting a signal global feature in the signal fusion image, and extracting a point cloud feature in the signal fusion image to obtain a signal local feature; and fusing the signal global feature and the signal local feature to obtain a signal fusion feature, and inputting the signal fusion feature into a deep neural network to obtain a blade damage classification result. The invention further provides a blade damage classification device based on the fan blade audio signal, electronic equipment and a storage medium. According to the invention, the accuracy of fan blade damage type classification can be improved.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent detection technology, and in particular relates to a blade damage classification method and device based on wind turbine blade audio signals. Background Art

[0002] Wind power generation is rapidly developing, but wind turbine blades are often exposed to harsh environments like plum rains and strong convection, which can easily cause damage such as holes and cracks. Blade damage classification based on wind turbine blade audio signals classifies blade conditions based on collected wind turbine blade audio information. Currently, timely classification and detection of blade conditions are necessary to improve operational efficiency and timeliness. Existing technologies typically use deep learning algorithms for classification and detection. However, deep learning cannot effectively extract information features from blade audio information when performing feature extraction. Furthermore, normal samples account for a large proportion of the collected rock data, resulting in low accuracy in blade damage classification using existing technologies. Summary of the Invention

[0003] The present invention provides a blade damage classification method and device based on fan blade audio signals, which can improve the accuracy of fan blade damage type classification.

[0004] To achieve the above objectives, the present invention provides a blade damage classification method based on wind turbine blade audio signals, comprising:

[0005] Acquire a fan audio signal, convert the fan audio signal into a first signal image using a first image conversion model, and convert the fan audio signal into a second signal image using a second image conversion model;

[0006] The first signal image and the second signal image are fused to obtain a signal fusion image, and a fusion feature extraction network is used to extract global signal features in the signal fusion image, and a point cloud feature extraction network is used to extract point cloud features in the signal fusion image to obtain local signal features;

[0007] The signal global features and signal local features are fused to obtain the signal fusion features, which are then input into the deep neural network to obtain the blade damage classification results.

[0008] Optionally, converting the fan audio signal into a first signal image using the first image conversion model includes:

[0009] Mapping the fan audio signal to a polar coordinate system to form multiple polar coordinate points;

[0010] A Gram matrix is ​​constructed through the angle relationship between the polar coordinate points, matrix elements in the Gram matrix are mapped to pixel values, and the Gram matrix is ​​converted into a first signal image.

[0011] Optionally, converting the fan audio signal into a second signal image using the second image conversion model includes:

[0012] Convert the fan audio signal into a discrete state sequence and calculate the transition probability of each state in the discrete state sequence;

[0013] The transition probabilities are aligned according to time to obtain a two-dimensional probability matrix, and the transition probability values ​​in the two-dimensional probability matrix are mapped to pixel values ​​to obtain a second signal image.

[0014] Optionally, fusing the first signal image and the second signal image to obtain a signal fusion image includes:

[0015] Assigning a first weight to the Gram matrix, assigning a second weight to the two-dimensional probability matrix, normalizing the weighted Gram matrix and the weighted two-dimensional probability matrix, and performing a matrix concatenation operation after normalization to obtain a fused image feature matrix;

[0016] Generate a signal fusion image based on the fusion image feature matrix.

[0017] Optionally, extracting global features of signals in the signal fusion image using a fusion feature extraction network includes:

[0018] The convolution module in the fusion feature extraction network is used to perform convolution processing on the signal fusion image to obtain a convolution feature map;

[0019] Perform global pooling operation on the convolution feature map to obtain the pooled feature map;

[0020] The pooled feature maps are aggregated to obtain the global signal features.

[0021] Optionally, the extracting point cloud features from the signal fusion image using a point cloud feature extraction network to obtain local signal features includes:

[0022] Use depth estimation to convert each pixel in the signal fusion image into a three-dimensional point;

[0023] Normalize the 3D points and remove outliers to obtain the target 3D points;

[0024] The point cloud feature extraction network is used to extract the attribute features of the target three-dimensional points and perform global pooling to obtain the local features of the signal.

[0025] Optionally, the fusing of the signal global feature and the signal local feature to obtain the signal fusion feature includes:

[0026] Assigning a first initial weight to the signal global feature and a second initial weight to the signal local feature, and using the signal global feature assigned the first initial weight as a key and a value in an attention mechanism, and using the signal local feature assigned the second initial weight as a query in the attention mechanism;

[0027] The local feature of the signal assigned the second initial weight is used as the query in the attention mechanism and is concatenated with the output value of the attention mechanism to obtain the first signal fusion feature;

[0028] The global feature of the signal assigned the first initial weight is used as the query in the attention mechanism, and the local feature of the signal assigned the second initial weight is used as the key and value in the attention mechanism, and the global feature of the signal assigned the first initial weight is used as the query in the attention mechanism and the output value of the attention mechanism is spliced ​​to obtain the second signal fusion feature;

[0029] The first signal fusion feature and the second signal fusion feature are concatenated to obtain a signal fusion feature.

[0030] In order to solve the above problems, the present invention further provides a blade damage classification device based on wind turbine blade audio signals, the device comprising:

[0031] an image conversion module, configured to obtain a fan audio signal, convert the fan audio signal into a first signal image using a first image conversion model, and convert the fan audio signal into a second signal image using a second image conversion model;

[0032] a feature fusion module, configured to fuse the first signal image and the second signal image to obtain a signal fusion image, and extract global signal features from the signal fusion image using a fusion feature extraction network, and extract point cloud features from the signal fusion image using a point cloud feature extraction network to obtain local signal features;

[0033] The blade damage type classification module is used to fuse the global signal features and the local signal features to obtain the signal fusion features, and input the signal fusion features into the deep neural network to obtain the blade damage classification results.

[0034] In order to solve the above problem, the present invention further provides an electronic device, comprising:

[0035] at least one processor; and,

[0036] a memory communicatively connected to the at least one processor; wherein,

[0037] The memory stores a computer program that can be executed by the at least one processor. The computer program is executed by the at least one processor so that the at least one processor can perform the above-mentioned blade damage classification method based on wind turbine blade audio signals.

[0038] In order to solve the above problems, the present invention also provides a computer-readable storage medium, in which at least one computer program is stored. The at least one computer program is executed by a processor in an electronic device to implement the above-mentioned blade damage classification method based on wind turbine blade audio signals.

[0039] The present invention converts audio signals into image signals, and can use image signals to represent audio signals, making audio signals easier to process. In addition, by fusing the converted images, the integrity of the features presented by the audio signals can be guaranteed. In addition, by fusing the global features of the signal and the point cloud features in the signal fusion image extracted using a point cloud feature extraction network, the accuracy of the classification of wind turbine blade damage types can be improved when the sample size is small. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A schematic flow chart of a method for classifying blade damage based on wind turbine blade audio signals according to an embodiment of the present invention;

[0041] Figure 2 This is a functional module diagram of a blade damage classification device based on wind turbine blade audio signals provided by one embodiment of the present invention;

[0042] Figure 3 A schematic structural diagram of an electronic device for implementing a blade damage classification method based on wind turbine blade audio signals provided in one embodiment of the present invention.

[0043] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0044] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0045] The embodiment of the present application provides a blade damage classification method based on the audio signal of the wind blade. The execution subject of the blade damage classification method based on the audio signal of the wind blade includes but is not limited to at least one of the electronic devices such as the server and the terminal that can be configured to execute the method provided by the embodiment of the present application. In other words, the blade damage classification method based on the audio signal of the wind blade can be executed by software or hardware installed on the terminal device or the server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc. The server can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0046] Reference Figure 1 FIG. 1 is a flow chart of a method for classifying blade damage based on wind turbine blade audio signals according to an embodiment of the present invention. In this embodiment, the method for classifying blade damage based on wind turbine blade audio signals includes:

[0047] S1. Acquire a fan audio signal, convert the fan audio signal into a first signal image using a first image conversion model, and convert the fan audio signal into a second signal image using a second image conversion model.

[0048] In an embodiment of the present invention, the fan audio signal refers to the audio signal collected by an audio collector when the fan is running. The audio collector can be an acoustic network array audio collector for collection. The acoustic network array audio collector has high sensitivity and wide bandwidth, and can capture weak audio signals under different operating conditions to ensure the integrity of the audio signal.

[0049] In the embodiment of the present invention, the first image conversion model refers to a conversion model used to extract static features in an audio signal. For example, the first image conversion model may be a Gramian Angular Field.

[0050] In an embodiment of the present invention, the second image conversion model refers to a conversion model used to extract dynamic features in the audio signal. For example, the second image conversion model may be a Markov transition field, which can extract dynamic time series features in the audio signal.

[0051] As an embodiment of the present invention, the method of converting the fan audio signal into the first signal image by using the first image conversion model further includes preprocessing the fan audio signal by using wavelet transform.

[0052] Furthermore, the wind turbine audio signal is preprocessed using wavelet transform, including:

[0053] Obtaining an outlier sampling point sequence of the fan audio signal, and normalizing the outlier sampling point sequence to obtain a normalized audio signal;

[0054] The normalized audio signal is decomposed into multiple scales according to the number of wavelet decomposition layers calculated based on the sampling frequency and signal frequency band. The wavelet transform noise threshold is set and the noise audio in the fan audio signal is filtered according to the noise threshold. After filtering, a standard fan audio signal is obtained.

[0055] In the embodiment of the present invention, wavelet transform refers to a time-frequency analysis tool that realizes multi-resolution analysis of the signal by decomposing the signal into "wavelet basis functions" of different time and frequency scales. Wavelet transform can simultaneously capture the time domain and frequency domain characteristics of the signal.

[0056] As an embodiment of the present invention, converting the fan audio signal into a first signal image using a first image conversion model includes:

[0057] Mapping the fan audio signal to a polar coordinate system to form multiple polar coordinate points;

[0058] A Gram matrix is ​​constructed through the angle relationship between the polar coordinate points, matrix elements in the Gram matrix are mapped to pixel values, and the Gram matrix is ​​converted into a first signal image.

[0059] As an embodiment of the present invention, converting the fan audio signal into a second signal image using a second image conversion model includes:

[0060] Convert the fan audio signal into a discrete state sequence and calculate the transition probability of each state in the discrete state sequence;

[0061] The transition probabilities are aligned according to time to obtain a two-dimensional probability matrix, and the transition probability values ​​in the two-dimensional probability matrix are mapped to pixel values ​​to obtain a second signal image.

[0062] S2. Fuse the first signal image and the second signal image to obtain a signal fusion image, and use a fusion feature extraction network to extract global signal features in the signal fusion image, and use a point cloud feature extraction network to extract point cloud features in the signal fusion image to obtain local signal features.

[0063] In an embodiment of the present invention, the fusion feature extraction network refers to a neural network that extracts image features, and the fusion feature extraction network may be a 3D convolutional neural network.

[0064] As an embodiment of this aspect, fusing the first signal image and the second signal image to obtain a signal fusion image includes:

[0065] Assigning a first weight to the Gram matrix, assigning a second weight to the two-dimensional probability matrix, normalizing the weighted Gram matrix and the weighted two-dimensional probability matrix, and performing a matrix concatenation operation after normalization to obtain a fused image feature matrix;

[0066] Generate a signal fusion image based on the fusion image feature matrix.

[0067] Furthermore, a fusion feature extraction network is used to extract the global features of the signal in the signal fusion image, including:

[0068] The convolution module in the fusion feature extraction network is used to perform convolution processing on the signal fusion image to obtain a convolution feature map;

[0069] Perform global pooling operation on the convolution feature map to obtain the pooled feature map;

[0070] The pooled feature maps are aggregated to obtain the global signal features.

[0071] The embodiment of the present invention can compress the convolution feature map into a 1×1×1 vector through a global pooling operation.

[0072] In an embodiment of the present invention, the point cloud feature extraction network refers to a neural network model for processing three-dimensional point cloud data, and the point cloud feature extraction network can be a PointNet / PointNet++ neural network.

[0073] Furthermore, the point cloud feature extraction network is used to extract the point cloud features in the signal fusion image to obtain the local features of the signal, including:

[0074] Use depth estimation to convert each pixel in the signal fusion image into a three-dimensional point;

[0075] Normalize the 3D points and remove outliers to obtain the target 3D points;

[0076] The point cloud feature extraction network is used to extract the attribute features of the target three-dimensional points and perform global pooling to obtain the local features of the signal.

[0077] In the embodiment of the present invention, the attribute feature refers to the color information of the three-dimensional point, which comes from the pixel value in the signal fusion image.

[0078] S3. Fuse the global signal features and the local signal features to obtain the signal fusion features, and input the signal fusion features into the deep neural network to obtain the blade damage classification results.

[0079] In the embodiment of the present invention, a deep neural network (DNN) refers to a machine learning model based on an artificial neural network. Its core feature is to extract abstract features of data layer by layer through multi-layer nonlinear transformations, thereby solving complex pattern recognition and prediction tasks.

[0080] As an embodiment of the present invention, the signal global feature and the signal local feature are fused to obtain the signal fusion feature, including:

[0081] Assigning a first initial weight to the signal global feature and a second initial weight to the signal local feature, and using the signal global feature assigned the first initial weight as a key and a value in an attention mechanism, and using the signal local feature assigned the second initial weight as a query in the attention mechanism;

[0082] The local feature of the signal assigned the second initial weight is used as the query in the attention mechanism and is concatenated with the output value of the attention mechanism to obtain the first signal fusion feature;

[0083] The global feature of the signal assigned the first initial weight is used as the query in the attention mechanism, and the local feature of the signal assigned the second initial weight is used as the key and value in the attention mechanism, and the global feature of the signal assigned the first initial weight is used as the query in the attention mechanism and the output value of the attention mechanism is spliced ​​to obtain the second signal fusion feature;

[0084] The first signal fusion feature and the second signal fusion feature are concatenated to obtain a signal fusion feature.

[0085] The embodiment of the present invention calculates the probability values ​​of signal fusion features mapped to different blade damage types through the deep neural network output layer, and obtains the blade damage classification results based on the probability values.

[0086] The present invention converts audio signals into image signals, and can use image signals to represent audio signals, making audio signals easier to process. In addition, by fusing the converted images, the integrity of the features presented by the audio signals can be guaranteed. In addition, by fusing the global features of the signal and the point cloud features in the signal fusion image extracted using a point cloud feature extraction network, the accuracy of the fan blade damage type classification can be achieved with a small sample size.

[0087] like Figure 2 , which is a functional module diagram of a blade damage classification device based on wind turbine blade audio signals provided by one embodiment of the present invention.

[0088] The blade damage classification device 100 based on wind turbine blade audio signals described in the present invention can be installed in an electronic device. Depending on the functionality implemented, the blade damage classification device 100 based on wind turbine blade audio signals can include an image conversion module 101, a feature fusion module 102, and a blade damage type classification module 103.

[0089] The module described in the present invention may also be referred to as a unit, which refers to a series of computer program segments that can be executed by a processor of an electronic device and can complete fixed functions, and is stored in a memory of the electronic device.

[0090] In this embodiment, the functions of each module / unit are as follows:

[0091] The image conversion module 101 is used to obtain the fan audio signal, convert the fan audio signal into a first signal image using a first image conversion model, and convert the fan audio signal into a second signal image using a second image conversion model.

[0092] In an embodiment of the present invention, the fan audio signal refers to the audio signal collected by an audio collector when the fan is running. The audio collector can be an acoustic network array audio collector for collection. The acoustic network array audio collector has high sensitivity and wide bandwidth, and can capture weak audio signals under different operating conditions to ensure the integrity of the audio signal.

[0093] In the embodiment of the present invention, the first image conversion model refers to a conversion model used to extract static features in an audio signal. For example, the first image conversion model may be a Gramian Angular Field.

[0094] In an embodiment of the present invention, the second image conversion model refers to a conversion model used to extract dynamic features in the audio signal. For example, the second image conversion model may be a Markov transition field, which can extract dynamic time series features in the audio signal.

[0095] As an embodiment of the present invention, the method of converting the fan audio signal into the first signal image by using the first image conversion model further includes preprocessing the fan audio signal by using wavelet transform.

[0096] Furthermore, the wind turbine audio signal is preprocessed using wavelet transform, including:

[0097] Obtaining an outlier sampling point sequence of the fan audio signal, and normalizing the outlier sampling point sequence to obtain a normalized audio signal;

[0098] The normalized audio signal is decomposed into multiple scales according to the number of wavelet decomposition layers calculated based on the sampling frequency and signal frequency band. The wavelet transform noise threshold is set and the noise audio in the fan audio signal is filtered according to the noise threshold. After filtering, a standard fan audio signal is obtained.

[0099] In the embodiment of the present invention, wavelet transform refers to a time-frequency analysis tool that realizes multi-resolution analysis of the signal by decomposing the signal into "wavelet basis functions" of different time and frequency scales. Wavelet transform can simultaneously capture the time domain and frequency domain characteristics of the signal.

[0100] As an embodiment of the present invention, converting the fan audio signal into a first signal image using a first image conversion model includes:

[0101] Mapping the fan audio signal to a polar coordinate system to form multiple polar coordinate points;

[0102] A Gram matrix is ​​constructed through the angle relationship between the polar coordinate points, matrix elements in the Gram matrix are mapped to pixel values, and the Gram matrix is ​​converted into a first signal image.

[0103] As an embodiment of the present invention, converting the fan audio signal into a second signal image using a second image conversion model includes:

[0104] Convert the fan audio signal into a discrete state sequence and calculate the transition probability of each state in the discrete state sequence;

[0105] The transition probabilities are aligned according to time to obtain a two-dimensional probability matrix, and the transition probability values ​​in the two-dimensional probability matrix are mapped to pixel values ​​to obtain a second signal image.

[0106] The feature fusion module 102 is used to fuse the first signal image and the second signal image to obtain a signal fusion image, and use a fusion feature extraction network to extract global signal features in the signal fusion image, and use a point cloud feature extraction network to extract point cloud features in the signal fusion image to obtain local signal features.

[0107] In an embodiment of the present invention, the fusion feature extraction network refers to a neural network that extracts image features, and the fusion feature extraction network may be a 3D convolutional neural network.

[0108] As an embodiment of this aspect, fusing the first signal image and the second signal image to obtain a signal fusion image includes:

[0109] Assigning a first weight to the Gram matrix, assigning a second weight to the two-dimensional probability matrix, normalizing the weighted Gram matrix and the weighted two-dimensional probability matrix, and performing a matrix concatenation operation after normalization to obtain a fused image feature matrix;

[0110] Generate a signal fusion image based on the fusion image feature matrix.

[0111] Furthermore, a fusion feature extraction network is used to extract the global features of the signal in the signal fusion image, including:

[0112] The convolution module in the fusion feature extraction network is used to perform convolution processing on the signal fusion image to obtain a convolution feature map;

[0113] Perform global pooling operation on the convolution feature map to obtain the pooled feature map;

[0114] The pooled feature maps are aggregated to obtain the global signal features.

[0115] The embodiment of the present invention can compress the convolution feature map into a 1×1×1 vector through a global pooling operation.

[0116] In an embodiment of the present invention, the point cloud feature extraction network refers to a neural network model for processing three-dimensional point cloud data, and the point cloud feature extraction network can be a PointNet / PointNet++ neural network.

[0117] Furthermore, the point cloud feature extraction network is used to extract the point cloud features in the signal fusion image to obtain the local features of the signal, including:

[0118] Use depth estimation to convert each pixel in the signal fusion image into a three-dimensional point;

[0119] Normalize the 3D points and remove outliers to obtain the target 3D points;

[0120] The point cloud feature extraction network is used to extract the attribute features of the target three-dimensional points and perform global pooling to obtain the local features of the signal.

[0121] In the embodiment of the present invention, the attribute feature refers to the color information of the three-dimensional point, which comes from the pixel value in the signal fusion image.

[0122] The blade damage type classification module 103 is used to fuse the global signal features and the local signal features to obtain a signal fusion feature, and input the signal fusion feature into a deep neural network to obtain a blade damage classification result.

[0123] In the embodiment of the present invention, a deep neural network (DNN) refers to a machine learning model based on an artificial neural network. Its core feature is to extract abstract features of data layer by layer through multi-layer nonlinear transformations, thereby solving complex pattern recognition and prediction tasks.

[0124] As an embodiment of the present invention, the signal global feature and the signal local feature are fused to obtain the signal fusion feature, including:

[0125] Assigning a first initial weight to the signal global feature and a second initial weight to the signal local feature, and using the signal global feature assigned the first initial weight as a key and a value in an attention mechanism, and using the signal local feature assigned the second initial weight as a query in the attention mechanism;

[0126] The local feature of the signal assigned the second initial weight is used as the query in the attention mechanism and is concatenated with the output value of the attention mechanism to obtain the first signal fusion feature;

[0127] The global feature of the signal assigned the first initial weight is used as the query in the attention mechanism, and the local feature of the signal assigned the second initial weight is used as the key and value in the attention mechanism, and the global feature of the signal assigned the first initial weight is used as the query in the attention mechanism and the output value of the attention mechanism is spliced ​​to obtain the second signal fusion feature;

[0128] The first signal fusion feature and the second signal fusion feature are concatenated to obtain a signal fusion feature.

[0129] The embodiment of the present invention calculates the probability values ​​of signal fusion features mapped to different blade damage types through the deep neural network output layer, and obtains the blade damage classification results based on the probability values.

[0130] like Figure 3 FIG. 1 is a schematic structural diagram of an electronic device for implementing a blade damage classification method based on wind turbine blade audio signals, provided by an embodiment of the present invention.

[0131] The electronic device may include a processor 10, a memory 11, a communication bus 12 and a communication interface 13, and may also include a computer program stored in the memory 11 and executable on the processor 10, such as a blade damage classification method program based on wind turbine blade audio signals.

[0132] In some embodiments, the processor 10 may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and a combination of various control chips. The processor 10 is the control core (Control Unit) of the electronic device, connecting the various components of the entire electronic device using various interfaces and lines, and executing the programs or modules stored in the memory 11 (for example, executing a blade damage classification method program based on wind turbine blade audio signals, etc.), as well as calling the data stored in the memory 11, to execute various functions of the electronic device and process data.

[0133] The memory 11 includes at least one type of readable storage medium, and the readable storage medium includes a flash memory, a mobile hard disk, a multimedia card, a card-type memory (for example, an SD or DX memory, etc.), a magnetic memory, a disk, an optical disk, etc. In some embodiments, the memory 11 may be an internal storage unit of an electronic device, such as a mobile hard disk of the electronic device. In other embodiments, the memory 11 may also be an external storage device of an electronic device, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card, etc. equipped on the electronic device. Furthermore, the memory 11 may also include both an internal storage unit and an external storage device of the electronic device. The memory 11 can not only be used to store application software and various types of data installed in the electronic device, such as the code of a blade damage classification method program based on wind turbine blade audio signals, but can also be used to temporarily store data that has been output or is to be output.

[0134] The communication bus 12 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is configured to enable communication between the memory 11 and at least one processor 10, etc.

[0135] The communication interface 13 is used for communication between the above-mentioned electronic device and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device and other electronic devices. The user interface may be a display (Display), an input unit (such as a keyboard (Keyboard)), optionally, the user interface may also be a standard wired interface, a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode, organic light-emitting diode) touch device, etc. Among them, the display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device and to display a visual user interface.

[0136] Figure 3Only the electronic device with components is shown, and it can be understood by those skilled in the art that Figure 3 The structure shown does not limit the electronic device, and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0137] For example, although not shown, the electronic device may further include a power source (such as a battery) for powering various components. Preferably, the power source may be logically connected to the at least one processor 10 via a power management device, thereby implementing functions such as charge management, discharge management, and power consumption management through the power management device. The power source may further include any components such as one or more DC or AC power sources, a recharging device, a power failure detection circuit, a power converter or inverter, a power status indicator, etc. The electronic device may further include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0138] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.

[0139] The program of a blade damage classification method based on wind turbine blade audio signals stored in the memory 11 of the electronic device is a combination of multiple instructions. When executed in the processor 10, the following can be achieved:

[0140] Acquire a fan audio signal, convert the fan audio signal into a first signal image using a first image conversion model, and convert the fan audio signal into a second signal image using a second image conversion model;

[0141] The first signal image and the second signal image are fused to obtain a signal fusion image, and a fusion feature extraction network is used to extract global signal features in the signal fusion image, and a point cloud feature extraction network is used to extract point cloud features in the signal fusion image to obtain local signal features;

[0142] The signal global features and signal local features are fused to obtain the signal fusion features, which are then input into the deep neural network to obtain the blade damage classification results.

[0143] Specifically, the specific implementation method of the processor 10 for the above instructions can refer to the description of the relevant steps in the corresponding embodiment of the accompanying drawings, which will not be repeated here.

[0144] Furthermore, if the modules / units integrated into the electronic device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0145] The present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program. When the computer program is executed by a processor of an electronic device, the computer program can implement:

[0146] Acquire a fan audio signal, convert the fan audio signal into a first signal image using a first image conversion model, and convert the fan audio signal into a second signal image using a second image conversion model;

[0147] The first signal image and the second signal image are fused to obtain a signal fusion image, and a fusion feature extraction network is used to extract global signal features in the signal fusion image, and a point cloud feature extraction network is used to extract point cloud features in the signal fusion image to obtain local signal features;

[0148] The signal global features and signal local features are fused to obtain the signal fusion features, which are then input into the deep neural network to obtain the blade damage classification results.

[0149] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the module division is merely a logical function division, and other division methods may be used in actual implementation.

[0150] The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed across multiple network elements. Some or all of the modules may be selected to achieve the purpose of the solution of this embodiment according to actual needs.

[0151] In addition, the functional modules in various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or hardware plus software functional modules.

[0152] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0153] Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims rather than the foregoing description, and all changes that come within the meaning and range of equivalents of the claims are intended to be embraced therein. Any reference to a figure in a claim should not be construed as limiting the claim to which it relates.

[0154] Blockchain, as used in this article, refers to a novel application model for computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Blockchain is essentially a decentralized database, a series of data blocks generated using cryptographic methods. Each block contains information about a batch of online transactions, used to verify the validity of this information (to prevent counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, the platform product service layer, and the application service layer.

[0155] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to achieve optimal results.

[0156] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a system claim may also be implemented by a single unit or device through software or hardware. Terms such as "first" and "second" are used to indicate names and do not imply any particular order.

[0157] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A blade damage classification method based on wind turbine blade audio signals, characterized in that: The method comprises: Acquire a fan audio signal, convert the fan audio signal into a first signal image using a first image conversion model, and convert the fan audio signal into a second signal image using a second image conversion model; The first signal image and the second signal image are fused to obtain a signal fusion image, and a fusion feature extraction network is used to extract global signal features in the signal fusion image, and a point cloud feature extraction network is used to extract point cloud features in the signal fusion image to obtain local signal features; The signal global features and signal local features are fused to obtain the signal fusion features, which are then input into the deep neural network to obtain the blade damage classification results.

2. The blade damage classification method based on the wind turbine blade audio signal according to claim 1, characterized in that: The converting of the fan audio signal into a first signal image by using the first image conversion model includes: Mapping the fan audio signal to a polar coordinate system to form multiple polar coordinate points; A Gram matrix is ​​constructed through the angle relationship between the polar coordinate points, matrix elements in the Gram matrix are mapped to pixel values, and the Gram matrix is ​​converted into a first signal image.

3. The blade damage classification method based on wind turbine blade audio signals according to claim 1, characterized in that: The converting of the fan audio signal into a second signal image by using the second image conversion model includes: Convert the fan audio signal into a discrete state sequence and calculate the transition probability of each state in the discrete state sequence; The transition probabilities are aligned according to time to obtain a two-dimensional probability matrix, and the transition probability values ​​in the two-dimensional probability matrix are mapped to pixel values ​​to obtain a second signal image.

4. The blade damage classification method based on wind turbine blade audio signals according to claim 1, characterized in that: The fusing the first signal image and the second signal image to obtain a signal fusion image includes: Assigning a first weight to the Gram matrix, assigning a second weight to the two-dimensional probability matrix, normalizing the weighted Gram matrix and the weighted two-dimensional probability matrix, and performing a matrix concatenation operation after normalization to obtain a fused image feature matrix; Generate a signal fusion image based on the fusion image feature matrix.

5. The blade damage classification method based on wind turbine blade audio signals according to claim 1, characterized in that: The method of extracting global signal features from the signal fusion image using a fusion feature extraction network includes: The convolution module in the fusion feature extraction network is used to perform convolution processing on the signal fusion image to obtain a convolution feature map; Perform global pooling operation on the convolution feature map to obtain the pooled feature map; The pooled feature maps are aggregated to obtain the global signal features.

6. The blade damage classification method based on wind turbine blade audio signals according to claim 1, characterized in that: The point cloud feature extraction network is used to extract point cloud features from the signal fusion image to obtain local signal features, including: Use depth estimation to convert each pixel in the signal fusion image into a three-dimensional point; Normalize the 3D points and remove outliers to obtain the target 3D points; The point cloud feature extraction network is used to extract the attribute features of the target three-dimensional points and perform global pooling to obtain the local features of the signal.

7. The blade damage classification method based on wind turbine blade audio signals according to claim 1, characterized in that: The signal fusion feature is obtained by fusing the signal global feature and the signal local feature, including: Assigning a first initial weight to the signal global feature and a second initial weight to the signal local feature, and using the signal global feature assigned the first initial weight as a key and a value in an attention mechanism, and using the signal local feature assigned the second initial weight as a query in the attention mechanism; The local feature of the signal assigned the second initial weight is used as the query in the attention mechanism and is concatenated with the output value of the attention mechanism to obtain the first signal fusion feature; The global feature of the signal assigned the first initial weight is used as the query in the attention mechanism, and the local feature of the signal assigned the second initial weight is used as the key and value in the attention mechanism, and the global feature of the signal assigned the first initial weight is used as the query in the attention mechanism and the output value of the attention mechanism is spliced ​​to obtain the second signal fusion feature; The first signal fusion feature and the second signal fusion feature are concatenated to obtain a signal fusion feature.

8. A blade damage classification device based on wind turbine blade audio signals, characterized in that: The device can implement the blade damage classification method based on wind turbine blade audio signals according to any one of claims 1 to 7, and the device comprises: an image conversion module, configured to obtain a fan audio signal, convert the fan audio signal into a first signal image using a first image conversion model, and convert the fan audio signal into a second signal image using a second image conversion model; a feature fusion module, configured to fuse the first signal image and the second signal image to obtain a signal fusion image, and extract global signal features from the signal fusion image using a fusion feature extraction network, and extract point cloud features from the signal fusion image using a point cloud feature extraction network to obtain local signal features; The blade damage type classification module is used to fuse the global signal features and the local signal features to obtain the signal fusion features, and input the signal fusion features into the deep neural network to obtain the blade damage classification results.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the blade damage classification method based on wind turbine blade audio signals as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the blade damage classification method based on wind turbine blade audio signals according to any one of claims 1 to 7 is implemented.

Images