A method and device for determining defects in a wind turbine blade

Abstract

The application provides a kind of wind turbine blade defect determination method and device, the determination method includes: real-time monitoring to target wind turbine blade, and the panoramic image and multichannel sound signal of monitoring are acquired;The multichannel sound signal is processed, and the beam set consisting of multiple spatial direction multiple beam is determined;The sound signal of the beam set is registered and fused with the panoramic image in space, and the panoramic acoustic image is determined;The correlation of blade position in panoramic image and acoustic image extreme point is used to lock the beam of the head of target wind turbine blade from the beam set, and the target beam is determined;According to the target beam and the anomaly classifier generated by convolutional neural network training, the abnormal monitoring result of the target wind turbine blade is determined.The present scheme can overcome the interference problem of existing monitoring, maintenance cost problem, and compared with prior art can further improve the reliability and accuracy of detection result.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of wind turbine blade detection, in particular to a wind turbine blade defect determination method and device. BACKGROUND

[0002] In recent years, with the aggravation of environmental pollution and energy crisis, wind energy as a clean energy has been paid more and more attention. As of 2023, the number of wind turbines in China has ranked first in the world, and the proportion of wind power generation has exceeded 30%, which has become an important energy. Usually, wind turbines work in a poor environment, and the working conditions are complex, and there are hidden dangers or a higher probability of failure. If the failure cannot be found in time, the maintenance cost is high and causes greater economic loss. Common wind turbine failures include gear type failures, generator failures, blade failures, and tower failures. Among them, blade failure accounts for about 40%, which is often eroded by wind and rain, hail and lightning, causing leading edge corrosion, leading edge cracking and other damage.

[0003] The traditional blade detection method usually monitors through pre-installed sensors, but this method not only affects the structural strength of the wind turbine blade, but also increases the subsequent sensor maintenance cost. Another visual-based detection method is usually loaded on a gimbal or unmanned aerial vehicle platform to track and shoot the moving blade, and then identify the defect pattern. However, under insufficient light conditions, combined with the movement of the target, it is difficult to accurately detect fine cracks. SUMMARY

[0004] Therefore, the purpose of the present application is to provide a wind turbine blade defect determination method and device, which overcomes the structural interference and maintenance cost problems of traditional sensor monitoring by combining panoramic images and multi-channel sound signals, and can collect information in different spatial directions through beam sets. Even in the case of insufficient light or target movement, the blade position and sound image extreme value point can still be accurately located, thereby ensuring the detection accuracy of fine cracks, so that this fusion method can effectively improve the reliability and accuracy of detection.

[0005] The present application provides a wind turbine blade defect determination method, which comprises:

[0006] Real-time monitoring of the target wind turbine blade and obtaining the monitored panoramic image and multi-channel sound signal;

[0007] Processing the multi-channel sound signal to determine a beam set composed of multiple beams in multiple spatial directions;

[0008] Registering and fusing the sound signals of the beam set with the panoramic image in space to determine a panoramic sound image;

[0009] The target wind turbine blade head beam is locked from the beam set by using the correlation between the blade position in the panoramic image and the extreme value point of the sound image, and the target beam is determined.

[0010] According to the target beam and the anomaly classifier generated by the convolutional neural network training, the anomaly monitoring result of the target wind turbine blade is determined.

[0011] Optionally, the panoramic image is obtained by the following steps:

[0012] The monitoring image of the target wind turbine blade is obtained when each camera in the circular array camera monitors the target wind turbine blade;

[0013] The panoramic image is determined by image stitching processing of all monitoring images.

[0014] Optionally, the multi-channel sound signal is obtained by the following steps:

[0015] The initial sound signal of the area where the target wind turbine blade is located is collected by each spherical unidirectional microphone in the spherical microphone array;

[0016] The multi-channel sound signal is determined according to the initial sound signals collected by all spherical unidirectional microphones.

[0017] Optionally, the processing of the multi-channel sound signal to determine the beam set composed of multiple beams in multiple spatial directions includes:

[0018] The spherical harmonic wave domain feature is determined by frequency domain information and spatial feature extraction processing of the multi-channel sound signal.

[0019] According to the spherical harmonic wave domain feature, the spatial grid generated by the spherical uniform sampling is divided by using the beam former to determine the multiple beam in different spatial directions, and the beam set is determined according to the multiple beam.

[0020] Optionally, the processing of the multi-channel sound signal to determine the spherical harmonic wave domain feature includes:

[0021] The frequency domain information corresponding to the channel sound signal is determined by short-time Fourier transform processing of the multi-channel sound signal.

[0022] The spherical harmonic coefficient mapped with the three-dimensional space signal is determined by spherical harmonic wave domain transformation processing of the frequency domain information.

[0023] The spherical harmonic wave domain feature is determined by feature extraction processing of the spherical harmonic coefficient.

[0024] Optionally, the determining the abnormal monitoring result of the target wind turbine blade according to the target beam and the abnormality classifier generated by the convolutional neural network training comprises:

[0025] performing spatial enhancement processing on the target beam, and obtaining spherical harmonic domain features and time-frequency domain features of the spatially enhanced target beam;

[0026] inputting the spherical harmonic domain features and the time-frequency domain features of the target beam into the abnormality classifier;

[0027] The abnormality classifier performs classification prediction according to the spherical harmonic domain features and the time-frequency domain features to determine the abnormal monitoring result of the target wind turbine blade.

[0028] Optionally, the beamformer is determined by the following steps:

[0029] constructing an acoustic pressure signal model corresponding to the spherical microphone array according to the acoustic pressure signal vector corresponding to the multi-channel sound signal, the steering matrix, and the microphone noise vector;

[0030] performing spherical harmonic transformation processing on the acoustic pressure signal model to determine a spherical harmonic representation steering matrix, and updating the acoustic pressure signal model;

[0031] determining a spherical harmonic domain model according to the updated acoustic pressure signal model;

[0032] determining a spherical harmonic domain-based beamformer according to the spherical harmonic domain model and the acoustic signal time-frequency space of the multi-channel sound signal.

[0033] The embodiments of the present application also provide a wind turbine blade defect determination device, the determination device comprising:

[0034] an acquisition module configured to monitor a target wind turbine blade in real time, and acquire a panoramic image and a multi-channel sound signal of the monitoring;

[0035] a first determination module configured to process the multi-channel sound signal to determine a beam set composed of multiple beams in multiple spatial directions;

[0036] a fusion module configured to register and fuse the sound signals of the beam set with the panoramic image in space to determine a panoramic acoustic image;

[0037] a second determination module configured to lock a beam of a head of the target wind turbine blade from the beam set by using the correlation between the blade position in the panoramic image and an acoustic image extreme point, and determine a target beam;

[0038] The monitoring module is configured to determine an abnormality monitoring result of the target wind turbine blade according to the target beam and an abnormality classifier generated by training of a convolutional neural network.

[0039] Optionally, the acquisition module is further configured to acquire the panoramic image by the following steps:

[0040] acquiring monitoring images captured by each camera in the circular array camera when monitoring the target wind turbine blade;

[0041] performing image stitching processing on all the monitoring images to determine the panoramic image.

[0042] Optionally, the acquisition module is further configured to acquire the multi-channel sound signal by the following steps:

[0043] acquiring initial sound signals of the area where the target wind turbine blade is located collected by each spherical unidirectional microphone in the spherical microphone array;

[0044] determining the multi-channel sound signal according to the initial sound signals collected by all the spherical unidirectional microphones.

[0045] Optionally, when the first determining module is configured to process the multi-channel sound signal to determine a beam set composed of multiple beams in multiple spatial directions, the first determining module is configured to:

[0046] performing frequency domain information and spatial feature extraction processing on the multi-channel sound signal to determine spherical harmonic wave domain features;

[0047] dividing a spatial grid generated by spherical uniform sampling using a beamformer according to the spherical harmonic wave domain features to determine multiple beams in different spatial directions, and determining a beam set according to the multiple beams.

[0048] Optionally, when the first determining module is configured to perform frequency domain information and spatial feature extraction processing on the multi-channel sound signal to determine spherical harmonic wave domain features, the first determining module is configured to:

[0049] performing short-time Fourier transform processing on the multi-channel sound signal to determine frequency domain information corresponding to the channel sound signal;

[0050] performing spherical harmonic wave domain transformation processing on the frequency domain information to determine spherical harmonic wave coefficients mapping three-dimensional spatial signals;

[0051] performing feature extraction processing on the spherical harmonic wave coefficients to determine spherical harmonic wave domain features.

[0052] Optionally, when the monitoring module is configured to determine an abnormality monitoring result of the target wind turbine blade according to the target beam and an abnormality classifier generated by training of a convolutional neural network, the monitoring module is configured to:

[0053] performing spatial enhancement processing on the target beam, and obtaining spherical harmonic domain features and time-frequency domain features of the target beam after spatial enhancement processing;

[0054] inputting the spherical harmonic domain features and the time-frequency domain features of the target beam into the anomaly classifier;

[0055] The anomaly classifier performs classification prediction according to the spherical harmonic domain features and the time-frequency domain features, and determines the abnormal monitoring result of the target wind turbine blade.

[0056] Optionally, the first determining module is further configured to determine the beamformer by the following steps:

[0057] constructing an acoustic pressure signal model corresponding to the spherical microphone array according to the acoustic pressure signal vector corresponding to the multi-channel sound signal, the steering matrix, and the microphone noise vector;

[0058] performing spherical harmonic transformation processing on the acoustic pressure signal model, determining a spherical harmonic representation steering matrix, and updating the acoustic pressure signal model;

[0059] determining a spherical harmonic domain model according to the updated acoustic pressure signal model;

[0060] determining a spherical harmonic domain-based beamformer according to the spherical harmonic domain model and an acoustic signal time-frequency space of the multi-channel sound signal.

[0061] The embodiment of the application further provides an electronic device, including a processor, a memory and a bus, the memory stores machine readable instructions executable by the processor, when the electronic device is running, the processor and the memory communicate through the bus, and the machine readable instructions are executed by the processor to perform the steps of the determination method as described above.

[0062] The embodiment of the application further provides a computer readable storage medium, the computer readable storage medium stores a computer program, and the computer program is executed by the processor to perform the steps of the xx method as described above.

[0063] The method comprises: performing real-time monitoring on a target wind turbine blade, and acquiring a panoramic image and a multi-channel sound signal of the monitoring; processing the multi-channel sound signal to determine a beam set composed of multiple beams in multiple spatial directions; performing spatial registration and fusion of the sound signal of the beam set and the panoramic image to determine a panoramic sound image; locking a beam of a head of the target wind turbine blade from the beam set by using the correlation between the blade position in the panoramic image and a sound image extreme point, and determining a target beam; and determining an abnormal monitoring result of the target wind turbine blade according to the target beam and an anomaly classifier generated by training of a convolutional neural network.

[0064] In this way, the panoramic image and the multi-channel sound signal are combined to overcome the structural interference and maintenance cost problems of the conventional sensor monitoring, and the beam set can collect information in different spatial directions, so that the blade position and the sound image extreme point can be accurately located even in the case of insufficient light or target movement, thereby ensuring the detection accuracy of the fine cracks, and the fusion method can effectively improve the reliability and accuracy of the detection.

[0065] To make the above objectives, features and advantages of the present application more apparent, the following will describe a preferred embodiment in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0066] To more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some of the embodiments of the present application, and therefore should not be considered as a limitation to the scope, and for those skilled in the art, other related drawings can also be obtained without creative labor.

[0067] Figure 1 A flow chart of a method for determining a wind turbine blade defect provided by the embodiments of the present application;

[0068] Figure 2 A schematic diagram of a detection process of a wind turbine blade defect provided by the present application;

[0069] Figure 3 A structural schematic diagram of a determination device provided by the embodiments of the present application;

[0070] Figure 4 A structural schematic diagram of an electronic device provided by the embodiments of the present application. DETAILED DESCRIPTION

[0071] To make the purposes, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application and are not all the embodiments. The components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments of the present application, every other embodiment obtained by a person skilled in the art without creative work belongs to the scope of protection of the present application.

[0072] In recent years, with the increasing problems of environmental pollution and energy crisis, wind energy as a clean energy has been widely concerned. As of 2023, the number of wind turbines in China has ranked first in the world, and the proportion of wind power generation has exceeded 30%, which has become an important energy. Usually, wind turbines work in a poor environment, and the working conditions are complex, with a high probability of hidden faults or failures. If the fault cannot be found in time, the maintenance cost is high and the economic loss is greater. Common faults of wind turbines include gear type faults, generator faults, blade faults, and tower faults. Among them, blade faults account for about 40%, which are often eroded by wind and rain, hail, and lightning, causing leading edge corrosion, leading edge cracking, and surface cracking.

[0073] Traditional blade detection methods usually monitor through pre-installed sensors, but this method not only affects the structural strength of the fan blades, but also increases the subsequent sensor maintenance cost. Another visual-based detection method is usually loaded on a gimbal or unmanned aerial vehicle platform to track and shoot the moving blades, and then identify the defect patterns. However, under insufficient light conditions, combined with the movement of the target, it is difficult to accurately detect fine cracks.

[0074] In addition, there is also a monitoring method based on blade voiceprints, which evaluates the state of the blade by detecting the noise characteristics generated by the blade surface defects as the blade rotates. Due to the advantages of non-contact, high efficiency, convenient installation, low cost, etc., this method has received widespread attention in recent years. However, the strong wind noise interference of the wind farm can seriously affect the accurate identification of voiceprint features, making fault detection still face challenges.

[0075] To solve the above technical problems, the existing patent "Fan fault detection method and system based on AI auscultation, fan safety system" (CN115163426 A) obtains fan blade noise signals through a single-channel microphone, and then measures the similarity with the prior knowledge base based on time domain, frequency domain, voiceprint and frequency band energy and other characteristics to determine whether there is a fault. However, since this method uses an omnidirectional sound signal acquisition method, it inevitably introduces background noise in all directions, affecting the extraction quality of the sound signal. In addition, the similarity-based algorithm has poor generalization ability, which may affect the performance of subsequent fault diagnosis.

[0076] The existing patent "Fan blade remote auscultation method based on acoustic diagnosis" (CN112067701 A) obtains blade wind sweeping sound through a single channel, mainly uses an adaptive method to extract octave band features, and detects blade state through an SVDD algorithm. Although this method has improved in signal extraction, it is still difficult to remove strong environmental background noise in the sound signal acquisition process. In addition, the SVDD-based detection model has high computational complexity when processing high-dimensional data, the sphere boundary has limitations, and lacks multi-classification ability, making it difficult to meet the needs of blade fault detection for generalization and scalability.

[0077] The existing patent "Offshore wind turbine blade fault non-contact monitoring system and monitoring method" (CN109763944A) obtains aerodynamic acoustic signals through an offshore meteorological and hydrological node, and then analyzes the fan blade wind sweeping signal through a neural network to determine whether there is an anomaly. However, this method has poor performance in sound signal acquisition capability, and the subsequent sound signal enhancement algorithm has not been improved, which seriously affects the effect of fault detection.

[0078] Another patent "Fan blade fault acoustic detection method and system based on deep learning" (CN116631442A) obtains sound signals through a microphone array, uses beamforming to filter and enhance the target signal in the spatial domain, and then uses a deep learning method to classify the Mel spectrum features to realize anomaly detection. Although this method enhances the blade noise extraction capability through linear array signal processing, the linear array has inconsistent directivity in space, which increases the difficulty of array deployment and affects the performance of spatial filtering enhancement, ultimately leading to a decline in blade fault detection performance.

[0079] Based on this, the embodiments of the present application provide a wind turbine blade defect determination method and device, which can overcome the interference problem of existing monitoring, the maintenance cost problem, and further improve the reliability and accuracy of the detection result compared with the prior art.

[0080] Please refer to Figure 1 , Figure 1A flowchart of a method for determining defects of a wind turbine blade is provided in embodiments of the present application. As shown in Figure 1 The determining method provided in embodiments of the present application includes:

[0081] S101, real-time monitoring of a target wind turbine blade is performed, and panoramic images and multi-channel sound signals are acquired.

[0082] S102, the multi-channel sound signals are processed to determine a beam set composed of multiple beams from multiple spatial directions.

[0083] S103, the sound signals of the beam set are spatially registered and fused with the panoramic images to determine a panoramic acoustic image.

[0084] S104, the correlation between the blade position in the panoramic image and the acoustic image extreme point is used to lock the beam of the head of the target wind turbine blade from the beam set, and a target beam is determined.

[0085] S105, according to the target beam and an anomaly classifier generated by convolutional neural network training, an anomaly monitoring result of the target wind turbine blade is determined.

[0086] For step S101, the target wind turbine blade is monitored in real time, and panoramic images and multi-channel sound signals are collected. The panoramic images can provide visual information of the blade, and the multi-channel sound signals can capture sound wave information of the blade in different spatial directions.

[0087] The multi-channel sound signals refer to sound signals collected simultaneously from multiple directions by a microphone array.

[0088] In an embodiment provided in the present application, the panoramic images are acquired by the following steps: acquiring monitoring images of the target wind turbine blade monitored by each camera in a circular array camera; and performing image stitching processing on all monitoring images to determine the panoramic images.

[0089] In this way, the circular array camera is composed of multiple cameras that simultaneously monitor the blade of the target wind turbine, and each camera independently captures the blade to obtain monitoring images, which can capture information at different angles and fields of view.

[0090] All acquired monitoring images are subjected to image stitching, i.e., the independent images are combined into a whole panoramic image, wherein the stitching processing generally includes image alignment, identification of overlapping areas, and seamless connection processing.

[0091] In another implementation provided in the application, the multi-channel sound signal is obtained by the following steps: obtaining initial sound signals of the area where the target wind turbine blade is located collected by each spherical unidirectional microphone in the spherical microphone array; and determining the multi-channel sound signal according to the initial sound signals collected by all the spherical unidirectional microphones.

[0092] Here, the spherical microphone array includes a plurality of spherical unidirectional microphones. In the spherical microphone array, each spherical unidirectional microphone independently collects initial sound signals of the area where the target wind turbine blade is located.

[0093] All the initial sound signals collected by the spherical unidirectional microphones are integrated to form a comprehensive multi-channel sound signal. This signal contains sound information collected from different spatial directions and can provide more comprehensive sound wave data.

[0094] Among them, the spherical microphone array and the circular array camera can be combined and installed, or can be installed independently.

[0095] For step S102, in this step, the obtained multi-channel sound signal is processed and decomposed into a plurality of sound beams collected from different spatial directions to obtain a beam set. These beams can reflect the sound wave information generated by the blade in different directions.

[0096] In an implementation provided in the application, the processing of the multi-channel sound signal to determine the beam set composed of a plurality of spatial direction beams includes:

[0097] S1021, performing frequency domain information and spatial feature extraction processing on the multi-channel sound signal to determine spherical harmonic wave domain features.

[0098] S1022, according to the spherical harmonic wave domain features, using a beamformer to divide the spatial grid generated by uniform sampling on the sphere to determine a plurality of beams in different spatial directions, and determining a beam set according to the plurality of beams.

[0099] For step S1021, in this step, the multi-channel sound signal is converted from the time domain to the frequency domain, the performance of the sound at different frequencies is analyzed, and the spatial distribution characteristics of the sound source are obtained. The spherical harmonic wave domain features reflecting the characteristics of the sound field, i.e., the sound wave distribution on the sphere, can be obtained.

[0100] In an implementation provided in the application, the frequency domain information and spatial feature extraction processing on the multi-channel sound signal to determine the spherical harmonic wave domain features includes:

[0101] S10211, performing short-time Fourier transform processing on the multi-channel sound signal to determine the frequency domain information corresponding to the channel sound signal.

[0102] S10212, performing spherical harmonic domain transformation on the frequency domain information to determine spherical harmonic coefficients of the three-dimensional space signal.

[0103] S10213, performing feature extraction on the spherical harmonic coefficients to determine spherical harmonic domain features.

[0104] For step S10211, in this step, the multi-channel sound signal is subjected to short-time Fourier transform (STFT) processing to convert the sound signal in the time domain to the frequency domain.

[0105] The process of performing short-time Fourier transform (STFT) on the multi-channel sound signal is as follows: the time-domain signal of each microphone is divided into multiple windows (each window has a fixed length and has an overlap). The Fourier transform is performed on the signal in each window to obtain the frequency spectrum information of the signal.

[0106] For step S10212, the extracted frequency domain information is subjected to spherical harmonic domain transformation to decompose the sound signal in the three-dimensional space into a set of spherical harmonic coefficients.

[0107] The spherical harmonic domain transformation on the extracted frequency domain information includes expanding the signal at each frequency point by a spherical harmonic basis function to obtain the spherical harmonic coefficients.

[0108] For step S10213, the spherical harmonic domain features can reflect important patterns and feature points of the sound signal in the spherical harmonic domain.

[0109] For step S1022, based on the extracted spherical harmonic domain features, a beamformer is used to generate a spatial grid uniformly sampled on a sphere. Each grid represents a spatial direction, and a plurality of beams are generated by focusing on the signal in each direction. These beams reflect the distribution of the sound signal in different spatial directions. Finally, all the beams form a beam set, i.e., a collection of sound signals in multiple spatial directions, which is used for subsequent monitoring and analysis.

[0110] Through steps S1021 and S1022, the conversion from the multi-channel sound signal to the multi-directional beam can be realized, so that the sound information in different spatial directions is accurately captured, which helps to improve the spatial resolution and accuracy of the wind turbine blade defect detection.

[0111] For the beamformer in step S1022, in an embodiment provided by the present application, the beamformer is determined by the following steps:

[0112] S201, constructing an acoustic pressure signal model corresponding to the spherical microphone array according to the acoustic pressure signal vector corresponding to the multi-channel sound signal, the steering matrix, and the microphone noise vector.

[0113] S202, performing spherical harmonic transform on the sound pressure signal model, determining a steering matrix of spherical harmonic representation, and updating the sound pressure signal model.

[0114] S203, determining a spherical harmonic domain model according to the updated sound pressure signal model.

[0115] S204, determining a spherical harmonic domain based beamformer according to the spherical harmonic domain model and a sound signal time-frequency space of the multi-channel sound signal.

[0116] For steps S201 to S204, the generation process of the beamformer is specifically exemplarily illustrated by the following steps.

[0117] A rigid spherical microphone array acquires sound signals, Q single-direction microphones are uniformly distributed on a rigid sphere with a radius of r, there are L far-field sound sources in space, and a rigid spherical sound propagation model is obtained. The spatial coordinates of the microphones are:

[0118] r q =[rcos(θ q )sin(φ q ),rsin(θ q )sin(φ q ),rcos(φ q )] T

[0119] Where θ q is the direction measured counterclockwise from the x-axis, and φ q is the elevation angle measured downward from the z-axis.

[0120] Similarly, the coordinates of the L sound sources are:

[0121] r l =[rcos(θ l )sin(φ l ),rsin(θ l )sin(φ l ),rcos(φ l )] T , l=1,…,L.

[0122] The received signal model of the spherical microphone array can be represented as:

[0123] p(k)=A(k)S(k)+z(k)

[0124] where k is the wave number, i.e., k = 2πf / c, f is the frequency, and c is the sound speed. p(k) is the sound pressure signal vector, A(k) is the steering matrix, S(k) is the sound source sound pressure signal vector, and z(k) is the sensor noise vector. p(k), A(k), and S(k) are expressed as:

[0125] p(k) = [p(k, r1), p(k, r2),..., p(k, rN)]T Q T ,

[0126] A(k) = [a(k, r1), a(k, r2),..., a(k, rN)]T L T ,

[0127] S(k) = [s1(k), s2(k),..., sN(k)]T L T

[0128] where p(k, r q ) is the sound signal received by the qth microphone, a(k, r l ) and s l (k) are the steering vector and the sound pressure intensity of the lth sound source, respectively. a(k, r l ) is:

[0129]

[0130] where k l is

[0131] k l = -k [cos(θ l ) sin(φ l ), sin(θ l ) sin(φ l ), cos(φ l )] T i 2 = -1

[0132] According to the spherical harmonic transform, can be approximately decomposed into truncated N-order spherical harmonics:

[0133]

[0134] where b n (kr) is the mode intensity for the unidirectional microphone:

[0135] b n (kr) = 4πi n [j n (kr) - ij' n (kr)],​​​​​​

[0136] where is a spherical harmonic basis function:

[0137]

[0138] is a Legendre function, and j n is a first kind spherical Bessel function. Thus, the steering vector can be expressed as:

[0139] A(k) = Y(Ω)B(kr)Y H (Γ),

[0140] where Y(Ω) is a Q x (N + 1) 2 matrix:

[0141] Y(Ω) = [y T (θ1, φ1),..., y T (θ Q , φ Q )] T ,

[0142]

[0143] From the L sound source directions, a L x (N + 1) 2 matrix Y H (Γ) can be obtained:

[0144] Y H (Γ) = [y T (θ1, φ1),..., y T (θ L , φ L )]

[0145] and a (N + 1) 2 x (N + 1) 2 diagonal matrix B(kr):

[0146] B(kr) = diag{b0(kr), b1(kr), b1(kr),..., b N (kr)}.

[0147] Thus, we can obtain

[0148] p(k) = Y(Ω)B(kr)Y H (Γ)S(k) + z(k)

[0149] Due to the orthogonality principle of spherical harmonics, for the approximate uniform spherical surface sampling, Y H (Ω)Y(Ω) = I, we can obtain p nm (k) as:

[0150] p nm (k) = B(kr)Y H (Γ)S(k) + z nm (k)

[0151] where p nm (k) = Y H (Ω)p(k). For a given spherical array configuration, we can obtain the spherical harmonic eigen-coefficients q nm (k):

[0152]

[0153] where q nm (k) = B -1 (kr)p nm (k), where it is worth noting that the new steering vector Y H (Γ) does not contain the frequency component, and thus can directly process coherent sources without the need for a frequency-domain focusing transform. For broadband sources, the acoustic imaging under reverberation conditions can be achieved by smoothing in a selected time-frequency domain through a short-time Fourier transform (STFT). Thus, the spherical harmonic domain model can be expressed as:

[0154]

[0155] where E is the time-frequency space of the acoustic signal. Further, we can obtain the beamformer based on the spherical harmonic domain:

[0156]

[0157] where N τ and N v are the number of time and frequency frames, respectively.

[0158] For step S103, here, the spatial distribution and energy information of the acoustic source can be visualized on the panoramic image based on the beamforming technology and image processing technology, i.e., the beam set is fused with the panoramic image to determine the panoramic acoustic image.

[0159] where the fusion of the beam set with the panoramic image mainly includes direction matching and energy mapping processing.

[0160] For step S104, for example, according to the blade position in the panoramic image and the extreme point (extreme position of the acoustic signal) in the acoustic image, the most relevant acoustic signal beam is found, the beam of the blade head is locked from the beam set, i.e., the target beam is determined, and the sound characteristics of the blade head are also accurately positioned.

[0161] where the target beam is dynamically determined. The blade head position is used for real-time dynamic determination.

[0162] For step S105, in this step, the acoustic signal of the target beam is analyzed by a pre-trained convolutional neural network (CNN) anomaly classifier using the target beam determined in step S104, and it is determined whether the blade is abnormal.

[0163] In an embodiment provided by the present application, the abnormal monitoring result of the target wind turbine blade is determined according to the target beam and an anomaly classifier generated by training of a convolutional neural network, and the method comprises the following steps:

[0164] S1051, performing spatial domain enhancement processing on the target beam, and obtaining spherical harmonic domain features and time-frequency domain features of the target beam after spatial domain enhancement.

[0165] S1052, inputting the spherical harmonic domain features and time-frequency domain features of the target beam into the anomaly classifier.

[0166] S1053, the anomaly classifier performs classification prediction according to the spherical harmonic domain features and the time-frequency domain features, and determines the abnormal monitoring result of the target wind turbine blade.

[0167] For step S1051, the target beam is subjected to spatial domain enhancement processing, that is, the signal is processed in the spatial dimension to improve the features related to the spatial distribution in the signal.

[0168] For step S1052, the anomaly classifier is generated by training of a convolutional neural network. The size of the convolution kernel of the convolutional neural network can be (N+1) 2 / 2×1.

[0169] For step S1052, when the anomaly classifier performs classification prediction according to the spherical harmonic domain features and the time-frequency domain features to determine the abnormal monitoring result of the target wind turbine blade, it can specifically include: after N_layer layers of convolution and rectified linear unit (ReLU) activation function processing, the number of convolution channels of each layer is C1,…,C L , and finally the acoustic feature of N_Feats dimension is obtained. Finally, the multi-layer perception network (MLP) classifier is used to classify the abnormal / normal state of the blade.

[0170] That is, the abnormal monitoring result includes two classification results of normal and abnormal.

[0171] For example, please refer to Figure 2 , Figure 2 A schematic diagram of a wind turbine blade defect detection process is provided in the present application. As shown in Figure 2As shown, Figure 2 The process of image and sound acquisition, data processing and monitoring result generation is recorded in the database, and the specific implementation is as described in steps S101-S105, which will not be repeated here.

[0172] In this way, the sound signal is acquired by the spherical unidirectional microphone, and the special spatial symmetry structure can realize the spatial consistency of directivity. At the same time, the spatial information is accurately acquired by combining the panoramic sound and light imaging technology, and the blade wind sweeping signal is accurately extracted by the beam spatial filtering method. Finally, the blade defects are identified by means of deep neural network. In this way, not only the anti-interference ability is enhanced, but also the panoramic detection method does not need to adjust the posture, and a more convenient inspection method is provided. And this method can capture the blade wind sweeping signal in all directions, and can accurately acquire the target sound source signal without adjusting the array posture, and with the aid of image information to assist noise source positioning, so as to quickly identify the sound source. And because the spherical array has a special symmetrical structure, the spatial directivity is consistent in all directions, this scheme not only can accurately lock the sound source through the beam, but also improves the anti-interference performance of the system.

[0173] Based on the same inventive concept, the determination method corresponding to the determination method is also provided in the embodiments of the present application. Since the principle of the device in the embodiments of the present application solves the problem, the implementation of the device can be referred to the implementation of the method, and the repeated parts will not be repeated.

[0174] Please refer to Figure 3 , Figure 3 The structure schematic diagram of a determination device provided by the embodiments of the present application is shown in FIG. 3. Figure 3 As shown in FIG. 3, the determination device 300 comprises:

[0175] The acquisition module 310 is configured to perform real-time monitoring on the target wind turbine blade, and acquire a panoramic image and a multi-channel sound signal of the monitoring;

[0176] The first determination module 320 is configured to process the multi-channel sound signal, and determine a beam set composed of multiple beams of multiple spatial directions;

[0177] The fusion module 330 is configured to register and fuse the sound signal of the beam set with the panoramic image in space, and determine a panoramic sound image;

[0178] The second determination module 340 is configured to lock the beam of the head of the target wind turbine blade from the beam set by using the correlation between the blade position in the panoramic image and the extreme value point of the sound image, and determine a target beam;

[0179] The monitoring module 350 is configured to determine an abnormal monitoring result of the target wind turbine blade according to the target beam and an abnormality classifier generated by training a convolutional neural network.

[0180] Optionally, the acquisition module 310 is further configured to acquire the panoramic image by the following steps:

[0181] acquiring a monitoring image of each camera in the circular array camera when monitoring the target wind turbine blade;

[0182] performing image stitching processing on all monitoring images to determine the panoramic image.

[0183] Optionally, the acquisition module 310 is further configured to acquire the multi-channel sound signal by the following steps:

[0184] acquiring an initial sound signal of the area where the target wind turbine blade is located collected by each spherical unidirectional microphone in the spherical microphone array;

[0185] determining the multi-channel sound signal according to the initial sound signals collected by all spherical unidirectional microphones.

[0186] Optionally, when the first determination module 320 is configured to process the multi-channel sound signal to determine a beam set composed of multiple beams of different spatial directions, the first determination module 320 is configured to:

[0187] performing frequency domain information and spatial feature extraction processing on the multi-channel sound signal to determine spherical harmonic domain features;

[0188] dividing a spatial grid generated by spherical uniform sampling using a beamformer according to the spherical harmonic domain features to determine multiple beams of different spatial directions, and determining a beam set according to the multiple beams.

[0189] Optionally, when the first determination module 310 is configured to perform frequency domain information and spatial feature extraction processing on the multi-channel sound signal to determine spherical harmonic domain features, the first determination module 310 is configured to:

[0190] performing short-time Fourier transform processing on the multi-channel sound signal to determine frequency domain information corresponding to the channel sound signal;

[0191] performing spherical harmonic domain transformation processing on the frequency domain information to determine spherical harmonic coefficients mapping three-dimensional space signals;

[0192] performing feature extraction processing on the spherical harmonic coefficients to determine spherical harmonic domain features.

[0193] Optionally, when the monitoring module 350 is configured to determine the abnormal monitoring result of the target wind turbine blade according to the target beam and the abnormality classifier trained by the convolutional neural network, the monitoring module 350 is configured to:

[0194] Performing spatial domain enhancement processing on the target beam, and obtaining spherical harmonic domain features and time-frequency domain features of the target beam after spatial domain enhancement;

[0195] Inputting the spherical harmonic domain features and time-frequency domain features of the target beam into the anomaly classifier;

[0196] The abnormality classifier performs classification prediction based on the spherical harmonic domain characteristics and the time-frequency domain characteristics to determine the abnormality monitoring result of the target wind turbine blade.

[0197] Optionally, the first determining module 320 is further configured to determine a beamformer through the following steps:

[0198] Constructing a sound pressure signal model corresponding to the spherical microphone array according to the sound pressure signal vector, the steering matrix, and the microphone noise vector corresponding to the multi-channel acoustic signal;

[0199] performing spherical harmonic transformation on the sound pressure signal model, determining a steering matrix represented by the spherical harmonics, and updating the sound pressure signal model;

[0200] Determining a spherical harmonic domain model according to the updated sound pressure signal model;

[0201] A beamformer based on the spherical harmonics domain is determined according to the spherical harmonics domain model and the acoustic signal time-frequency space of the multi-channel acoustic signal.

[0202] See also Figure 4 , Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 4 As shown in FIG, the electronic device 400 includes a processor 410 , a memory 420 and a bus 430 .

[0203] The memory 420 stores machine-readable instructions executable by the processor 410. When the electronic device 400 is running, the processor 410 communicates with the memory 420 via the bus 430. When the machine-readable instructions are executed by the processor 410, the above-mentioned Figure 1 as well as Figure 2 The specific implementation of the steps in the method embodiment shown can be found in the method embodiment and will not be repeated here.

[0204] The embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the computer program can execute the above-mentioned Figure 1 as well as Figure 2 The specific implementation of the steps in the method embodiment shown can be found in the method embodiment and will not be repeated here.

[0205] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system, device and unit described above can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0206] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some communication interfaces, devices or units, and can be electrical, mechanical or other forms.

[0207] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0208] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit.

[0209] If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a nonvolatile computer readable storage medium executable by a processor. Based on this understanding, the technical solutions of the present application or the essential part or part of the technical solutions that make contributions to the prior art can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The foregoing storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and various program codes that can be stored in the medium.

[0210] Finally, it should be noted that the above-described embodiments are merely specific embodiments of the present application, which are used to illustrate the technical solutions of the present application, but not to limit the same. The protection scope of the present application is not limited thereto. Although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those skilled in the art that any skilled person in the art can still modify or easily think of changes to the technical solutions recorded in the foregoing embodiments, or make equivalent replacements to some of the technical features, within the technical scope disclosed by the present application. The modifications, changes or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method of determining defects in a wind turbine blade, characterized by, The determination method comprises: Real-time monitoring of the target wind turbine blade and obtaining a panoramic image and a multi-channel sound signal of the monitoring; Processing the multi-channel sound signal to determine a beam set composed of multiple beams of multiple spatial directions; Spatially registering and fusing the sound signal of the beam set with the panoramic image to determine a panoramic sound image; Using the correlation between the blade position in the panoramic image and the extreme value point of the sound image to lock the beam of the head of the target wind turbine blade from the beam set to determine a target beam; According to the target beam and an abnormality classifier generated by training of a convolutional neural network, determining an abnormality monitoring result of the target wind turbine blade; The processing of the multi-channel sound signal to determine a beam set composed of multiple beams of multiple spatial directions comprises: Performing frequency domain information and spatial feature extraction processing on the multi-channel sound signal to determine spherical harmonic domain features; According to the spherical harmonic domain features, using a beamformer to divide a spatial grid generated by uniform sampling of a sphere to determine multiple beams of different spatial directions, and determining a beam set according to the multiple beams; The frequency domain information and spatial feature extraction processing on the multi-channel sound signal to determine spherical harmonic domain features comprises: Performing short-time Fourier transform processing on the multi-channel sound signal to determine frequency domain information corresponding to the channel sound signal; Performing spherical harmonic domain transformation processing on the frequency domain information to determine spherical harmonic coefficients mapping three-dimensional space signals; Performing feature extraction processing on the spherical harmonic coefficients to determine spherical harmonic domain features; The beamformer is determined by the following steps: According to a sound pressure signal vector corresponding to the multi-channel sound signal, a steering matrix, and a microphone noise vector, constructing a sound pressure signal model corresponding to a spherical microphone array; Performing spherical harmonic transformation processing on the sound pressure signal model to determine a spherical harmonic representation steering matrix, and updating the sound pressure signal model; According to the updated sound pressure signal model, determining a spherical harmonic domain model; According to the spherical harmonic domain model and a sound signal time-frequency space of the multi-channel sound signal, determining a spherical harmonic domain-based beamformer.

2. The determination method according to claim 1, characterized in that, The panoramic image is obtained by the following steps: Obtaining monitoring images captured by each camera in a circular array camera when monitoring the target wind turbine blade; Performing image stitching processing on all the monitoring images to determine the panoramic image.

3. The determination method according to claim 1, characterized in that, The multi-channel sound signal is obtained by the following steps: Obtaining initial sound signals collected by each spherical unidirectional microphone in a spherical microphone array in an area where the target wind turbine blade is located; According to the initial sound signals collected by all the spherical unidirectional microphones, determining the multi-channel sound signal.

4. The determination method according to claim 1, characterized in that, The determination of the abnormality monitoring result of the target wind turbine blade according to the target beam and the abnormality classifier generated by training of a convolutional neural network comprises: Performing spatial domain enhancement processing on the target beam, and obtaining spherical harmonic domain features and time-frequency domain features of the target beam after spatial domain enhancement; Inputting the spherical harmonic domain features and the time-frequency domain features of the target beam into the abnormality classifier; The anomaly classifier performs classification prediction according to the spherical harmonic domain feature and the time-frequency domain feature, and determines the abnormal monitoring result of the target wind turbine blade.

5. An apparatus for determining defects in a wind turbine blade, the apparatus comprising: The determining device comprises: An acquisition module is configured to monitor a target wind turbine blade in real time, and acquire a panoramic image and a multi-channel sound signal of the monitoring; A first determining module is configured to process the multi-channel sound signal, and determine a beam set composed of multiple beams of different spatial directions; A fusion module is configured to register and fuse the sound signal of the beam set with the panoramic image in space, and determine a panoramic sound image; A second determining module is configured to lock a beam of a head of the target wind turbine blade from the beam set by using the correlation between the blade position in the panoramic image and the extreme value point of the sound image, and determine a target beam; A monitoring module is configured to determine an abnormal monitoring result of the target wind turbine blade according to the target beam and an anomaly classifier trained by a convolutional neural network. When the first determining module is configured to process the multi-channel sound signal, and determine a beam set composed of multiple beams of different spatial directions, the first determining module is configured to: perform frequency domain information and spatial feature extraction processing on the multi-channel sound signal, and determine a spherical harmonic domain feature; divide a spatial grid generated by uniform sampling of a sphere using a beamformer according to the spherical harmonic domain feature, determine multiple beams of different spatial directions, and determine a beam set according to the multiple beams; When the first determining module is configured to perform frequency domain information and spatial feature extraction processing on the multi-channel sound signal, and determine a spherical harmonic domain feature, the first determining module is configured to: perform short-time Fourier transform processing on the multi-channel sound signal, and determine frequency domain information corresponding to the channel sound signal; perform spherical harmonic domain transformation processing on the frequency domain information, and determine spherical harmonic coefficients mapping a three-dimensional space signal; perform feature extraction processing on the spherical harmonic coefficients, and determine a spherical harmonic domain feature; The first determining module is further configured to determine a beamformer by the following steps: construct an acoustic pressure signal model corresponding to a spherical microphone array according to an acoustic pressure signal vector corresponding to the multi-channel sound signal, a steering matrix, and a microphone noise vector; perform spherical harmonic transformation processing on the acoustic pressure signal model, determine a spherical harmonic representation steering matrix, and update the acoustic pressure signal model; determine a spherical harmonic domain model according to the updated acoustic pressure signal model; determine a spherical harmonic domain-based beamformer according to the spherical harmonic domain model and a sound signal time-frequency space of the multi-channel sound signal.

6. An electronic device, comprising: It comprises: a processor, a memory, and a bus, the memory stores machine readable instructions executable by the processor, when the electronic device is running, the processor and the memory communicate through the bus, the machine readable instructions are executed by the processor to perform the steps of the determination method of any one of claims 1 to 4.

7. A computer readable storage medium characterized in that, The computer readable storage medium stores a computer program, which is executed by the processor to perform the steps of the determination method of any one of claims 1 to 4.

Images