Determination method and device for blade defects of wind turbine generator

Abstract

The invention provides a wind turbine generator blade defect determination method and device, and the method comprises the steps: carrying out the real-time monitoring of a target wind turbine generator blade, and obtaining a monitored panoramic image and a multi-channel sound signal; processing the multi-channel sound signal, and determining a beam set composed of multiple beams of beams in multiple space directions; performing spatial registration and fusion on the sound signals of the beam set and the panoramic image to determine a panoramic sound image; by using the relevance between the blade position in the panoramic image and the sound image extreme point, locking the wave beam of the blade head of the target wind turbine generator from the wave beam set, and determining a target wave beam; and according to the target wave beam and an anomaly classifier generated by convolutional neural network training, determining an anomaly monitoring result of the target wind turbine generator blade. According to the scheme, the interference problem and the maintenance cost problem of existing monitoring can be solved, and compared with the prior art, the reliability and the accuracy of a detection result can be further improved.

Description

Technical Field

[0001] The present application relates to the technical field of wind turbine blade detection, and in particular to a method and device for determining defects in wind turbine blades. Background Art

[0002] In recent years, with the worsening of environmental pollution and the energy crisis, wind power has garnered increasing attention as a clean energy source. By 2023, my country had the largest number of wind turbines in the world, accounting for over 30% of total wind power generation, making it a vital energy source. Wind turbines typically operate in harsh environments and face complex operating conditions, creating potential for failure or a high probability of occurrence. Failure to detect failures promptly can lead to high repair costs and even greater economic losses. Common wind turbine failures include gear failures, generator failures, blade failures, and tower failures. Blade failures account for approximately 40% of all wind turbine failures. Blades are often damaged by wind, rain, hail, and lightning, resulting in damage such as leading edge corrosion, cracking, and surface fissures.

[0003] Traditional blade inspection methods typically use pre-installed sensors for monitoring, but this approach not only compromises the structural strength of wind turbine blades but also increases subsequent sensor maintenance costs. Another vision-based inspection method, typically mounted on a gimbal or drone platform, tracks and images the blades before identifying defect patterns. However, in low light conditions and due to target motion, accurate detection of fine cracks is difficult. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a method and device for determining defects in wind turbine blades. By combining panoramic images and multi-channel acoustic signals, the structural interference and maintenance cost problems of traditional sensor monitoring are overcome, and the beam set can collect information in different spatial directions. Even in insufficient light or moving targets, the blade position and its acoustic and image extreme points can still be accurately located, thereby ensuring the detection accuracy of fine cracks. Therefore, this fusion method can effectively improve the reliability and accuracy of detection.

[0005] The present invention provides a method for determining defects in wind turbine blades. The method includes:

[0006] Monitor the target wind turbine blades in real time and obtain panoramic images and multi-channel acoustic signals;

[0007] Processing the multi-channel acoustic signal to determine a beam set consisting of multiple beams in multiple spatial directions;

[0008] spatially registering and fusing the acoustic signal of the beam set with the panoramic image to determine a panoramic sound image;

[0009] Using the correlation between the blade position in the panoramic image and the extreme value point of the acoustic image, the beam at the head of the blade of the target wind turbine is locked from the beam set to determine the target beam;

[0010] An abnormality monitoring result of the target wind turbine blade is determined according to the target beam and an abnormality classifier generated by convolutional neural network training.

[0011] Optionally, obtain a panoramic image by performing the following steps:

[0012] Acquire a monitoring image captured by each camera in the circular array of cameras when monitoring the blades of the target wind turbine;

[0013] Perform image stitching processing on all monitoring images to determine the panoramic image.

[0014] Optionally, obtain a multi-channel acoustic signal by performing the following steps:

[0015] Acquiring an initial acoustic signal collected by each spherical unidirectional microphone in the spherical microphone array in the area where the target wind turbine blade is located;

[0016] The multi-channel acoustic signal is determined according to the initial acoustic signals collected by all spherical unidirectional microphones.

[0017] Optionally, processing the multi-channel acoustic signal to determine a beam set consisting of multiple beams in multiple spatial directions includes:

[0018] Performing frequency domain information and spatial feature extraction processing on the multi-channel acoustic signal to determine spherical harmonic domain features;

[0019] According to the spherical harmonic domain characteristics, a beamformer is used to divide the space grid generated by uniform spherical sampling, multiple beams in different spatial directions are determined, and a beam set is determined based on the multiple beams.

[0020] Optionally, performing frequency domain information and spatial feature extraction processing on the multi-channel acoustic signal to determine spherical harmonic domain features includes:

[0021] Performing short-time Fourier transform processing on the multi-channel acoustic signal to determine frequency domain information corresponding to the channel acoustic signal;

[0022] Performing spherical harmonic domain transformation on the frequency domain information to determine spherical harmonic coefficients mapped with three-dimensional spatial signals;

[0023] The spherical harmonic coefficients are subjected to feature extraction processing to determine spherical harmonic domain features.

[0024] Optionally, determining the abnormality monitoring result of the target wind turbine blade according to the target beam and an abnormality classifier generated by convolutional neural network training includes:

[0025] Performing spatial domain enhancement processing on the target beam, and obtaining spherical harmonic domain features and time-frequency domain features of the target beam after spatial domain enhancement;

[0026] Inputting the spherical harmonic domain features and time-frequency domain features of the target beam into the anomaly classifier;

[0027] The abnormality classifier performs classification prediction based on the spherical harmonic domain characteristics and the time-frequency domain characteristics to determine the abnormality monitoring result of the target wind turbine blade.

[0028] Optionally, determine the beamformer by following these steps:

[0029] Constructing a sound pressure signal model corresponding to the spherical microphone array according to the sound pressure signal vector, the steering matrix, and the microphone noise vector corresponding to the multi-channel acoustic signal;

[0030] performing spherical harmonic transformation on the sound pressure signal model, determining a steering matrix represented by the spherical harmonics, and updating the sound pressure signal model;

[0031] Determining a spherical harmonic domain model according to the updated sound pressure signal model;

[0032] A beamformer based on the spherical harmonics domain is determined according to the spherical harmonics domain model and the acoustic signal time-frequency space of the multi-channel acoustic signal.

[0033] The embodiment of the present application further provides a device for determining defects in blades of a wind turbine generator set, the device comprising:

[0034] An acquisition module is used to monitor the blades of the target wind turbine in real time and obtain the monitored panoramic images and multi-channel acoustic signals;

[0035] a first determining module, configured to process the multi-channel acoustic signal and determine a beam set consisting of multiple beams in multiple spatial directions;

[0036] a fusion module, configured to spatially register and fuse the acoustic signals of the beam set with the panoramic image to determine a panoramic sound image;

[0037] A second determination module is configured to utilize the correlation between the blade position in the panoramic image and the extreme value point of the sound image to lock the beam of the blade head of the target wind turbine from the beam set to determine the target beam;

[0038] A monitoring module is used to determine an abnormality monitoring result of the target wind turbine blade based on the target beam and an abnormality classifier generated by convolutional neural network training.

[0039] Optionally, the acquisition module is further configured to acquire a panoramic image through the following steps:

[0040] Acquire a monitoring image captured by each camera in the circular array of cameras when monitoring the blades of the target wind turbine;

[0041] Perform image stitching processing on all monitoring images to determine the panoramic image.

[0042] Optionally, the acquisition module is further configured to acquire a multi-channel acoustic signal through the following steps:

[0043] Acquiring an initial acoustic signal collected by each spherical unidirectional microphone in the spherical microphone array in the area where the target wind turbine blade is located;

[0044] The multi-channel acoustic signal is determined according to the initial acoustic signals collected by all spherical unidirectional microphones.

[0045] Optionally, when the first determining module is used to process the multi-channel acoustic signal to determine a beam set consisting of multiple beams in multiple spatial directions, the first determining module is used to:

[0046] Performing frequency domain information and spatial feature extraction processing on the multi-channel acoustic signal to determine spherical harmonic domain features;

[0047] According to the spherical harmonic domain characteristics, a beamformer is used to divide the space grid generated by uniform spherical sampling, multiple beams in different spatial directions are determined, and a beam set is determined based on the multiple beams.

[0048] Optionally, when the first determining module is used to perform frequency domain information and spatial feature extraction processing on the multi-channel acoustic signal to determine the spherical harmonic domain features, the first determining module is used to:

[0049] Performing short-time Fourier transform processing on the multi-channel acoustic signal to determine frequency domain information corresponding to the channel acoustic signal;

[0050] Performing spherical harmonic domain transformation on the frequency domain information to determine spherical harmonic coefficients mapped with three-dimensional spatial signals;

[0051] The spherical harmonic coefficients are subjected to feature extraction processing to determine spherical harmonic domain features.

[0052] Optionally, when the monitoring module is used to determine the abnormality monitoring result of the target wind turbine blade according to the target beam and the abnormality classifier generated by convolutional neural network training, the monitoring module is used to:

[0053] Performing spatial domain enhancement processing on the target beam, and obtaining spherical harmonic domain features and time-frequency domain features of the target beam after spatial domain enhancement;

[0054] Inputting the spherical harmonic domain features and time-frequency domain features of the target beam into the anomaly classifier;

[0055] The abnormality classifier performs classification prediction based on the spherical harmonic domain characteristics and the time-frequency domain characteristics to determine the abnormality monitoring result of the target wind turbine blade.

[0056] Optionally, the first determining module is further configured to determine a beamformer through the following steps:

[0057] Constructing a sound pressure signal model corresponding to the spherical microphone array according to the sound pressure signal vector, the steering matrix, and the microphone noise vector corresponding to the multi-channel acoustic signal;

[0058] performing spherical harmonic transformation on the sound pressure signal model, determining a steering matrix represented by the spherical harmonics, and updating the sound pressure signal model;

[0059] Determining a spherical harmonic domain model according to the updated sound pressure signal model;

[0060] A beamformer based on the spherical harmonics domain is determined according to the spherical harmonics domain model and the acoustic signal time-frequency space of the multi-channel acoustic signal.

[0061] An embodiment of the present application also provides an electronic device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps of the determination method as described above are performed.

[0062] An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned method xx are executed.

[0063] An embodiment of the present application provides a method and device for determining defects in wind turbine blades. The method includes: real-time monitoring of target wind turbine blades and obtaining a monitored panoramic image and multi-channel acoustic signals; processing the multi-channel acoustic signals to determine a beam set consisting of multiple beams in multiple spatial directions; spatially aligning and fusing the acoustic signals of the beam set with the panoramic image to determine a panoramic sound image; utilizing the correlation between the blade position in the panoramic image and the extreme points of the sound image, locking the beam at the head of the target wind turbine blade from the beam set to determine the target beam; and determining the abnormal monitoring result of the target wind turbine blade based on the target beam and an abnormality classifier generated by convolutional neural network training.

[0064] In this way, the present application overcomes the structural interference and maintenance cost problems of traditional sensor monitoring by combining panoramic images and multi-channel acoustic signals, and can collect information in different spatial directions through beam sets. Even in insufficient light or target movement, it can still accurately locate the blade position and its acoustic image extreme points, thereby ensuring the detection accuracy of fine cracks. Therefore, this fusion method can effectively improve the reliability and accuracy of detection.

[0065] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0067] Figure 1 A flow chart of a method for determining defects in wind turbine blades provided in an embodiment of the present application;

[0068] Figure 2 A schematic diagram of a wind turbine blade defect detection process provided in this application;

[0069] Figure 3 A schematic diagram of the structure of a determination device provided in an embodiment of the present application;

[0070] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0071] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application for which protection is claimed, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, each other embodiment obtained by those skilled in the art without making creative work falls within the scope of protection of the present application.

[0072] In recent years, with the worsening of environmental pollution and the energy crisis, wind power has garnered increasing attention as a clean energy source. By 2023, my country had the largest number of wind turbines in the world, accounting for over 30% of total wind power generation, making it a vital energy source. Wind turbines typically operate in harsh environments and face complex operating conditions, creating potential for failure or a high probability of occurrence. Failure to detect failures promptly can lead to high repair costs and even greater economic losses. Common wind turbine failures include gear failures, generator failures, blade failures, and tower failures. Blade failures account for approximately 40% of all wind turbine failures. Blades are often damaged by wind, rain, hail, and lightning, resulting in damage such as leading edge corrosion, cracking, and surface fissures.

[0073] Traditional blade inspection methods typically use pre-installed sensors for monitoring, but this approach not only compromises the structural strength of wind turbine blades but also increases subsequent sensor maintenance costs. Another vision-based inspection method, typically mounted on a gimbal or drone platform, tracks and images the blades before identifying defect patterns. However, in low light conditions and due to target motion, accurate detection of fine cracks is difficult.

[0074] Another method, based on blade soundprint monitoring, assesses blade condition by detecting the noise signatures generated by blade rotation caused by surface defects. This method has attracted widespread attention in recent years due to its non-contact, efficient, easy-to-install, and low-cost advantages. However, strong wind noise interference from wind farms can severely affect the accurate identification of soundprint signatures, making fault detection still challenging.

[0075] In order to solve the above technical problems, the existing patent "Fan Fault Detection Method and System Based on AI Auscultation, Fan Safety System" (CN115163426 A) obtains the fan blade noise signal through a single-channel microphone, and then performs similarity measurement with the prior knowledge base based on characteristics such as time domain, frequency domain, sound pattern and frequency band energy to determine whether there is a fault. However, since this method uses an omnidirectional acoustic signal acquisition method, it inevitably introduces all-directional background noise, affecting the extraction quality of the acoustic signal. In addition, the similarity-based algorithm may affect the performance of subsequent fault diagnosis due to its poor generalization ability.

[0076] The existing patent, "Remote Auscultation Method for Wind Turbine Blades Based on Acoustic Diagnosis" (CN112067701 A), acquires the sound of wind sweeping from blades through a single channel, primarily using an adaptive method to extract octave features, and detects blade status using the SVDD algorithm. Although this method has improved signal extraction, it still struggles to remove strong ambient background noise during the acoustic signal acquisition process. Furthermore, the SVDD-based detection model has high computational complexity when processing high-dimensional data, has limitations in sphere boundaries, and lacks multi-classification capabilities, making it difficult to meet the generalization and scalability requirements for blade fault detection.

[0077] The existing patent, "A Non-Contact Monitoring System and Method for Offshore Wind Turbine Blade Faults" (CN109763944A), uses offshore meteorological and hydrological nodes to acquire aeroacoustic signals. It then uses a neural network to analyze the wind sweep signals of the wind turbine blades to determine if an anomaly exists. However, this method is insufficient in its ability to acquire acoustic signals, and the subsequent acoustic signal enhancement algorithm remains unimproved, severely impacting fault detection effectiveness.

[0078] Another patent, "A Deep Learning-Based Acoustic Detection Method and System for Wind Blade Faults" (CN116631442A), acquires acoustic signals through a microphone array, uses beamforming to spatially filter and enhance the target signal, and then uses deep learning methods to classify Mel-spectrogram features to achieve anomaly detection. Although this method enhances blade noise extraction capabilities through linear array signal processing, the inconsistent spatial directionality of linear arrays increases the difficulty of array deployment, affects the performance of spatial filtering enhancement, and ultimately leads to a decline in blade fault detection performance.

[0079] Based on this, the embodiments of the present application provide a method and device for determining defects in wind turbine blades, which can overcome the interference problems and maintenance cost problems of existing monitoring, and can further improve the reliability and accuracy of detection results compared to existing technologies.

[0080] See also Figure 1 , Figure 1This is a flow chart of a method for determining defects in a wind turbine blade provided in an embodiment of the present application. Figure 1 As shown in , the determination method provided in the embodiment of the present application includes:

[0081] S101: Monitor the blades of a target wind turbine in real time, and obtain a panoramic image and multi-channel acoustic signals of the monitoring.

[0082] S102: Process the multi-channel acoustic signal to determine a beam set consisting of multiple beams in multiple spatial directions.

[0083] S103: spatially register and fuse the acoustic signal of the beam set with the panoramic image to determine a panoramic sound image.

[0084] S104 , using the correlation between the blade position in the panoramic image and the extreme value point of the sound image, lock the beam at the head of the blade of the target wind turbine from the beam set to determine the target beam.

[0085] S105 . Determine an abnormality monitoring result of the target wind turbine blade according to the target beam and an abnormality classifier generated by convolutional neural network training.

[0086] Regarding step S101, in this step, the blades of the target wind turbine are monitored in real time, and panoramic images and multi-channel acoustic signals are collected. The panoramic images can provide visual information of the blades, and the multi-channel acoustic signals can capture the acoustic wave information of the blades in different spatial directions.

[0087] The multi-channel acoustic signal refers to the sound signal collected simultaneously from multiple directions by a microphone array.

[0088] In one embodiment provided in the present application, a panoramic image is obtained by the following steps: obtaining a monitoring image taken by each camera in a circular array camera when monitoring the target wind turbine blades; performing image stitching processing on all monitoring images to determine the panoramic image.

[0089] In this way, the circular array camera is composed of multiple cameras, which monitor the blades of the target wind turbine at the same time, and each camera independently shoots the blades to obtain monitoring images, which can capture information from different angles and fields of view.

[0090] All acquired monitoring images are stitched together, that is, each independent image is combined into an overall panoramic image, wherein the stitching process generally includes image alignment, identification of overlapping areas, and seamless connection processing.

[0091] In another embodiment provided in the present application, a multi-channel acoustic signal is obtained by the following steps: obtaining an initial acoustic signal collected by each spherical unidirectional microphone in a spherical microphone array in the area where the target wind turbine blade is located; and determining the multi-channel acoustic signal based on the initial acoustic signals collected by all spherical unidirectional microphones.

[0092] Here, the spherical microphone array includes a plurality of spherical unidirectional microphones. In the spherical microphone array, each spherical unidirectional microphone independently collects the initial acoustic signal of the area where the target wind turbine blade is located.

[0093] The initial acoustic signals collected by all spherical unidirectional microphones are integrated to form a comprehensive multi-channel acoustic signal. This signal contains sound information collected from different spatial directions and can provide more comprehensive acoustic wave data.

[0094] The spherical microphone array and the circular array camera can be installed in combination or independently.

[0095] Regarding step S102, in this step, the acquired multi-channel acoustic signal is processed and decomposed into multiple acoustic beams collected from different spatial directions to obtain a beam set. These beams can reflect the acoustic wave information generated by the blade in different directions.

[0096] In one embodiment provided in the present application, processing the multi-channel acoustic signal to determine a beam set consisting of multiple beams in multiple spatial directions includes:

[0097] S1021 . Perform frequency domain information and spatial feature extraction processing on the multi-channel acoustic signal to determine spherical harmonic domain features.

[0098] S1022: Divide the spatial grid generated by uniform spherical sampling using a beamformer according to the spherical harmonic domain characteristics, determine multiple beams in different spatial directions, and determine a beam set based on the multiple beams.

[0099] Regarding step S1021, in this step, the multi-channel sound signal is converted from the time domain to the frequency domain, the performance of the sound at different frequencies is analyzed, and then the spatial distribution characteristics of the sound source are obtained to obtain the spherical harmonic domain characteristics reflecting the sound field characteristics, that is, the distribution of sound waves on the spherical surface.

[0100] In one embodiment provided in the present application, the extracting frequency domain information and spatial features of the multi-channel acoustic signal to determine the spherical harmonic domain features includes:

[0101] S10211. Perform short-time Fourier transform processing on the multi-channel acoustic signal to determine frequency domain information corresponding to the channel acoustic signal.

[0102] S10212: Perform spherical harmonic domain transformation on the frequency domain information to determine spherical harmonic coefficients mapped to the three-dimensional space signal.

[0103] S10213. Perform feature extraction processing on the spherical harmonic coefficients to determine spherical harmonic domain features.

[0104] Regarding step S10211, in this step, short-time Fourier transform (STFT) processing is performed on the multi-channel acoustic signal to convert the acoustic signal in the time domain into the frequency domain.

[0105] The process of performing short-time Fourier transform (STFT) on multi-channel acoustic signals is as follows: the time domain signal of each microphone is divided into multiple windows (each window has a fixed length and overlaps), and the signal in each window is Fourier transformed to obtain the signal's spectrum information.

[0106] Regarding step S10212, in this step, a spherical harmonic domain transformation is performed on the extracted frequency domain information to decompose the acoustic signal in the three-dimensional space into a set of spherical harmonic coefficients.

[0107] The extracted frequency domain information is transformed into the spherical harmonic domain, including expanding the signal at each frequency point through the spherical harmonic basis function to obtain the spherical harmonic coefficients.

[0108] Regarding step S10213, the spherical harmonic domain features can reflect important modes and feature points of the acoustic signal in the spherical harmonic domain.

[0109] In step S1022, a beamformer is used to uniformly sample the sphere based on the extracted spherical harmonics domain features to generate a spatial grid. Each grid represents a spatial direction, and by focusing the signal in each direction, multiple beams are generated. These beams reflect the distribution of the acoustic signal in different spatial directions. Finally, all beams are formed into a beam set, a collection of acoustic signals in multiple spatial directions, for subsequent monitoring and analysis.

[0110] Through steps S1021 and S1022, the conversion from multi-channel acoustic signals to multi-directional beams can be achieved, so that acoustic information in different spatial directions can be accurately captured, which helps to improve the spatial resolution and accuracy of wind turbine blade defect detection.

[0111] Regarding the beamformer described in step S1022, in one embodiment provided in this application, the beamformer is determined by the following steps:

[0112] S201: Construct a sound pressure signal model corresponding to a spherical microphone array according to a sound pressure signal vector, a steering matrix, and a microphone noise vector corresponding to the multi-channel sound signal.

[0113] S202: Perform spherical harmonic transformation on the sound pressure signal model, determine a steering matrix represented by spherical harmonics, and update the sound pressure signal model.

[0114] S203: Determine a spherical harmonic domain model according to the updated sound pressure signal model.

[0115] S204: Determine a beamformer based on the spherical harmonics domain according to the spherical harmonics domain model and the acoustic signal time-frequency space of the multi-channel acoustic signal.

[0116] With respect to steps S201 to S204 , the generation process of the beamformer is specifically and exemplarily described through the following steps.

[0117] A spherical microphone array acquires acoustic signals. Q unidirectional microphones are evenly distributed on a rigid sphere with a radius of r. There are L far-field sound sources in space, and a rigid spherical sound propagation model is obtained. The spatial coordinates of the microphones are:

[0118] r q =[rcos(θ q )sin(φ q ),rsin(θ q )sin(φ q ),rcos(φ q )] T

[0119] where θ q is measured counterclockwise from the x-axis, φ q is the elevation angle measured downward from the z-axis.

[0120] Similarly, the coordinates of the L sound sources are:

[0121] r l =[rcos(θ l )sin(φ l ),rsin(θ l )sin(φ l ),rcos(φ l )] T , l=1,…,L.

[0122] The received signal model of the spherical microphone array can be expressed as:

[0123] p(k)=A(k)S(k)+z(k)

[0124] Where k is the wave number, i.e., k = 2πf / c, f is the frequency, and c is the speed of sound. p(k) is the sound pressure signal vector, A(k) is the steering matrix, S(k) is the sound source sound pressure signal vector, and z(k) is the sensor noise vector. p(k), A(k), and S(k) are expressed as:

[0125] p(k)=[p(k,r1),p(k,r2),…,p(k,r Q )] T ,

[0126] A(k)=[a(k,r1),a(k,r2),…,a(k,r L )] T ,

[0127] S(k)=[s1(k),s2(k),…,s L (k)] T

[0128] where p(k,r q ) is the acoustic signal received by the qth microphone, a(k,r l ) and s l (k) are the guiding vector and sound pressure intensity of the lth divine light respectively. l )for:

[0129]

[0130] where k l for

[0131] k l =-k[cos(θ l )sin(φ l ),sin(θ l )sin(φ l ),cos(φ l )] T ,i 2 =-1

[0132] According to the spherical harmonic transform, It can be approximately decomposed into truncated N-order spherical harmonics:

[0133]

[0134] where b n (kr) is the mode strength for a unidirectional microphone:

[0135] b n (kr)=4πi n [j n (kr)-ij′ n (kr)],

[0136] in is the spherical harmonic basis function:

[0137]

[0138] is the Legendre function, and j n is a spherical Bessel function of the first kind. Therefore, the steering vector can be expressed as:

[0139] A(k)=Y(Ω)B(kr)Y H (Γ),

[0140] Where Y(Ω) is Q×(N+1) 2 The matrix:

[0141] Y(Ω)=[y T (θ1,φ1),…,y T (θ Q ,φ Q )] T ,

[0142]

[0143] From the spherical harmonics of L sound source directions, we can get L×(N+1) 2 The matrix Y H (Γ):

[0144] Y H (Γ)=[y T (θ1,φ1),…,y T (θ L ,φ L )]

[0145] and (N+1) 2 ×(N+1) 2 The diagonal matrix B(kr):

[0146] B(kr)=diag{b0(kr),b1(kr),b1(kr),…,b N (kr)}..

[0147] From this we can get

[0148] p(k)=Y(Ω)B(kr)Y H (Γ)S(k)+z(k)

[0149] Due to the orthogonality principle of spherical harmonics, for approximately uniform spherical space sampling, Y H (Ω)Y(Ω)=I, we can get p nm (k) is:

[0150] p nm (k)=B(kr)Y H (Γ)S(k)+z nm (k)

[0151] where p nm (k) = Y H (Ω)p(k). For a given spherical array configuration, we can obtain the spherical harmonic characteristic coefficient q nm (k):

[0152]

[0153] where q nm (k) = B -1 (kr)p nm (k), it is worth noting here that the new steering vector Y H (Γ) does not contain frequency components, and thus can directly process coherent sources without the need for frequency domain focusing transforms. For broadband sound sources, acoustic imaging under reverberant conditions can be achieved by smoothing in the selected time-frequency domain using the short-time Fourier transform (STFT). Therefore, the spherical harmonic domain model can be expressed as:

[0154]

[0155] Where E is the time-frequency space of the acoustic signal. Then, we can get the beamformer based on the spherical harmonics domain:

[0156]

[0157] where N τ and N v are the number of time and frequency frames, respectively.

[0158] Regarding step S103 , here, the spatial distribution and energy information of the sound source can be visualized on the panoramic image based on beamforming technology and image processing technology, that is, the beam set is fused with the panoramic image to determine the panoramic sound image.

[0159] Among them, the fusion of beam sets and panoramic images mainly includes direction matching and energy mapping processing.

[0160] For step S104, for example, based on the blade position in the panoramic image and the extreme point in the sound image (the extreme position of the sound signal), the most relevant sound signal beam is found, and the beam of the blade head is locked from the beam set, that is, the target beam is determined, and the sound characteristics of the blade head are accurately located.

[0161] The target beam is determined dynamically in real time according to the position of the blade head.

[0162] Regarding step S105, in this step, the target beam determined in step S104 is used to analyze the acoustic signal of the beam through a pre-trained convolutional neural network (CNN) anomaly classifier to determine whether there is an abnormality in the blade.

[0163] In one embodiment provided in the present application, determining the abnormality monitoring result of the target wind turbine blade according to the target beam and the abnormality classifier generated by convolutional neural network training includes:

[0164] S1051. Perform spatial domain enhancement processing on the target beam, and obtain spherical harmonic domain characteristics and time-frequency domain characteristics of the target beam after spatial domain enhancement.

[0165] S1052: Input the spherical harmonic domain features and time-frequency domain features of the target beam into the anomaly classifier.

[0166] S1053: The abnormality classifier performs classification prediction based on the spherical harmonic domain characteristics and the time-frequency domain characteristics to determine the abnormality monitoring result of the target wind turbine blade.

[0167] With respect to step S1051, the target beam is subjected to spatial enhancement processing, that is, the signal is processed in the spatial dimension to enhance the features of the signal related to the spatial distribution.

[0168] For step S1052, the abnormality classifier is generated by training a convolutional neural network. The size of the convolution kernel of the convolutional neural network can be (N+1) 2 / 2×1.

[0169] For step S1052, when the abnormality classifier performs classification prediction based on the spherical harmonic domain features and the time-frequency domain features to determine the abnormality monitoring result of the target wind turbine blade, it may specifically include: after N_layer convolution and Rectified Linear Unit (ReLU) activation function processing, the number of convolution channels of each layer is C1, ..., C L Finally, the voiceprint features of N_Feats dimension are obtained. Finally, the abnormal / normal status of the blade is classified through the Multi-Layer Perception Network (MLP) classifier.

[0170] That is, the abnormal monitoring results include two classification results: normal and abnormal.

[0171] For examples, see Figure 2 , Figure 2 This is a schematic diagram of the wind turbine blade defect detection process provided by this application. Figure 2As shown, Figure 2 The process from image and sound collection, data processing and monitoring result generation is recorded in FIG. 5 . The specific implementation is as described in steps S101 to S105 and will not be repeated here.

[0172] In this way, a spherical unidirectional microphone is used to obtain acoustic signals, and its special spatially symmetrical structure can achieve consistent directionality in all directions. At the same time, panoramic sound and optical imaging technologies are combined to accurately obtain spatial information, and the blade sweep signal is accurately extracted through the beam spatial domain filtering method. Finally, blade defects are identified with the help of a deep neural network. This not only enhances the anti-interference ability, but also its panoramic detection method does not require posture adjustment, providing a more convenient inspection method. In addition, this method can capture the blade sweep signal in all directions, accurately obtain the target sound source signal without adjusting the array posture, and use image information to assist in noise source positioning, thereby quickly identifying the sound source. And because the spherical array has a special symmetrical structure, its spatial directivity is consistent in all directions. This solution can not only accurately lock the sound source through the beam, but also improve the anti-interference performance of the system.

[0173] Based on the same inventive concept, the embodiments of the present application also provide a determination device corresponding to the determination method. Since the principle of solving the problem by the device in the embodiments of the present application is similar to the above-mentioned determination method in the embodiments of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0174] See also Figure 3 , Figure 3 This is a schematic diagram of the structure of a determination device provided in an embodiment of the present application. Figure 3 As shown in , the determining device 300 includes:

[0175] An acquisition module 310 is used to monitor the target wind turbine blades in real time and obtain a panoramic image and multi-channel acoustic signals of the monitoring;

[0176] A first determining module 320 is configured to process the multi-channel acoustic signal and determine a beam set consisting of multiple beams in multiple spatial directions;

[0177] a fusion module 330 for spatially registering and fusing the acoustic signals of the beam set with the panoramic image to determine a panoramic sound image;

[0178] The second determination module 340 is configured to utilize the correlation between the blade position in the panoramic image and the extreme value point of the acoustic image to lock the beam at the head of the target wind turbine blade from the beam set, thereby determining the target beam;

[0179] The monitoring module 350 is configured to determine an abnormality monitoring result of the target wind turbine blade based on the target beam and an abnormality classifier generated by convolutional neural network training.

[0180] Optionally, the acquisition module 310 is further configured to acquire a panoramic image through the following steps:

[0181] Acquire a monitoring image captured by each camera in the circular array of cameras when monitoring the blades of the target wind turbine;

[0182] Perform image stitching processing on all monitoring images to determine the panoramic image.

[0183] Optionally, the acquisition module 310 is further configured to acquire a multi-channel acoustic signal through the following steps:

[0184] Acquiring an initial acoustic signal collected by each spherical unidirectional microphone in the spherical microphone array in the area where the target wind turbine blade is located;

[0185] The multi-channel acoustic signal is determined according to the initial acoustic signals collected by all spherical unidirectional microphones.

[0186] Optionally, when the first determining module 320 is used to process the multi-channel acoustic signal and determine a beam set consisting of multiple beams in multiple spatial directions, the first determining module 320 is used to:

[0187] Performing frequency domain information and spatial feature extraction processing on the multi-channel acoustic signal to determine spherical harmonic domain features;

[0188] According to the spherical harmonic domain characteristics, a beamformer is used to divide the space grid generated by uniform spherical sampling, multiple beams in different spatial directions are determined, and a beam set is determined based on the multiple beams.

[0189] Optionally, when the first determining module 310 is used to perform frequency domain information and spatial feature extraction processing on the multi-channel acoustic signal to determine the spherical harmonic domain features, the first determining module 310 is used to:

[0190] Performing short-time Fourier transform processing on the multi-channel acoustic signal to determine frequency domain information corresponding to the channel acoustic signal;

[0191] Performing spherical harmonic domain transformation on the frequency domain information to determine spherical harmonic coefficients mapped with three-dimensional spatial signals;

[0192] The spherical harmonic coefficients are subjected to feature extraction processing to determine spherical harmonic domain features.

[0193] Optionally, when the monitoring module 350 is used to determine the abnormality monitoring result of the target wind turbine blade according to the target beam and the abnormality classifier generated by convolutional neural network training, the monitoring module 350 is used to:

[0194] Performing spatial domain enhancement processing on the target beam, and obtaining spherical harmonic domain features and time-frequency domain features of the target beam after spatial domain enhancement;

[0195] Inputting the spherical harmonic domain features and time-frequency domain features of the target beam into the anomaly classifier;

[0196] The abnormality classifier performs classification prediction based on the spherical harmonic domain characteristics and the time-frequency domain characteristics to determine the abnormality monitoring result of the target wind turbine blade.

[0197] Optionally, the first determining module 320 is further configured to determine a beamformer through the following steps:

[0198] Constructing a sound pressure signal model corresponding to the spherical microphone array according to the sound pressure signal vector, the steering matrix, and the microphone noise vector corresponding to the multi-channel acoustic signal;

[0199] performing spherical harmonic transformation on the sound pressure signal model, determining a steering matrix represented by the spherical harmonics, and updating the sound pressure signal model;

[0200] Determining a spherical harmonic domain model according to the updated sound pressure signal model;

[0201] A beamformer based on the spherical harmonics domain is determined according to the spherical harmonics domain model and the acoustic signal time-frequency space of the multi-channel acoustic signal.

[0202] See also Figure 4 , Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 4 As shown in FIG, the electronic device 400 includes a processor 410 , a memory 420 and a bus 430 .

[0203] The memory 420 stores machine-readable instructions executable by the processor 410. When the electronic device 400 is running, the processor 410 communicates with the memory 420 via the bus 430. When the machine-readable instructions are executed by the processor 410, the above-mentioned Figure 1 as well as Figure 2 The specific implementation of the steps in the method embodiment shown can be found in the method embodiment and will not be repeated here.

[0204] The embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the computer program can execute the above-mentioned Figure 1 as well as Figure 2 The specific implementation of the steps in the method embodiment shown can be found in the method embodiment and will not be repeated here.

[0205] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0206] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. There may be other division methods in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed may be through some communication interface, indirect coupling or communication connection of devices or units, which may be electrical, mechanical or other forms.

[0207] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0208] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0209] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0210] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The scope of protection of the present application is not limited thereto. Although the present application has been described in detail with reference to the above-mentioned embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above-mentioned embodiments within the technical scope disclosed in the present application, or perform equivalent replacements for some of the technical features thereof. These modifications, changes, or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A method for determining defects of wind turbine blades, characterized in that: The determination method comprises: Monitor the target wind turbine blades in real time and obtain panoramic images and multi-channel acoustic signals; Processing the multi-channel acoustic signal to determine a beam set consisting of multiple beams in multiple spatial directions; spatially registering and fusing the acoustic signal of the beam set with the panoramic image to determine a panoramic sound image; By utilizing the correlation between the blade position in the panoramic image and the extreme value point of the sound image, the beam of the blade head of the target wind turbine is locked from the beam set to determine the target beam; The abnormal monitoring result of the target wind turbine blade is determined according to the target beam and the abnormal classifier generated by convolutional neural network training.

2. The determination method according to claim 1, characterized in that: Acquire a panoramic image by following these steps: Acquire a monitoring image taken by each camera in the circular array camera when monitoring the blades of the target wind turbine; All monitoring images are stitched together to determine the panoramic image.

3. The determination method according to claim 1, characterized in that: Acquire multi-channel acoustic signals by following the steps below: Acquire an initial acoustic signal of the area where the blades of the target wind turbine are located, collected by each spherical unidirectional microphone in the spherical microphone array; The multi-channel acoustic signal is determined according to the initial acoustic signals collected by all spherical unidirectional microphones.

4. The determination method according to claim 1, characterized in that: The processing of the multi-channel acoustic signal to determine a beam set consisting of multiple beams in multiple spatial directions includes: Performing frequency domain information and spatial feature extraction processing on the multi-channel acoustic signal to determine spherical harmonic domain features; According to the spherical harmonic domain characteristics, a beamformer is used to divide the space grid generated by uniform spherical sampling, multiple beams in different spatial directions are determined, and a beam set is determined according to the multiple beams.

5. The determination method according to claim 4, characterized in that: The extracting and processing the frequency domain information and the spatial features of the multi-channel acoustic signal to determine the spherical harmonic domain features includes: Performing short-time Fourier transform processing on the multi-channel acoustic signal to determine frequency domain information corresponding to the channel acoustic signal; Performing spherical harmonic domain transformation processing on the frequency domain information to determine spherical harmonic coefficients mapped with three-dimensional spatial signals; The spherical harmonic coefficients are subjected to feature extraction processing to determine spherical harmonic domain features.

6. The determination method according to claim 1, characterized in that: Determining the abnormal monitoring result of the target wind turbine blade according to the target beam and the abnormal classifier generated by convolutional neural network training includes: Performing spatial domain enhancement processing on the target beam, and obtaining spherical harmonic domain characteristics and time-frequency domain characteristics of the target beam after spatial domain enhancement; Inputting the spherical harmonic domain features and time-frequency domain features of the target beam into the anomaly classifier; The abnormality classifier performs classification prediction according to the spherical harmonic domain characteristics and the time-frequency domain characteristics to determine the abnormality monitoring result of the blade of the target wind turbine generator set.

7. The determination method according to claim 4, characterized in that: The beamformer is determined by the following steps: Constructing a sound pressure signal model corresponding to the spherical microphone array according to the sound pressure signal vector, the steering matrix and the microphone noise vector corresponding to the multi-channel acoustic signal; Performing spherical harmonic transformation processing on the sound pressure signal model, determining a steering matrix represented by spherical harmonics, and updating the sound pressure signal model; Determining a spherical harmonic domain model according to the updated sound pressure signal model; A beamformer based on the spherical harmonics domain is determined according to the spherical harmonics domain model and the acoustic signal time-frequency space of the multi-channel acoustic signal.

8. A device for determining defects of wind turbine blades, characterized in that: The determining device comprises: An acquisition module is used to monitor the blades of the target wind turbine in real time and obtain the monitored panoramic images and multi-channel acoustic signals; A first determination module is used to process the multi-channel acoustic signal to determine a beam set consisting of multiple beams in multiple spatial directions; A fusion module, used for spatially registering and fusing the acoustic signal of the beam set with the panoramic image to determine the panoramic sound image; The second determination module is used to utilize the correlation between the blade position in the panoramic image and the extreme value point of the sound image to lock the beam of the blade head of the target wind turbine from the beam set to determine the target beam; The monitoring module is used to determine the abnormal monitoring result of the target wind turbine blade according to the target beam and the abnormal classifier generated by convolutional neural network training.

9. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and the machine-readable instructions are executed by the processor to perform the steps of the determination method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the determination method according to any one of claims 1 to 7 are executed.

Images