A wind turbine blade surface damage visual detection method under a complex scene

Abstract

The application discloses a kind of wind turbine blade surface damage visual inspection methods under complex scene, mainly include the following steps: constructing blade damage dataset;From the problems such as damage feature submersion of low-light image, background noise interference and damage feature easy to ignore loss, enhance low-light image by fusing Retinex and atmospheric scattering model;Design a multi-dimensional information complementary semantic block segmentation method, extract target blade area from background interference image;Construct a kind of deep network model integrated residual network and damage feature coding, identify blade surface damage mode of different scales.The method can overcome the influence of light change, background interference, small damage size and mode diversity factors, enhance the difference between damage area and non-damage area The ability of characterization, accurately and efficiently identify blade surface damage.

Description

Technical Field

[0001] This invention relates to the field of wind turbine blade damage detection technology, and in particular to a visual detection method for surface damage of wind turbine blades in complex scenarios. Background Technology

[0002] With the rapid development of my country's wind power industry, the operational safety of wind turbine generators has become increasingly prominent. As the core and critical component for wind turbine generators to harvest wind energy, blades operate in harsh natural environments for extended periods, enduring complex stresses and potential erosion from rain and snow, as well as damage from lightning strikes. This makes them highly susceptible to surface damage such as cracks, corrosion and wear, coating peeling, and lightning strike ablation, which can lead to significant economic and property losses or even personal injury.

[0003] Currently, the mainstream method for detecting damage to wind turbine blades is manual inspection, which requires observing the damage through telescopes or suspended platforms after the wind turbine is shut down. This method suffers from low efficiency, is heavily influenced by weather and human subjective factors, and is prone to safety accidents. Existing online detection technologies such as vibration and acoustic emission often face shortcomings such as severe interference from vibration and noise from equipment inside the nacelle, easy attenuation and aliasing during fault signal transmission, low sensitivity to minor damage, and high sensor maintenance costs.

[0004] With the development of computer vision, image visual diagnosis has provided a new solution for blade surface damage detection, offering advantages such as high efficiency, immunity to subjective factors, and precise location of damage types and locations. However, wind turbines operate outdoors for extended periods, and under natural low-light conditions such as cloudy days, backlighting, and dusk, the contrast of damaged areas is low, severely impacting subsequent damage feature extraction and model training. Furthermore, image acquisition inevitably includes background information such as terrain features, coupled with various textures and colors that are similar to damage characteristics, creating interference noise. Moreover, the causes of blade surface damage are complex, with damage exhibiting regional distribution and significant differences in area size, shape, color, and distribution. Therefore, this invention focuses on image enhancement under low-light conditions, blade area localization and segmentation, and accurate identification and capture of damage area information, disclosing a visual detection method for wind turbine blade surface damage in complex scenarios. Summary of the Invention

[0005] This invention addresses the problems and challenges in identifying surface damage on wind turbine blades mentioned above, and provides a visual detection method for surface damage on wind turbine blades in complex scenarios. This invention can be achieved through the following technical solutions:

[0006] Step 1: Collect images of different types of blade damage in the field wind field, manually clean, filter and label the damage categories, expand the damage samples by horizontal, vertical and mirror flipping, and establish a blade surface damage image dataset.

[0007] Step 2: Combine the Retinex and ASM models to enhance the illumination of the blade surface damage dataset, as follows:

[0008] (1) The original low-light leaf image is inverted and used as input to ASM to derive and establish a Retinex model:

[0009]

[0010] Among them, L r (x,y)=1-(1-L(x,y)) / A, where L(x,y) represents the original image, R(x,y) represents the real scene without illumination, T(x,y) represents transmittance, which refers to the proportion of light energy that is propagated to the observer's position when passing through particles in the atmosphere, and A represents atmospheric light in the scene; by estimating the transmittance and atmospheric light in the current scene, the expected R(x,y) is calculated inversely.

[0011] (2) Estimate the atmospheric ray A in the inverted image based on the dark channel prior theory, where the dark channel image can be represented as:

[0012]

[0013] Among them, I c (·) represents a certain color channel of image I, Ω x This represents a region window centered at x; considering the dense fog approximation characteristics of low-light inverted images, the brightest 0.1% of pixels in the dark channel image are selected, and the brightest pixel in the inverted image is determined based on its index position, thus estimating atmospheric light A;

[0014] (3) Using the guided image filtering as the central wrap function, through L r The (x,y) convolution operation estimates the transmittance T(x,y);

[0015] (4) Based on the atmospheric light and transmittance obtained from (2) and (3), calculate the real scene image without the influence of illumination by model inversion, and quantize the RGB channels using the histogram equalization method with limited contrast.

[0016] Step 3: Employ a semantic block segmentation and clustering method with complementary multidimensional information to locate and segment the leaf regions in the augmented dataset, avoiding interference from terrain background information on damage identification, as detailed below:

[0017] (1) Perform initial semantic block division and semantic label allocation on the enhanced leaf image, solve the gradient magnitude of each pixel in the image, and use the point with the lowest gradient in the semantic block as the initial seed.

[0018] (2) Calculate the Euclidean distance between a pixel and its various neighboring sub-points in the feature space as the semantic block membership degree:

[0019]

[0020] Where, x i and d corresponds to the positions of the current pixel and the cluster center in the feature space, respectively. lab d represents the distance between two pixels in the CIELAB color space. xy d represents the relative distance between two pixels. s Represents the gradient distance between pixels; assigns pixels to the semantic block with the highest membership degree:

[0021]

[0022] in, The label to which the current pixel belongs is represented, K represents the number of semantic blocks, and i represents the domain. For any pixel within, c k Let represent the cluster center of the k-th semantic block; then, update the semantic labels of the pixels, calculate the mean vector of each semantic block, and iteratively update the cluster centers until the convergence condition is met:

[0023]

[0024] in, This represents the center value of the semantic block in the t-th clustering result;

[0025] (3) Using the cluster centers of semantic blocks as samples, perform fuzzy clustering on the segmented semantic blocks. The objective function is:

[0026]

[0027] Where C is the number of clusters, m is the weighting coefficient, and γ is the weighting factor. i,k T represents the membership degree between the k-th semantic block and the i-th class. i Let represent the i-th cluster center; the termination condition for iteration is:

[0028]

[0029] in, This represents semantic block c after the t-th update. k For cluster center T i Membership degree; update semantic labels based on clustering results to complete leaf region segmentation and localization;

[0030] Step 4: Construct a deep network recognition model based on ResNet and damage feature encoding, and train and test the damage recognition model using the augmented dataset, as follows:

[0031] (1) The ResNet network is used to extract the local convolutional features of the segmented leaf sample images in stages. The number of feature vector channels is quantized through the convolutional layer, and then linearized and flattened and concatenated with the damage category vector to construct the feature space vector of the damage area on the leaf surface.

[0032] (2) Normalize the regional feature space vector, learn the damage attention weights of different regions in parallel through the multi-head attention mechanism (MHAM), and at the same time use the residual network structure to connect the input and MHAM output across layers to quickly capture global damage features.

[0033] (3) Normalize the damage feature encoding vector, use a fully connected layer to perform upsampling and downsampling at the same multiple, randomly set the output value of a portion of neurons to 0 using the DropOut regularization method, and then use a residual network to superimpose the input and output vectors to learn and identify the differences between damaged and non-damaged regions.

[0034] (4) Iterate (2) and (3) L times, and the output results are passed through a fully connected layer and a SoftMax function. The model determines the damage type of the input image based on the results. The damage dataset with completed leaf segmentation is used for model training and accuracy verification.

[0035] The advantages of this invention are:

[0036] 1. This invention constructs a low-light image enhancement method that integrates Retinex and ASM models, which can effectively enhance the texture details of damaged areas in natural scenes and is more suitable for leaf surface damage detection under low light conditions;

[0037] 2. This invention designs a semantic block segmentation method with complementary multi-dimensional information, which can accurately locate and segment the target leaf region and effectively avoid interference from various coupled terrain background noises;

[0038] 3. This invention proposes a deep network recognition model that integrates ResNet and damage feature encoding. This method can enhance the ability to characterize the differences between damaged and undamaged areas, improve the network's sensitivity to damage features, and accurately and efficiently identify blade surface damage. Attached Figure Description

[0039] Figure 1 This is a flowchart illustrating the method of the present invention.

[0040] Figure 2This is a schematic diagram of the process for enhancing low-light image damage.

[0041] Figure 3 A schematic diagram of the image leaf region segmentation and localization method.

[0042] Figure 4 The network structure diagram for extracting feature vectors from ResNet50 images.

[0043] In the diagram, an example of an n*n Conv sn BN ReLU representation is shown. n*n Conv represents a convolution operation with an n*n kernel size, s represents the stride, n represents the number of kernels, BN represents a Batch Norm layer, and ReLU is the activation function.

[0044] Figure 5 This is a network structure diagram of the damage feature encoder.

[0045] Among them, ① is the MHAM module, and ② is the multilayer perceptron module.

[0046] Figure 6 This is a flowchart of the invention process. Implementation

[0047] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in detail below with reference to the accompanying drawings. The following embodiments are for illustrative purposes only and should not be used to limit the scope of the present invention.

[0048] A visual detection method for surface damage of wind turbine blades in complex scenarios, such as Figure 1 As shown, it includes the following steps:

[0049] Step 1: Establish a dataset of blade surface damage images for model training and validation, as follows:

[0050] (1) Take images of surface damage on wind turbine blades at the wind farm;

[0051] (2) Manually clean, screen, and label the damage categories of the captured images;

[0052] (3) The damage data image is expanded by methods such as horizontal, vertical and mirror flipping.

[0053] Step 2: Fuse the ASM model and the Retinex model to enhance the damage in the low-light images of the leaf damage dataset. (Refer to...) Figure 2 The details are as follows:

[0054] (1) Use the inverted image 1-L(x,y) of the original low-light image as the input to ASM, that is:

[0055]

[0056] Where L(x,y) represents the original image, R(x,y) represents the real scene without illumination, T(x,y) represents transmittance, which refers to the proportion of light energy that is transmitted to the observer's position when passing through particles in the atmosphere, and A represents atmospheric light in the scene; through derivation, and letting L... r (x,y)=1-[1-L(x,y)] / A,R r (x,y)=1-[1-R(x,y)] / A, resulting in the model:

[0057] L r (x,y)=R r (x,y)T(x,y)

[0058] The desired R(x,y) can be calculated by estimating the transmittance and atmospheric light in the current scene.

[0059] (2) The atmospheric ray A in the inverted image is estimated using the dark channel prior theory. The dark channel prior theory can be expressed as:

[0060]

[0061] Among them, I c (·) represents a certain color channel of image I, Ω x This represents a region window centered at x. To address the dense fog approximation characteristics of low-light inverted images, the brightest 0.1% of pixels in the dark channel image are selected, and the brightest pixel in the inverted image is determined based on its index position, thus estimating atmospheric light A.

[0062] (3) Using guided image filtering as the center wrap function and L r The transmittance T(x,y) is calculated by convolving (x,y).

[0063] (4) Based on the atmospheric light and transmittance obtained from (2) and (3), the real scene image is calculated by model inversion, and the RGB channel is quantized using the histogram equalization method with limited contrast to obtain the damaged blade image.

[0064] Step 3: Localize and segment the leaf regions in the constructed damage dataset to avoid interference from complex backgrounds in damage identification, achieving data augmentation. (Refer to...) Figure 3 Specifically, it includes:

[0065] (1) The image is initially divided by specifying the number or size of the initial semantic blocks, then Gaussian filtering is applied to the original image, the gradient of each pixel in the image is calculated using the Sobel operator, and the point with the lowest gradient in the semantic block is used as the initial seed point.

[0066] (2) Calculate the Euclidean distance between a pixel and its various neighboring sub-points in the feature space as the semantic block membership degree:

[0067]

[0068] Where, x i and d corresponds to the positions of the current pixel and the cluster center in the feature space, respectively. lab d represents the distance between two pixels in the CIELAB color space. xy d represents the relative distance between two pixels. s Represents the gradient distance between pixels; assigns pixels to the semantic block with the highest membership degree:

[0069]

[0070] in, The label to which the current pixel belongs is represented, K represents the number of semantic blocks, and i represents the domain. For any pixel within, c k Let represent the cluster center of the k-th semantic block; then, update the semantic labels of the pixels, calculate the mean vector of each semantic block, and iteratively update the cluster centers until the convergence condition is met:

[0071]

[0072] in, ε represents the center value of the semantic block in the t-th clustering result, and ε can take the value 0.01;

[0073] (3) Using the cluster centers of semantic blocks as samples, perform fuzzy clustering on the segmented semantic blocks. The objective function is:

[0074]

[0075] Where C is the number of clusters, K represents the number of semantic blocks, m represents the weighting coefficient, and γ i,k T represents the membership degree between the k-th semantic block and the i-th class. i This represents the i-th cluster center; the membership degree and cluster center are iteratively updated until the termination condition is:

[0076]

[0077] in, This represents semantic block c after the t-th update. k For cluster center T i The membership degree, ε, can be 0.01; the semantic labels are updated based on the clustering results to complete the segmentation and localization of the leaf region.

[0078] Step 4: Construct a deep network recognition model integrating ResNet and damage feature encoding, and train and validate the model using the damage dataset. Specifically, this includes:

[0079] (1) The ResNet network is used to extract local convolutional features from the segmented leaf sample images in stages. The number of feature vector channels is quantized through the convolutional layer, then linearized and flattened, and concatenated with the damage category vector to construct the feature space vector of the damaged area on the leaf surface. For the specific structure, please refer to [reference needed]. Figure 4 ;

[0080] Stage 0 involves 64 convolutional operations of size 7*7 and BatchNorm normalization, followed by a 2*2 MaxPool layer for image preprocessing. Stage 1 mainly consists of three residual modules for initial extraction of image damage features. The input vector is primarily extracted through 64 convolutional layers of size 3*3, followed by channel quantization through 256 convolutional layers of size 1*1. Stages 2 and 3 have similar structures to Stage 1 and are mainly used for deeper feature extraction. Assuming the original image size is H×W×C, where H and W represent the image resolution and C represents the number of channels, after three stages of convolutional feature extraction, the output dimension is (H / 16)×(W / 16)×1024. The feature vector channel number is adjusted through 768 convolutional layers of size 1*1, and the vector is linearly flattened to (HW / 256)×1024. The feature vector is then concatenated with the damage category vector to construct the feature space vector of the damaged area on the blade surface.

[0081] (2) The MHAM module can use the LayerNorm layer to normalize the regional feature space vector, learn the damage attention weights of different regions in parallel through MHAM, and the number of attention heads can be selected as 6. At the same time, the residual network structure is used to connect the input and MHAM output across layers to quickly capture global damage features.

[0082] (3) The multilayer perceptron module can normalize the feature encoding vector through the LayerNorm layer, use the fully connected layer to upsample and downsample the output of MHAM by 4 times, randomly set the output value of 50% of neurons to 0 through the DropOut regularization method, use GeLu as the activation function, and then use the residual network to superimpose the input and output vectors to learn to identify the differences between damaged and non-damaged areas.

[0083] (4) Iterate the structures of (2) and (3) 12 times. For details, please refer to [the relevant documentation]. Figure 5 Its output is passed through a fully connected layer and a SoftMax function, and the model determines the damage type of the input image based on its results;

[0084] (5) Randomly divide the samples in the augmented dataset, use 80% as training set data to train the damage recognition model, and use 20% as validation set to test the accuracy of defect recognition, so as to obtain a well-trained damage recognition model.

Claims

1. A visual detection method for surface damage of wind turbine blades in complex scenarios, characterized in that, The method includes the following steps: S1. Collect images of different types of blade damage in the field wind field, manually clean, filter and label the damage categories, expand the damage samples by horizontal, vertical and mirror flipping, and establish a blade surface damage image dataset; S2. To address the issues of low contrast and blurred features in leaf damage areas under low light conditions, the leaf surface damage image dataset is enhanced by fusing the Retinex model and the Atmospheric Scattering Model (ASM). Step S2 specifically includes: S201. Invert the original low-light leaf image and use it as input to the ASM model to derive and establish a Retinex model: in, , Represents the original image. This represents a realistic scene without the influence of lighting. Transmittance refers to the proportion of light energy that is transmitted to the observer's location when it passes through particulate matter in the atmosphere. This represents atmospheric light in the scene; by estimating the transmittance of the current scene and atmospheric light, the desired value is calculated through inversion. ; S202. To address the dense fog approximation characteristics of low-light inverted images, the brightest 0.1% of pixels in the dark channel image are selected, and the pixel with the highest brightness in the inverted image is determined based on its index position, thus estimating the atmospheric light intensity. ; S203. Using the guided image filtering as the central wrap function, through... Convolution operation to estimate transmittance ; S204. Based on S202 and S203, atmospheric light and transmittance are obtained. The real scene image without illumination is calculated by model inversion. The RGB channels are quantized using a histogram equalization method with limited contrast. S3. A semantic block segmentation and clustering method with complementary multidimensional information is used to locate and segment the leaf region in the augmented dataset, avoiding interference from terrain background information on damage identification; S4. Establish a deep network recognition model that integrates a residual network (ResNet) and damage feature encoding, and use the augmented dataset to train and test the blade damage recognition model.

2. The method for visually detecting surface damage to wind turbine blades in complex scenarios according to claim 1, characterized in that, Step S3 includes: S301. Perform initial semantic block division and semantic label assignment on the enhanced leaf image, solve the gradient magnitude of each pixel in the image, and select the point with the lowest gradient in the semantic block as the initial seed. S302. Calculate the Euclidean distance between a pixel and its various neighboring sub-points in the feature space as the semantic block membership degree: in, and These correspond to the positions of the current pixel and the cluster center in the feature space, respectively. This represents the distance between two pixels in the CIELAB color space. This represents the relative distance between two pixels. This represents the gradient distance between pixels; Assign pixels to the semantic block with the highest membership degree: in, This indicates the label to which the current pixel belongs. Indicates the number of semantic blocks. Representation of domain Any pixel within, Indicates the first The cluster centers of each semantic block are determined; then, the semantic labels of the pixels are updated, and the mean vector of each semantic block is calculated to iteratively update the cluster centers until the convergence condition is met. in, Indicates the semantic block number The center value of the sub-clustering result; S303. Using the cluster centers of semantic blocks as samples, perform fuzzy clustering on the segmented semantic blocks. The objective function is: in, It is the number of clusters. Indicates the weighting coefficient. Indicates the first The semantic block and the first Membership degree of each class Indicates the first There are 1 cluster center; the termination condition for the iteration is: in, Indicates the first semantic block after the next update Cluster centers The membership degree is determined; finally, the semantic labels are updated based on the clustering results to complete the segmentation and localization of the leaf region.

3. The method for visually detecting surface damage to wind turbine blades in complex scenarios according to claim 1, characterized in that, Step S4 includes: S401. The ResNet network is used to extract the local convolutional features of the segmented leaf sample images in stages. The number of feature vector channels is quantized through the convolutional layer, and then linearized and flattened and concatenated with the damage category vector to construct the feature space vector of the damage area on the leaf surface. S402. Normalize the regional feature space vector, learn the damage attention weights of different regions in parallel through the multi-head attention mechanism (MHAM), and at the same time use the residual network structure to connect the input and MHAM output across layers to quickly capture the surface damage features of the blade at different scales. S403. Normalize the feature encoding vector, use a fully connected layer to perform upsampling and downsampling by the same multiple, randomly set the output value of some neurons to 0 through the DropOut regularization method, and then use a residual network to superimpose the input and output vectors to learn and identify the differences between the damaged and undamaged areas of the wind turbine blade. S404. Iterate S402 and S403 L times, and use the output results through a fully connected layer and the SoftMax function to determine the damage mode of the input blade image; use the damage dataset with completed blade segmentation for model training and accuracy verification.

Images