A method for detecting surface defects of wind turbine blades based on sharing of multiple feature maps

Abstract

The present invention discloses a method for detecting surface defects of fan blades based on sharing of multiple feature maps, which comprises the following steps in sequence: obtaining a target image of fan blade defects and establishing a data set; obtaining a target image of fan blade defects; data screening; labeling; making a VOC data set; target image preprocessing; image graying; Gaussian filtering; Canny edge detection; histogram equalization; network building, training and tuning; building a backbone network; designing a sharing unit; building a multi-feature map sharing network; training set data enhancement; model training and tuning. In order to achieve real-time detection and improve the accuracy of defect recognition.

Description

Technical Field

[0001] The present invention relates to the field of surface defect detection, and in particular to a method for detecting surface defects of wind turbine blades based on sharing of multiple feature maps. Background Art

[0002] Since wind turbines are usually installed in areas with harsh climatic conditions and are subjected to long-term high-load operation, the blades of the wind turbines are very likely to fail, causing a significant decrease in the efficiency of the wind turbines and triggering a series of failures.

[0003] Traditional wind turbine blade inspection relies on manual observation, which requires staff to observe each blade one by one, or after the wind turbine blades are shut down, workers will descend by rope to inspect each blade.

[0004] In recent years, traditional image detection methods have been applied to the detection of surface defects of wind turbine blades. Traditional image detection methods first require region selection, then feature extraction, and then classification through machine learning methods, and finally post-processing.

[0005] Among the above existing methods, manual inspection has low efficiency and safety, and is easily affected by the environment. The accuracy of wind turbine blade surface defect detection depends on the worker's prior knowledge. Traditional image processing methods have poor generalization ability and need to design different algorithms for different defects. Secondly, when designing algorithms, they also rely on expert experience and require high prior knowledge. In addition, each processing takes a long time and cannot use the current advanced GPU computing resources. Due to the harsh environment of wind turbines and the complex surface structure of wind turbine blades, single-task learning cannot meet the requirements of real-time and practicality in surface defect detection tasks, resulting in large defect detection errors. Summary of the invention

[0006] The purpose of the present invention is to provide a method for detecting surface defects of wind turbine blades based on sharing of multiple feature maps, so as to achieve real-time detection and improve the accuracy of defect identification.

[0007] The objective of the present invention is achieved in this way: a method for detecting surface defects of fan blades based on sharing of multiple feature maps, comprising the following steps in sequence: step S100, obtaining a target image of fan blade defects and establishing a data set; step S110, obtaining a target image of fan blade defects; step S120, data screening; step S130, labeling; step S140, making a VOC data set; step S200, target image preprocessing; step S210, image grayscale; step S220, Gaussian filtering; step S230, Canny edge detection; step S240, histogram equalization; step S300, network construction, training and tuning; step S310, building a backbone network; step S320, designing a shared unit; step S330, building a multi-feature map sharing network; step S340, training set data enhancement; step S350, model training and tuning.

[0008] The beneficial effects of the present invention are:

[0009] High efficiency, strong timeliness, and low risk; high accuracy and strong generalization ability; high flexibility, and the backbone network can be flexibly configured according to the specific GPU resources in the hands of the customer and the accuracy requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 It is a schematic diagram of the process of the present invention.

[0011] Figure 2 This is a thumbnail of the ResNet network structure.

[0012] Figure 3 This is a schematic diagram of a shared unit.

[0013] Figure 4 It is a schematic diagram of the multi-feature graph sharing network structure.

[0014] Figure 5 Figure 1. Network configuration diagram for ResNet101. DETAILED DESCRIPTION

[0015] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments.

[0016] like Figure 1-4 As shown, a method for detecting surface defects of wind turbine blades based on sharing of multiple feature maps includes the following steps.

[0017] Step S100, acquiring a target image of fan blade defects and establishing a data set.

[0018] Step S110, image acquisition, obtains a target image of the defects of the wind turbine blade, and the target image can be obtained by a terminal device with a camera function, such as a mobile phone, a camera, a drone, etc. The target image should contain the overall information of the wind turbine blade, including surface defects, noise and other information.

[0019] Step S120, data screening: sort and classify the target images, remove target images with incomplete information, and remove images with a large number of tiny defects that are not conducive to defect classification.

[0020] Step S130: labeling. This method is defect detection based on supervised learning. After obtaining a useful target image, the target image is labeled with the defect type using LabelImage software.

[0021] Step S140, create a VOC data set. After labeling the image using LabelImage software, generate an xml file in the VOC data set format. The xml file content is in a tree structure, including information such as file path, file name, defect name, defect location, etc.

[0022] Step S200: preprocessing the target image. A key point of sharing multiple feature maps is to obtain associated feature maps.

[0023] Step S210: grayscale the image. Perform grayscale processing on the target image to obtain target image 1. The grayscale processing formula is as follows:

[0024] gray(x,y)=0.299*Red(x,y)+0.587*Green(x,y)+0.114*Blue(x,y)(1);

[0025] Perform grayscale processing on the target image, convert the target image into a grayscale image for dimensionality reduction, reduce the amount of input data, and facilitate subsequent processing.

[0026] Step S220, Gaussian filtering. Perform Gaussian filtering on target image 1 to obtain target image 2. The Gaussian kernel used in Gaussian filtering is the product of two one-dimensional Gaussians x and y. The standard deviations σ in the two dimensions are usually the same. The processing formula is as shown in formula (2):

[0027]

[0028] The wind farm environment is complex and affected by light, which will cause noise in the acquired target image. Use Gaussian filtering to smooth the target image and reduce the noise in the target image.

[0029] Step S230, Canny edge detection. After binarization of target image 2, Canny edge detection is performed to obtain target image 3 (feature map C). Canny edge detection is not easily affected by noise, and can identify strong and weak edges of the target image, and comprehensively give the overall edge information of the image by combining the positional relationship between the strong and weak edges.

[0030] Step S240, histogram equalization. Due to factors such as illumination, the contrast of the target image is relatively small, which is not convenient for texture recognition in the image. Histogram equalization (feature map A) is performed on the target image 2 to obtain the target image 4, thereby improving the image contrast and highlighting the texture features in the image.

[0031] Step S300, build the network, train and tune. The backbone network used in this patent can be replaced, and VGGNet, InceptionNet, etc. can be used. This example uses ResNet101.

[0032] Step S310, building a backbone network. This example proposes a network based on sharing multiple feature maps, using ResNet101 as the backbone network to complete the joint learning of three related feature maps.

[0033] ResNet101 was proposed by He Kaiming in 2015 and won the ILSVRC championship that year. ResNet proposed inter-layer residual skipping and introduced forward information, which effectively solved the problem of gradient disappearance caused by the depth of deep learning neural networks. Common ResNet network depths include 34, 50, 101, 152, etc. This example selects 101-layer ResNet. Figure 5 As shown, this is the ResNet101 network configuration table.

[0034] Before pooling, BatchNorm is performed to accelerate the convergence of the neural network and improve the stability of the training process. BatchNorm normalizes the data of a batch according to the format.

[0035] After batch normalization, the Relu activation function is used for nonlinearization, which solves the gradient vanishing problem of Sigmoid and makes the convergence faster.

[0036] The pooling layer selects Max Pooling.

[0037] Use random dropout technology to randomly ignore some neurons in training to prevent overfitting.

[0038] Finally, the feature map is expanded into a vector through global average pooling and input into the fully connected layer.

[0039] In this paper, the number of neurons in the fully connected layer is 1000, but in practice it is set according to the number of classifications required, 2 neurons for binary classification and 10 neurons for 10 classifications.

[0040] Step S320, designing a shared unit. In this example, a shared unit is added after each maximum pooling layer and before the fully connected layer based on the backbone network. The shared unit completes the feature sharing of the three feature maps and ensures that the data dimension of the input to the next layer of the network does not change.

[0041] A shared unit is designed, the specific structure is as follows Figure 4 As shown. And by introducing a series of weight parameters [α i1 , α i2 , α i3 , α i4 ], the feature maps of the specific layers between tasks are shared through the linear combination of equations (3) to (5), that is, feature map A is shared with feature map B and feature map C at the same time, and feature map B and feature map C are only shared with feature map A separately. The shared feature maps are then normally input into their respective backbone networks to complete the subsequent learning and training. Among them, each parameter should satisfy the constraint relationship of equation (6) when the network is initialized, but the learning of the parameters is not constrained during the training process. The purpose is to obtain the most effective linear combination weights through autonomous learning.

[0042]

[0043] Step S330: Build a multi-feature graph sharing network. Based on the previously built backbone network and sharing units, build a multi-feature graph sharing network. The network structure is as follows: Figure 5 .

[0044] Step S340: training set data enhancement: rotating, translating, scaling, flipping, and other operations are performed on the target image of the data set to enhance the data set to support the massive data requirements of the deep learning model.

[0045] Step S350: Model training and tuning. The prepared data set is divided into a ratio of 7:2:1, with 70% as a training set, 20% as a validation set, and 10% as a test set.

[0046] Set training parameters. Set the number of iterations (Rpoch) to 200. The more iterations, the more accurate the training results. However, it should not be too many to avoid overfitting. Set the learning rate to 0.01. A high learning rate is conducive to the network's rapid convergence to the optimal value, but it will cause instability. A large learning rate is used in the early stage of training, which is conducive to rapid convergence of the network. As the training continues, the learning rate will decay. Polynomial decay is used to make the network reach the optimal state. Set the training loss function and use Dice Loss+Cross entropy loss. Dice loss is suitable for situations where the data set samples are unevenly distributed, and Cross entropy loss is suitable for multi-classification tasks. The two are combined to optimize the cross entropy loss function to give it the advantages of the Dice loss function. The optimization algorithm uses Adam, which is more stable and suitable for larger data sets.

[0047] The above are preferred embodiments of the present invention. A person skilled in the art may make various changes or improvements on this basis. Without departing from the general concept of the present invention, these changes or improvements should fall within the scope of protection required by the present invention.

Claims

1. A method for detecting surface defects of wind turbine blades based on sharing of multiple feature maps, characterized in that: The method comprises the following steps in sequence: step S100, obtaining a target image of a defect in a fan blade and establishing a data set; step S110, obtaining a target image of a defect in a fan blade; step S120, data screening; step S130, labeling; Step S140, creating a VOC data set; Step S200, preprocessing the target image; Step S210, image grayscale, grayscale processing is performed on the target image, the target image is converted into a grayscale image for dimensionality reduction, target image one is obtained, and target image one is defined as feature map A; step S220, Gaussian filtering, Gaussian filtering is performed on target image one to obtain target image two; step S230, Canny edge detection, target image two is binarized and then subjected to Canny edge detection, target image three is obtained, and target image three is defined as feature map C; step S240, histogram equalization, histogram equalization is performed on target image two to obtain target image four, and target image four is defined as feature map B, image contrast is improved, and texture features in the image are highlighted; step S300, network construction, training and tuning; step S310, backbone network construction; step S320, shared unit design; step S330, multi-feature map sharing network construction; step S340, training set data enhancement; step S350, model training and tuning; In step S320, based on the backbone network, a shared unit is added after each maximum pooling layer and before the fully connected layer, and the feature sharing of the three feature maps is completed in the shared unit, and the data dimension input to the next layer of the network is ensured not to change; By introducing the weight parameter [α i1 , α i2 , α i3 , α i4 ], the feature maps of the specific layers between tasks are shared by the linear combination of equations (3) to (5), that is, feature map A is shared with feature map B and feature map C at the same time, and feature map B and feature map C are only shared with feature map A separately. The shared feature maps are then normally input into their respective backbone networks to complete subsequent learning and training; Each parameter should satisfy the constraint relationship of formula (6) when the network is initialized, but no constraint is imposed on the learning of parameters during the training process. The purpose is to obtain the most effective linear combination weights through autonomous learning.

2. The method for detecting surface defects of wind turbine blades based on sharing of multiple feature maps according to claim 1 is characterized in that: In step S120, the target images are sorted and classified, and target images with incomplete information are removed, and images with a large number of tiny defects that are not conducive to defect classification are removed.

3. The method for detecting surface defects of wind turbine blades based on sharing of multiple feature maps according to claim 2 is characterized in that: In step S130, after obtaining a useful target image, the defect type of the target image is labeled using LabelImage software.

4. The method for detecting surface defects of wind turbine blades based on sharing of multiple feature maps according to claim 1 is characterized in that: In step S330, a multi-feature graph sharing network is built based on the previously built backbone network and sharing unit.

5. The method for detecting surface defects of wind turbine blades based on sharing of multiple feature maps according to claim 4 is characterized in that: In step S340, the target image of the data set is rotated, translated, scaled, flipped, and other operations are performed to enhance the data set to support the massive data requirements of the deep learning model; in step S350, the prepared data set is divided into a training set, a validation set, and a test set in proportion.

Images