A fan blade surface defect identification system and method

Abstract

This invention discloses a system and method for identifying surface defects on wind turbine blades. The system includes an FCCNN network model based on VGG16 for surface defect identification. The FCCNN network model comprises a feature extractor and a classifier. The feature extractor includes several convolutional layers, several max-pooling layers, and several global average pooling layers. The classifier includes several dropout layers and several fully connected layers. The convolutional layers in the feature extractor use weights pre-trained based on ImageNet as initial weights. This invention solves the problem of low accuracy in computer vision-based identification of product surface defects, achieving accurate identification of product surface defects. It has good applicability and effectiveness in defect identification, enabling more accurate defect identification, effectively ensuring product quality, and providing accurate reference for process parameter optimization and production control management decisions.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, specifically relating to a system and method for identifying surface defects in wind turbine blades. Background Technology

[0002] Surface defects are widespread in products such as metals, ceramics, wood, and textiles. Taking a certain steel product as an example, approximately 95.4% of its defects occur on the product surface, causing a significant amount of unnecessary production waste. Product surface defects generally refer to localized abnormalities caused by changes in physical or chemical properties or uneven distribution. Common product surface defects include scratches (steel), inclusions (glass, ceramics), and breakage (textiles). Surface defects severely affect the appearance and comfort of products, hindering sales and causing substantial economic losses. Furthermore, they can affect product performance, such as surface friction, leading to safety hazards during use. Therefore, accurately identifying product surface defects has become one of the key issues that urgently needs to be addressed in product quality control.

[0003] From the perspective of convolutional neural network models, most existing methods only make simple adjustments to the model without in-depth exploration, and the recognition accuracy still needs further improvement. Compared with ordinary image recognition problems, defect recognition problems are more difficult to train because the images are more complex and the distinction between the defective subject and the background is low. Therefore, further analysis is needed to develop a guiding mechanism to guide the entire learning process of the convolutional neural network and achieve better recognition results. Summary of the Invention

[0004] To address the technical problems existing in the prior art, the present invention aims to provide a system and method for identifying surface defects in wind turbine blades.

[0005] To achieve the above objectives and technical effects, the technical solution adopted by this invention is as follows:

[0006] A wind turbine blade surface defect recognition system includes an FCCNN network model for surface defect recognition based on VGG16. The FCCNN network model includes a feature extractor and a classifier. The feature extractor includes several convolutional layers, several max pooling layers and several global average pooling layers. The classifier includes several dropout layers and several fully connected layers. The convolutional layers in the feature extractor all use weights pre-trained based on ImageNet as initial weights.

[0007] Furthermore, the feature extractor adopts a VGG16 network structure, and the activation function in the feature extractor is the ReLU activation function. The feature extractor includes five modules arranged in sequence. The first and second modules each contain two convolutional layers and one max pooling layer formed by sequential connection. The third and fourth modules each contain three convolutional layers and one max pooling layer formed by sequential connection. The fifth module contains three convolutional layers and one global average pooling layer formed by sequential connection. The first, second, third, and fourth modules are separated by max pooling layers.

[0008] Furthermore, the number of convolutional kernels corresponding to the convolutional layers are [64, 128, 256, 512, 512], and the size of each convolutional kernel is 3*3.

[0009] Furthermore, the pooling window of the maximum pooling layer is 2*2.

[0010] Furthermore, the classifier is a 5-layer multilayer perceptron, which includes one Dropout layer and two fully connected layers formed by sequential connections. Except for the last layer, which uses the Softmax activation function to output the probability corresponding to each defect category, the classifier uses the ReLU activation function as the activation function.

[0011] A method for identifying surface defects in wind turbine blades includes the following steps:

[0012] Step 1: Construct an FCCNN network model with dual outputs of feature and recognition label based on VGG16;

[0013] Step 2: Import the ImageNet-pretrained weights into the feature extractor of the FCCNN network model;

[0014] Step 3: Perform feature learning. Train the feature extractor and classifier alternately through feature clustering mechanism and minimizing recognition error to obtain the loss function of the feature extractor and classifier, and optimize the FCCNN network model.

[0015] Step 4: After training, input the image to be tested into the model for recognition and obtain the defect classification result.

[0016] Furthermore, in step one, the FCCNN network model includes a feature extractor and a classifier. The feature extractor includes several convolutional layers, several max pooling layers, and several global average pooling layers. The classifier includes several dropout layers and several fully connected layers. The convolutional layers in the feature extractor all use weights pre-trained based on ImageNet as initial weights. The FCCNN network model adopts an alternating iterative optimization method. The feature extractor iterates once for every 10 iterations of the classifier.

[0017] Furthermore, in step three, the construction of the loss function for the feature extractor includes:

[0018] Let the features output by the feature extractor be h. First, based on the actual category y∈[1,2,3,…,m] corresponding to each feature (i.e., there are m categories in the actual labels), h is divided into m feature subspaces Si∈m. Then, based on the confidence y′ corresponding to each feature, the features in each feature subspace Si are sorted from high to low, thus forming a candidate set Cs_i for cluster centers of each feature subspace. Next, the top 30% of features are selected from the candidate set Cs_i, and these features are clustered using the 1-NN method to determine the top_k feature cluster centers, and the feature vector corresponding to each center is calculated. Finally, the Euclidean distance is used to calculate the nearest cluster center of each feature in the current feature subspace, thus determining the target vector t corresponding to each feature hi. i Determine the target vector t corresponding to each feature. i Then, the loss function Lcluster for constructing the feature extractor based on the mean squared error is:

[0019]

[0020] Furthermore, in step three, the loss function of the classifier is:

[0021]

[0022] Where n is the total number of samples, y i ' represents the probability that the current input belongs to the i-th class, y' represents the probability that the input belongs to the i-th class. i The true label is the defect's real label. The FCCNN network model's learning method is labeled learning. The training set consists of labeled images of blades with surface defect types (for example, if a wind turbine blade image shows a crack defect, then its true label is the code corresponding to the crack). Users need to prepare their own training set for their products, preferably more than 2000 images.

[0023] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0024] This invention discloses a wind turbine blade surface defect identification system and method, the system including an FCCNN network model based on VGG16 for surface defect identification. This method involves downloading the node weights of a VGG16 network trained on ImageNet from the Keras library after constructing the network structure, importing them into the constructed model, and then inputting a set of images of wind turbine blade defects to train the network. After training, an FCCNN network for wind turbine blade recognition is obtained. Finally, it is used in factories. The factory installs industrial cameras to take pictures of the blade surface and inputs the photos into the network. The network will provide the most likely defect classification, which is convenient for later viewing and analysis by staff. This method solves the problem of low recognition accuracy in computer vision for product surface defects, achieving accurate identification of product surface defects. It has good applicability and effectiveness in defect identification problems, enabling more accurate defect identification, effectively ensuring product quality, and providing accurate reference for process parameter optimization and production control management decisions, thus strongly promoting the intelligentization and automation of the production process. Users can deploy industrial cameras in the production inspection line to take pictures of each product and send them to the network model to determine whether it has surface defects. The operation is simple, convenient, and efficient. It can be used for the diagnosis of factory defects in wind turbine blades or for other products with simple surface structures. Attached Figure Description

[0025] Figure 1 This is a flowchart of the present invention;

[0026] Figure 2 This is a schematic diagram of the FCCNN network model of the present invention. Detailed Implementation

[0027] The present invention will now be described in detail so that its advantages and features can be more easily understood by those skilled in the art, thereby providing a clearer and more explicit definition of the scope of protection of the present invention.

[0028] The following provides a brief overview of one or more aspects to offer a basic understanding of them. This overview is not an exhaustive summary of all conceived aspects, nor is it intended to identify key or decisive elements of all aspects, nor to define the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form to prepare for the more detailed descriptions that follow.

[0029] Definitions:

[0030] VGG: Visual Geometry Group;

[0031] ImageNet: A set of images used for deep learning training;

[0032] GAP stands for Global Average Pooling, used for model fine-tuning.

[0033] Softmax: A neural network activation function;

[0034] ReLU: A neural network activation function;

[0035] Dropout: Used to selectively shut down neural nodes to avoid overfitting;

[0036] Keras Library: Keras is a high-level neural network library written in pure Python.

[0037] like Figure 1-2 As shown, a wind turbine blade surface defect recognition system includes an FCCNN network model based on VGG16 for surface defect recognition. This network model includes a feature extractor and a classifier. The feature extractor adopts a VGG16 network structure and includes five modules, separated by max pooling layers. The first two modules, namely the first and second modules, each contain two convolutional layers and one max pooling layer. The remaining three modules each contain three convolutional layers and one pooling layer. In the five modules of the feature extractor, the number of convolutional kernels corresponding to each convolutional layer are [64, 128, 256, 512, 512], and the size of each convolutional kernel is 3*3. In the five modules, the pooling layers of the first four modules all adopt the max pooling strategy, and their pooling layer windows are 2*2. The pooling layer of the last module adopts the Global Average Pooling (GAP) layer. Meanwhile, all convolutional layers in the feature extractor use weights pre-trained on ImageNet as initial weights, and can directly load initialization parameters pre-trained on ImageNet models.

[0038] FCCNN uses an alternating iterative optimization approach, where the feature extractor iterates once for every 10 iterations of the classifier.

[0039] First, the loss function of the feature extractor is introduced. Let the feature output by the feature extractor be h. This invention first divides h into m feature subspaces Si∈m based on the actual category y∈[1,2,3,…,m] corresponding to each feature, i.e., there are m categories in the actual labels. Then, based on the confidence score y′ corresponding to each feature, the features in each feature subspace Si are sorted from high to low, thus forming a candidate set Cs_i for cluster centers in each feature subspace. Next, the top 30% of features are selected from the candidate set Cs_i, and these features are clustered using the 1-NN method to determine the top_k feature cluster centers, and the feature vector corresponding to each center is calculated. Finally, the Euclidean distance is used to calculate the nearest cluster center for each feature in the current feature subspace, thereby determining the target vector t corresponding to each feature hi. i Determine the target vector t corresponding to each feature. i Then, the loss function Lcluster for constructing the feature extractor based on the mean squared error is:

[0040]

[0041] The classifier is a 5-layer multilayer perceptron. To further improve the defect recognition performance of FCCNN, a Dropout layer or feature output layer is added to the fully connected layer classifier to extract the features learned by the FCCNN model and perform clustering operations on the feature space. The loss function of the classifier is:

[0042]

[0043] Where n is the total number of samples, yi′ is the probability that the current input belongs to the i-th class, and yi is the true label.

[0044] Meanwhile, to align with the ImageNet-based pre-training parameters, the feature extractor uses the ReLU activation function. In the classifier, the parameters of the Dropout layer are set to 0.1-0.5, preferably 0.2, meaning each neuron has a 20% chance of being turned off during training. Except for the last layer, which uses the Softmax activation function to output the probability corresponding to each defect category, all other layers use ReLU as the activation function.

[0045] A method for identifying surface defects in wind turbine blades includes the following steps:

[0046] Step 1: Construct an FCCNN network model with dual outputs of feature and recognition label based on VGG16;

[0047] Step 2: Import the pre-trained weights based on ImageNet images into the feature extractor of the FCCNN network model.

[0048] The weights here are the node weights of a pre-trained model (a VGG16 network trained on ImageNet images). This model can be found directly in the Keras library, a high-level neural network library written entirely in Python. The Python code is as follows:

[0049] model = NET1()

[0050] state_dict=model.state_dict()

[0051] weights = torch.load(weights_path)['model_state_dict'] # Read the pre-trained model weights

[0052] model.load_state_dict(weights)

[0053] #NET1 is the model we want to train, and #model_state_dict is the weight file of the pre-trained model;

[0054] Step 3: Perform feature learning. Train the feature extractor and classifier alternately through feature clustering mechanism and minimizing recognition error to obtain the loss function of the feature extractor and classifier, and optimize the FCCNN network model.

[0055] In step three, FCCNN employs an alternating iterative optimization approach. For every 10 iterations of the classifier, the feature extractor iterates once. That is, for every 10 iterations of the classifier, the feature extractor performs one iteration, striving to minimize their respective loss functions. This can be achieved by adding a counter in the code; every ten counts for the classifier, the feature extractor iterates once.

[0056] Step 4: After training, input the image to be tested into the model for recognition and obtain the defect classification result.

[0057] In step four, the image to be tested is obtained by taking a picture of the product using an industrial camera, and then the image to be tested is input into the FCCNN network model.

[0058] In step one, the FCCNN network model includes several convolutional layers, several max pooling layers, several global average pooling layers, several fully connected layers, and several Dropout layers.

[0059] As a specific implementation, the FCCNN network model includes a feature extractor and a classifier. The feature extractor uses a VGG16 network structure and comprises five modules spaced apart by max-pooling layers. The first two modules each contain two convolutional layers and one max-pooling layer, while the remaining three modules each contain three convolutional layers and one pooling layer. The number of kernels in each convolutional layer within the five modules is [64, 128, 256, 512, 512], and the kernel size is 3*3. The pooling layers in the first four modules employ max-pooling with a 2-2 window, while the pooling layer in the last module uses Global Average Pooling (GAP). All convolutional layers use weights pre-trained on ImageNet as initial weights. The classifier is a 5-layer multilayer perceptron, consisting of two fully connected layers and one Dropout layer. To further improve the defect recognition performance of FCCNN, a Dropout layer or feature output layer is added before the fully connected layers to extract the features learned by the FCCNN model and perform clustering operations on the feature space. Feature extraction is performed through the convolutional layers, max pooling layers, and global average pooling layers of the feature extractor to obtain defect features, which are then input into the Dropout layer of the classifier. The features are then classified through the Dropout layer and the fully connected layers to obtain the defect classification result.

[0060] In step three, the construction steps of the loss function Lcluster for the feature extractor include:

[0061] Let the feature output by the feature extractor be h. First, based on the actual category y∈[1,2,3,…,m] corresponding to each feature (i.e., there are m categories in the actual labels), h is divided into m feature subspaces Si∈m. Then, based on the confidence score y′ corresponding to each feature, the features in each feature subspace Si are sorted from high to low, thus forming a candidate set Cs_i for cluster centers of each feature subspace. Next, the top 30% of features are selected from the candidate set Cs_i, and these features are clustered using the 1-NN method to determine the top_k feature cluster centers, and the feature vector corresponding to each center is calculated. Finally, the Euclidean distance is used to calculate the nearest cluster center of each feature in the current feature subspace, thus determining the cluster center of each feature h. i The corresponding target vector t i Determine the target vector t corresponding to each feature. i Then, the loss function Lcluster for constructing the feature extractor based on the mean squared error is:

[0062]

[0063] The loss function of the classifier is:

[0064]

[0065] Where n is the total number of samples, yi′ is the probability that the current input belongs to the i-th class, and yi is the true label.

[0066] Meanwhile, to align with the ImageNet-based pre-training parameters, the feature extractor uses the ReLU activation function. In the classifier, the parameters of the Dropout layer are set to 0.1-0.5, preferably 0.2, meaning each neuron has a 20% chance of being turned off during training. Except for the last layer, which uses the Softmax activation function to output the probability corresponding to each defect category, the remaining layers all use ReLU as the activation function.

[0067] Compared with the prior art, the present invention has at least the following beneficial effects:

[0068] Existing technologies, lacking the input of ImageNet pre-trained weights into the FCCNN network model for direct feature learning, are prone to misleading the model's training process and causing it to get trapped in local optima. This invention inputs ImageNet pre-trained weights into the FCCNN network model, which helps improve the recognition rate of blade surface defects. After adding a feature clustering mechanism, the recognition effect is improved to a certain extent. The network training time of this invention is shorter, and the recognition accuracy is higher.

[0069] Any parts or structures not specifically described in this invention can be made using existing technologies or products, and will not be elaborated upon here.

[0070] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention specification, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A wind turbine blade surface defect identification system, characterized by, The FCCNN network model for surface defect identification based on VGG16 includes a feature extractor and a classifier. The feature extractor includes several convolutional layers, several max pooling layers and several global average pooling layers. The classifier includes several dropout layers and several fully connected layers. The convolutional layers in the feature extractor all use weights pre-trained based on ImageNet as initial weights. The feature extractor adopts a VGG16 network structure. The activation function in the feature extractor is ReLU activation function. The feature extractor includes five modules arranged in sequence. The first and second modules each contain two convolutional layers and one max pooling layer formed by sequential connection. The third and fourth modules each contain three convolutional layers and one max pooling layer formed by sequential connection. The fifth module contains three convolutional layers and one global average pooling layer formed by sequential connection. The first, second, third and fourth modules are separated by max pooling layers. The number of convolutional kernels corresponding to the convolutional layers are [64, 128, 256, 512, 512], and the size of each convolutional kernel is 3*3; The pooling window of the maximum pooling layer is 2*2; The classifier is a 5-layer multilayer perceptron, which includes one Dropout layer and two fully connected layers formed by sequential connections. Except for the last layer, which uses the Softmax activation function to output the probability corresponding to each defect category, the classifier uses the ReLU activation function as the activation function. The wind turbine blade surface defect identification method of the wind turbine blade surface defect identification system includes the following steps: Step 1: Construct an FCCNN network model with dual outputs of feature and recognition label based on VGG16; Step 2: Import the ImageNet-pretrained weights into the feature extractor of the FCCNN network model; Step 3: Perform feature learning. Train the feature extractor and classifier alternately through feature clustering mechanism and minimizing recognition error to obtain the loss function of the feature extractor and classifier, and optimize the FCCNN network model. Step 4: After training, input the image to be tested into the model for recognition and obtain the defect classification result; In step one, the FCCNN network model includes a feature extractor and a classifier. The feature extractor includes several convolutional layers, several max pooling layers, and several global average pooling layers. The classifier includes several dropout layers and several fully connected layers. The convolutional layers in the feature extractor all use weights pre-trained based on ImageNet as initial weights. The FCCNN network model adopts an alternating iterative optimization method. The feature extractor iterates once for every 10 iterations of the classifier. Step three, the steps for constructing the loss function of the feature extractor include: Let h be the feature output by the feature extractor; first, based on the actual category corresponding to each feature... [1,2,3,⋯ ,m], meaning there are m categories in the actual labels, and h is divided into m feature subspaces Si∈m; then, based on the confidence y′ corresponding to each feature, the features in each feature subspace Si are sorted from high to low, thus forming a candidate set Cs_i for cluster centers of each feature subspace; then, the top 30% of features are selected from the candidate set Cs_i, and these features are clustered using the 1-NN method to determine the top_k feature cluster centers, and the feature vector corresponding to each center is calculated; then, the Euclidean distance is used to calculate all features in the current feature subspace and their nearest cluster centers, thus determining the target vector t corresponding to each feature ℎi. i Determine the target vector t corresponding to each feature. i Then, the loss function Lcluster for constructing the feature extractor based on the mean squared error is: ； In step three, the loss function of the classifier is: ； where n is the number of all samples, y i is the probability that the current input belongs to the i-th class, y i is the true label of the defect.

Images