A method and system for health monitoring of offshore wind turbine blades

Abstract

The present invention relates to the technical field of health monitoring of offshore wind turbine blades, and discloses a health monitoring method and system for offshore wind turbine blades. A health monitoring method for offshore wind turbine blades includes the following steps: Step S101, collecting environmental data and vibration data; Step S102, training a first neural network model; Step S103, collecting ordinary images and infrared images; Step S104, constructing an image sequence; Step S105, training a second neural network model; Step S106, completing the health monitoring of the offshore wind turbine blade through the second neural network model. The present invention extracts features from the vibration data at multiple points through the first neural network model, migrates the first neural network model to the second neural network model to extract features from the ordinary images and infrared images, and uses the environmental data as the target of pre-training, thereby reducing the interference of environmental data on health monitoring and improving the monitoring accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of health monitoring of offshore wind turbine blades, and more specifically, to a method and system for health monitoring of offshore wind turbine blades. Background Art

[0002] With the continuous growth of global energy demand and the improvement of environmental protection awareness, offshore wind power generation, as a clean and renewable energy form, has received extensive attention. Compared with onshore wind farms, offshore wind power generation has the advantages of higher annual average wind speed and more stable wind speed. However, offshore wind power generation is exposed to high humidity, strong sea breeze, salt spray corrosion and extreme temperature changes for a long time, which accelerates the aging and damage of wind turbine blades. As a key structural component in wind turbine generators, the operating state of wind turbine blades directly affects power generation efficiency and equipment safety. Therefore, in order to ensure the safe operation of wind turbine generators, the health monitoring of offshore wind turbine blades is particularly important.

[0003] Currently, vibration sensors are used to collect vibration signals on the surface of the blades to analyze whether there are faults in the blades. However, there are certain similarities between the vibrations caused by strong winds or surges and the vibration signals of blade faults, and icing or snow cover will also affect the vibration of the blades, which is prone to misjudgment. In addition, high-definition cameras are used to collect image data on the surface of the blades to analyze whether there are faults in the blades. However, image data can only capture the defect information on the surface of the blades and it is difficult to detect hidden faults inside the blades (such as internal cracks, loose connections, etc.).

[0004] Therefore, there is an urgent need for a method for health monitoring of offshore wind turbine blades to solve the above problems. Summary of the Invention

[0005] The present invention provides a method and system for health monitoring of offshore wind turbine blades to solve the technical problems in the above background art.

[0006] The present invention provides a method for health monitoring of offshore wind turbine blades, including the following steps:

[0007] Step S101, collect the environmental data at the location of the offshore wind turbine blade and collect the vibration data at M points on the offshore wind turbine blade;

[0008] The environmental data includes: temperature, rainfall, snowfall, wind speed, and the angle between the wind direction and the blade orientation;

[0009] The vibration data at M points are all represented by the acceleration values at K time points;

[0010] Step S102, train the first neural network model with the environmental data and the vibration data at M points;

[0011] The input of the first neural network model is the vibration data of M points, and the output is the first health state of the offshore wind turbine blade. The first health state includes: normal blade, ice accumulation on the blade, and loose connection.

[0012] Step S103, within a preset time period T, collect the ordinary image and infrared image of the offshore wind turbine blade at a preset time interval t.

[0013] Step S104, preprocess the ordinary image and infrared image of the offshore wind turbine blade to obtain an image sequence.

[0014] The image sequence includes N sequence units, and each sequence unit consists of the preprocessed ordinary image and infrared image, where N = T / t.

[0015] Step S105, transfer the trained first neural network model to the second neural network model, and train the second neural network model with the image sequence.

[0016] The input of the second neural network model includes the input of the first neural network model, the image sequence, and environmental data. The output is the second health state of the offshore wind turbine blade. The second health state includes: the first health state, surface stains, surface cracks, surface corrosion, and internal cracks.

[0017] Step S106, input the environmental data, the vibration data of M points, and the image sequence into the trained second neural network model, and output the second health state of the offshore wind turbine blade.

[0018] Further, the number of points M, the number of time points K of the vibration data, the preset time period T, and the preset time interval t are all user-defined parameters.

[0019] Further, preprocessing the ordinary image and infrared image of the offshore wind turbine blade to obtain an image sequence includes the following steps:

[0020] Step S201, grayscale the ordinary image and scale it to the same preset size as the infrared image, where the preset size is a user-defined parameter.

[0021] Step S202, perform noise reduction processing on the ordinary image and infrared image through Gaussian filtering.

[0022] Step S203, remove the background of the ordinary image and infrared image through a semantic segmentation model.

[0023] Step S204, perform normalization processing on the ordinary image and infrared image through the Min-Max method.

[0024] Step S205, randomly select a data augmentation method to perform data augmentation on the ordinary image and the infrared image, where the data augmentation methods include: rotation, flipping, and translation.

[0025] Furthermore, the first neural network model consists of M transformation layers, M feature extraction layers, 1 sequence construction layer, and 1 time series analysis layer;

[0026] The M transformation layers are used to transform the vibration data at M points into Markov matrices;

[0027] The M feature extraction layers are used to extract features from the M Markov matrices to obtain feature vectors, and the M feature extraction layers share weight parameters;

[0028] The sequence construction layer is used to splice the M feature vectors to obtain a feature sequence, the feature sequence includes M sequence units, and each sequence unit corresponds to a feature vector;

[0029] The time series analysis layer is used to perform time series analysis on the feature sequence to obtain an update vector. The time series analysis layer is constructed based on the Transformer model, and the dimension number of the update vector is a custom parameter;

[0030] During the training process of the first neural network model, the update vector is input into the first classifier. The classification space of the first classifier represents the first health state of the offshore wind turbine blade, and the sample labels of the training samples used to train the first neural network model are obtained through manual annotation.

[0031] Furthermore, the transformation layer transforms the vibration data into a Markov matrix, including the following steps:

[0032] Step S301, perform normalization processing on the vibration data through the Min - Max method;

[0033] Control the value range of the acceleration values at K time points between 0 and 1;

[0034] Step S302, perform discretization processing on the normalized vibration data to obtain a state sequence;

[0035] The value range of the normalized acceleration values is evenly divided into Q non - overlapping state intervals. Each state interval corresponds to an increasing state value ranging from 1 to Q. Then, determine the state interval where the normalized acceleration values at K time points are located to obtain the state sequence, where Q is a custom positive integer;

[0036] Step S303, count the total number of occurrences of different state values in the state sequence, and traverse the transition times of adjacent state values in the state sequence to construct a Markov transition matrix;

[0037] The size of the Markov transition matrix is Q×Q, and the element value in the a-th row and b-th column represents the transition probability from the a-th state value to the b-th state value, where 1 ≤ a ≤ Q and 1 ≤ b ≤ Q;

[0038] The probability that the a-th state value transitions to the b-th state value is calculated as follows:

[0039] , where represents the number of times the a-th state value transitions to the b-th state value, represents the total number of times the a-th state value appears;

[0040] Step S304, construct a Markov matrix based on the state sequence and the Markov transition matrix;

[0041] The size of the Markov matrix is K×K, and the element value in the c-th row and d-th column represents the probability that the c-th state value of the state sequence transitions to the d-th state value, where 1 ≤ c ≤ K and 1 ≤ d ≤ K.

[0042] Furthermore, the calculation formula of the feature extraction layer is as follows:

[0043] ;

[0044] where Vector represents the feature vector output by the feature extraction layer, MTF represents the Markov matrix input to the feature extraction layer, , and represent convolution operations with a convolution kernel size of 3×3 and convolution kernel numbers of 32, 64, and 128 respectively, represents a convolution operation with a convolution kernel size of 1×1 and a convolution kernel number of 256, represents a max pooling operation with a pooling window size of 2×2, and represent dilated convolution operations with a convolution kernel size of 3×3 and a convolution kernel number of 64 and dilation rates of 2 and 4 respectively. GlobalMaxPooling represents a global max pooling operation, and Dense represents a fully connected layer;

[0045] Definition: The padding methods of the convolution operation and the dilated convolution operation are both SAME, and the stride of the convolution kernel is 1. The activation functions of the convolution operation and the dilated convolution operation are both ReLU activation functions; the stride of the pooling window of the max pooling operation is 2; the activation function of the fully connected layer is the Swish activation function.

[0046] Further, before training the first neural network model, it is pre-trained first. During the pre-training process of the first neural network model, the update vector is input into the second classifier, the third classifier, the fourth classifier, the fifth classifier, and the sixth classifier. The classification spaces of the above classifiers respectively represent the temperature, rainfall, snowfall, wind speed, the angle between the wind direction and the blade orientation.

[0047] Further, the second neural network model consists of the trained first neural network model, N first convolutional layers, N second convolutional layers, a concatenation layer, a hidden layer, an extraction layer, and a seventh classifier;

[0048] The first convolutional layer is used to extract features from the pre-processed ordinary image to obtain a first vector, and the number of dimensions of the first vector is a custom parameter;

[0049] The second convolutional layer is used to extract features from the pre-processed infrared image to obtain a second vector, and the number of dimensions of the second vector is a custom parameter;

[0050] The concatenation layer is used to concatenate the update vector output by the trained first neural network model with the environmental data, the first vector, and the second vector to obtain a combined vector, and convert each sequence unit of the image sequence into a combined vector representation;

[0051] The hidden layer includes N hidden units. The nth hidden unit inputs the combined vector corresponding to the nth sequence unit of the image sequence and outputs a hidden vector, where 1 ≤ n ≤ N;

[0052] The extraction layer is used to extract the hidden vector output by the Nth hidden unit and input it into the seventh classifier. The classification space of the seventh classifier represents the second health state of the offshore wind turbine blade. The sample labels of the training samples used to train the second neural network model are obtained through manual annotation.

[0053] Further, the calculation formula of the nth hidden unit includes:

[0054] ;

[0055] ;

[0056] where and respectively represent the hidden vectors output by the nth hidden unit and the (n - 1)th hidden unit, is assigned 0, represents the combined vector corresponding to the nth sequence unit of the image sequence input to the nth hidden unit, represents the gating vector of the nth hidden unit, and its size is the same as the size of the hidden vector, and respectively represent the first weight parameter and the second weight parameter corresponding to the nth hidden unit, and respectively represent the first bias parameter and the second bias parameter corresponding to the nth hidden unit, denotes element-wise multiplication, Swish denotes the Swish activation function, and sigmoid denotes the sigmoid activation function.

[0057] The present invention provides an offshore wind turbine blade health monitoring system, including:

[0058] A data acquisition module, which is used to acquire the environmental data at the location of the offshore wind turbine blade and acquire the vibration data at M points on the offshore wind turbine blade;

[0059] A first training module, which is used to train the first neural network model with the environmental data and the vibration data at M points;

[0060] An image acquisition module, which is used to acquire the ordinary image and the infrared image of the offshore wind turbine blade at a preset time interval t within a preset time period T;

[0061] An image processing module, which is used to preprocess the ordinary image and the infrared image of the offshore wind turbine blade to obtain an image sequence;

[0062] A second training module, which is used to transfer the trained first neural network model to the second neural network model and train the second neural network model with the image sequence;

[0063] A wind turbine blade health monitoring module, which is used to input the environmental data, the vibration data at M points and the image sequence into the trained second neural network model and output the second health state of the offshore wind turbine blade.

[0064] The beneficial effects of the present invention are as follows: The present invention extracts and fuses the features of the vibration data at multiple points through the first neural network model, transfers the first neural network model to the second neural network model to extract and fuse the features of the ordinary image and the infrared image, so as to realize the health monitoring of the offshore wind turbine blade, and uses the environmental data as the pre-training target, thereby reducing the interference of the environmental data on the health monitoring and improving the monitoring accuracy of the model. Description of the Drawings

[0065] Figure 1 is a flowchart of a method for monitoring the health of an offshore wind turbine blade according to the present invention;

[0066] Figure 2 is a flowchart of preprocessing to obtain an image sequence according to the present invention;

[0067] Figure 3It is a flowchart of the conversion layer of the present invention converting vibration data into a Markov matrix;

[0068] Figure 4 It is a schematic diagram of a health monitoring system for an offshore wind turbine blade of the present invention;

[0069] Figure 5 It is a schematic diagram of the first neural network model of the present invention;

[0070] Figure 6 It is a schematic diagram of the second neural network model of the present invention.

[0071] In the figure: data acquisition module 401, first training module 402, image acquisition module 403, image processing module 404, second training module 405, wind turbine blade health monitoring module 406. Detailed implementation manners

[0072] Now, the subject matter described herein will be discussed with reference to exemplary embodiments. It should be understood that discussing these embodiments is only to enable those skilled in the art to better understand and thus implement the subject matter described herein. Without departing from the scope of protection of the content of this specification, changes can be made to the functions and arrangements of the elements discussed. Each example can omit, substitute, or add various processes or components as needed. Additionally, the features described relative to some examples can also be combined in other examples.

[0073] It should be noted that unless otherwise defined, the technical terms or scientific terms used in one or more embodiments of the present invention should have the ordinary meaning understood by those of ordinary skill in the art to which the present invention pertains. The "first", "second", and similar terms used in one or more embodiments of the present invention do not denote any order, quantity, or importance, but are only used to distinguish different components. The terms such as "including" or "comprising" mean that the elements or objects appearing before this term cover the elements or objects listed after this term and their equivalents, without excluding other elements or objects. The terms such as "connected" or "coupled" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The terms such as "upper", "lower", "left", "right", etc. are only used to represent relative positional relationships, and when the absolute position of the object being described changes, the relative positional relationship may also change accordingly.

[0074] As Figures 1 to 6 shown, a health monitoring method for an offshore wind turbine blade includes the following steps:

[0075] Step S101, collect the environmental data at the location of the offshore wind turbine blade and collect the vibration data at M points on the offshore wind turbine blade;

[0076] The environmental data includes: temperature, rainfall, snowfall, wind speed, and the angle between the wind direction and the blade orientation;

[0077] The vibration data at M points are all represented by the acceleration values at K time points;

[0078] Step S102, train the first neural network model with the environmental data and the vibration data at M points;

[0079] The input of the first neural network model is the vibration data at M points, and the output is the first health state of the offshore wind turbine blade. The first health state includes: normal blade, ice accumulation on the blade, and loose connection;

[0080] Step S103, within a preset time period T, collect the ordinary images and infrared images of the offshore wind turbine blade at a preset time interval t;

[0081] Step S104, preprocess the ordinary images and infrared images of the offshore wind turbine blade to obtain an image sequence;

[0082] The image sequence includes N sequence units, and each sequence unit consists of the preprocessed ordinary image and infrared image, where N = T / t;

[0083] Step S105, transfer the trained first neural network model to the second neural network model, and train the second neural network model with the image sequence;

[0084] The input of the second neural network model includes the input of the first neural network model, as well as the image sequence and environmental data. The output is the second health state of the offshore wind turbine blade. The second health state includes: the first health state, surface stains, surface cracks, surface corrosion, and internal cracks;

[0085] Step S106, input the environmental data, the vibration data at M points, and the image sequence into the trained second neural network model, and output the second health state of the offshore wind turbine blade.

[0086] It should be noted that the first neural network model is mainly used to extract the features related to vibration data, environmental data, and mechanical failures, and fuse the features at multiple points to reflect the vibration characteristics of the entire offshore wind turbine blade; on the basis of the first neural network, the second neural network model increases the perception ability of image data. The ordinary images provide blade surface information and are mainly used to identify surface stains, surface cracks, and surface corrosion. The infrared images provide blade internal information and are mainly used to identify hidden faults inside the blade; the second neural network model can effectively reduce the training cost, improve the generalization ability and robustness of the model through transfer learning.

[0087] In an embodiment of the present invention, the number of points M, the number of time points K of the vibration data, the preset time period T, and the preset time interval t are all user-defined parameters. Preferably, the number of points M is set to 9, the number of time points K of the vibration data is set to 512, the preset time period T is set to 1 minute, and the preset time interval t is set to 5 seconds, then N is equal to 12.

[0088] It should be noted that a wind turbine generator includes 3 wind turbine blades, and vibration sensors can be installed at the root, middle, main shaft, or hub of each blade respectively, so there are a total of 9 points. Among them, the root of the blade is close to the connection between the blade and the hub, which belongs to the stress concentration area and is prone to problems such as fatigue damage and connection looseness. The middle of the blade can capture the overall bending and torsional vibrations of the blade, and the main shaft and hub are also high-incidence areas of mechanical failures.

[0089] In an embodiment of the present invention, as Figure 2 shown, preprocess the ordinary image and infrared image of the offshore wind turbine blade to obtain an image sequence, including the following steps:

[0090] Step S201, grayscale the ordinary image and scale it to the same preset size as the infrared image;

[0091] The preset size is a user-defined parameter. Preferably, the preset size is set to 512×512;

[0092] Step S202, perform noise reduction processing on the ordinary image and infrared image through Gaussian filtering;

[0093] Step S203, remove the background of the ordinary image and infrared image through a semantic segmentation model;

[0094] The semantic segmentation model can be U-Net, DeepLab series, Mask R-CNN, etc., which will not be elaborated here;

[0095] Step S204, perform normalization processing on the ordinary image and infrared image through the Min-Max method;

[0096] Step S205, randomly select a data augmentation method to perform data augmentation on the ordinary image and infrared image;

[0097] The data augmentation methods include: rotation, flipping, and translation.

[0098] In an embodiment of the present invention, the first neural network model is composed of M conversion layers, M feature extraction layers, 1 sequence construction layer, and 1 time series analysis layer;

[0099] The M conversion layers are used to convert the vibration data of the M points into a Markov matrix;

[0100] M feature extraction layers are used to extract features from M Markov matrices to obtain feature vectors, and the M feature extraction layers share weight parameters;

[0101] The sequence construction layer is used to splice the M feature vectors to obtain a feature sequence, and the feature sequence includes M sequence units, and each sequence unit corresponds to a feature vector;

[0102] The time series analysis layer is used to perform time series analysis on the feature sequence to obtain an update vector.

[0103] In an embodiment of the present invention, during the training process of the first neural network model, the update vector is input into the first classifier, and the classification space of the first classifier represents the first health state of the offshore wind turbine blade. The sample labels of the training samples used to train the first neural network model are obtained through manual annotation.

[0104] In an embodiment of the present invention, as Figure 3 shown, the conversion layer converts the vibration data into a Markov matrix, including the following steps:

[0105] Step S301, normalize the vibration data by the Min-Max method;

[0106] Control the value range of the acceleration values at K time points between 0 and 1;

[0107] Step S302, discretize the normalized vibration data to obtain a state sequence;

[0108] The value range of the normalized acceleration values is evenly divided into Q non-overlapping state intervals, and each state interval corresponds to an increasing state value ranging from 1 to Q. Then, determine the state interval where the normalized acceleration values at K time points are located to obtain a state sequence, where Q is a user-defined positive integer. Preferably, Q is set to 10;

[0109] Step S303, count the total number of occurrences of different state values in the state sequence, and traverse the transition times of adjacent state values in the state sequence to construct a Markov transition matrix;

[0110] The size of the Markov transition matrix is Q×Q, and the element value in the a-th row and b-th column represents the transition probability from the a-th state value to the b-th state value, where 1≤a≤Q, 1≤b≤Q;

[0111] The probability that the a-th state value transitions to the b-th state value is calculated as follows:

[0112] ;

[0113] where represents the number of times the a-th state value transfers to the b-th state value, represents the total number of times the a-th state value appears;

[0114] Step S304, construct a Markov matrix according to the state sequence and the Markov transition matrix;

[0115] The size of the Markov matrix is K×K, and the element value in the c-th row and d-th column represents the probability that the c-th state value of the state sequence transfers to the d-th state value, where 1 ≤ c ≤ K and 1 ≤ d ≤ K.

[0116] For example, assume that the normalized vibration data is represented as: {0.1, 0.3, 0.4, 0.7, 0.9}, Q is set to 3, and the value range [0, 1] is divided into {[0, 0.33), [0.33, 0.66), [0.66, 1]}, and the corresponding state values are 1, 2, 3 respectively. Then the state sequence is represented as: {1, 1, 2, 3, 3}. The total number of times the state value 1 appears is counted as 2, the total number of times the state value 2 appears is counted as 1, and the total number of times the state value 3 appears is counted as 2. Then traverse the transition probability of adjacent state values in the state sequence. The number of times the state value 1 transfers to the state value 1 is 1, the number of times the state value 1 transfers to the state value 2 is 1, the number of times the state value 2 transfers to the state value 3 is 1, and the number of times the state value 3 transfers to the state value 3 is 1. Then the Markov transition matrix is represented as: , then the element value in the 1st row and 1st column of the Markov matrix represents the probability that the state value 1 transfers to the state value 1, which is 0.5. The element value in the 1st row and 2nd column of the Markov matrix represents the probability that the state value 1 transfers to the state value 2, which is also 0.5. And so on. The finally constructed Markov matrix is represented as follows:

[0117] .

[0118] It should be noted that converting the vibration data into a Markov matrix fully preserves the discretized time-domain information, can more comprehensively reflect the dynamic mode of blade vibration, enhance the model's understanding of the blade vibration mode. Secondly, it is convenient for the subsequent feature extraction layer to extract features and improve the accuracy of fault analysis.

[0119] In an embodiment of the present invention, the calculation formula of the feature extraction layer is as follows:

[0120] ;

[0121] where Vector represents the feature vector output by the feature extraction layer, and MTF represents the Markov matrix input to the feature extraction layer, , and Denote the convolution operations where the sizes of the convolution kernels are all 3×3 and the numbers of convolution kernels are 32, 64, and 128 respectively. Denote the convolution operation where the size of the convolution kernel is 1×1 and the number of convolution kernels is 256. Denote the max pooling operation where the size of the pooling window is 2×2. and Denote the dilated convolution operations where the size of the convolution kernel is 3×3, the number of convolution kernels is 64, and the dilation rates are 2 and 4 respectively. GlobalMaxPooling denotes the global max pooling operation, and Dense denotes the fully connected layer.

[0122] Definition: The padding methods of the convolution operation and the dilated convolution operation are both SAME, and the strides of the convolution kernels are both 1. The activation functions of the convolution operation and the dilated convolution operation are both ReLU activation functions. The strides of the pooling windows of the max pooling operation are both 2. The activation function of the fully connected layer is the Swish activation function.

[0123] It should be noted that, for example, if the size of the Markov matrix is 512×512, then after a feature map of size 512×512×32 is obtained, where 32 represents the number of channels of the feature map. After a feature map of size 256×256×32 is obtained, and so on. After a feature map of size 64×64×256 is obtained, and then through the global max pooling operation (selecting the maximum value on the entire feature map of each channel), a feature map of size 64×64×1 (unrolled into a vector of size 1×4096) is obtained. The fully connected layer is used to reduce the feature dimension, and at the same time, through non-linear transformation, the feature expression ability is enhanced and the computational complexity is reduced. For example, the vector of size 1×4096 is reduced to 512, which will not be elaborated here.

[0124] In an embodiment of the present invention, the time series analysis layer is constructed based on the Transformer model, and can also be constructed based on the LSTM model or the GRU model. And the number of dimensions of the updated vector obtained by performing time series analysis on the feature sequence is a custom parameter. Preferably, the number of dimensions of the updated vector is set to 64, which will not be elaborated here.

[0125] In an embodiment of the present invention, before training the first neural network model, it is pre-trained first. During the pre-training process of the first neural network model, the updated vector is input into the second classifier, the third classifier, the fourth classifier, the fifth classifier, and the sixth classifier. The classification spaces of the above classifiers respectively represent the temperature, rainfall, snowfall, wind speed, the angle between the wind direction and the blade orientation.

[0126] It should be noted that pre-training the first neural network model can effectively reduce the number of subsequent training samples, speed up the training speed of the model, and pre-training extracts the features related to environmental data in the vibration data, which can reduce the interference of environmental data on the fault monitoring of offshore wind turbine blades to a certain extent, thereby improving the monitoring accuracy of the model.

[0127] In an embodiment of the present invention, the second neural network model is composed of a trained first neural network model, N first convolutional layers, N second convolutional layers, a splicing layer, a hidden layer, an extraction layer, and a seventh classifier;

[0128] The first convolutional layer is used to extract features from the preprocessed ordinary image to obtain a first vector, and the number of dimensions of the first vector is a custom parameter. Preferably, the number of dimensions of the first vector is set to 64;

[0129] The second convolutional layer is used to extract features from the preprocessed infrared image to obtain a second vector, and the number of dimensions of the second vector is a custom parameter. Preferably, the number of dimensions of the second vector is set to 64;

[0130] The splicing layer is used to splice the updated vector output by the trained first neural network model with environmental data, the first vector, and the second vector to obtain a combined vector, and convert each sequence unit of the image sequence into a combined vector representation, that is, the number of dimensions of the combined vector is 64 (updated vector) + 5 (environmental data) + 64 (first vector) + 64 (second vector) = 197;

[0131] The hidden layer includes N hidden units. The nth hidden unit inputs the combined vector corresponding to the nth sequence unit of the image sequence and outputs a hidden vector, where 1 ≤ n ≤ N;

[0132] The extraction layer is used to extract the hidden vector output by the Nth hidden unit and input it into the seventh classifier. The classification space of the seventh classifier represents the second health state of the offshore wind turbine blade, and the sample labels of the training samples used to train the second neural network model are obtained through manual annotation.

[0133] In an embodiment of the present invention, the calculation formula of the nth hidden unit includes:

[0134] ;

[0135] ;

[0136] where and respectively represent the hidden vectors output by the nth hidden unit and the (n - 1)th hidden unit, is assigned 0, denotes the combined vector corresponding to the n-th sequence unit of the image sequence input to the n-th hidden unit, denotes the gating vector of the n-th hidden unit, whose size is the same as that of the hidden vector, and denote the first weight parameter and the second weight parameter corresponding to the n-th hidden unit respectively, and denote the first bias parameter and the second bias parameter corresponding to the n-th hidden unit respectively, denotes element-wise multiplication, Swish denotes the Swish activation function, and sigmoid denotes the sigmoid activation function.

[0137] It should be noted that the weight parameters and bias parameters in the hidden unit are all learnable hyperparameters. For example, the size of the hidden vector can be designed as 1×R, then the size of the gating vector is also 1×R, the second weight parameter can be designed as a matrix of size R×R, the size of the combined vector is 1×197, and the first weight parameter can be designed as a matrix of size 197×R, where R is a user-defined parameter. Preferably, R is set to 64.

[0138] In an embodiment of the present invention, as Figure 4 shown, an offshore wind turbine blade health monitoring system includes:

[0139] A data acquisition module 401, which is used to acquire the environmental data at the location of the offshore wind turbine blade and acquire the vibration data at M points on the offshore wind turbine blade;

[0140] A first training module 402, which is used to train the first neural network model with the environmental data and the vibration data at M points;

[0141] An image acquisition module 403, which is used to acquire the normal image and the infrared image of the offshore wind turbine blade at a preset time interval t within a preset time period T;

[0142] An image processing module 404, which is used to preprocess the normal image and the infrared image of the offshore wind turbine blade to obtain an image sequence;

[0143] A second training module 405, which is used to transfer the trained first neural network model to the second neural network model and train the second neural network model with the image sequence;

[0144] A wind turbine blade health monitoring module 406, which is used to input the environmental data, the vibration data at M points and the image sequence into the trained second neural network model and output the second health state of the offshore wind turbine blade.

[0145] The above has described the embodiments of this example, but this example is not limited to the above specific implementation manners. The above specific implementation manners are merely illustrative rather than restrictive. Under the inspiration of this example, those of ordinary skill in the art can also make many forms, all of which fall within the protection scope of this example.

Claims

1. A method for monitoring the health of offshore wind turbine blades, characterized in that: The following steps are involved: Step S101, collecting environmental data of the offshore wind turbine blades, and collecting vibration data of M points on the offshore wind turbine blades; Environmental data include: temperature, rainfall, snowfall, wind speed, and the angle between wind direction and blade orientation; The vibration data of M points are represented by the acceleration values ​​at K time points; Step S102, training a first neural network model using environmental data and vibration data of M points; The input of the first neural network model is the vibration data of M points, and the output is the first health state of the offshore wind turbine blade, which includes: normal blade, ice accumulation on the blade and loose connection; Step S103, collecting ordinary images and infrared images of offshore wind turbine blades at preset time intervals t within a preset time period T; Step S104, preprocessing the common image and infrared image of the offshore wind turbine blade to obtain an image sequence; The image sequence includes N sequence units, each of which consists of a preprocessed ordinary image and an infrared image, where N = T / t; Step S105, migrating the trained first neural network model to a second neural network model, and training the second neural network model through the image sequence; The input of the second neural network model includes the input of the first neural network model, and also includes an image sequence and environmental data, and the output is a second health state of the offshore wind turbine blade, the second health state includes: the first health state, surface stains, surface cracks, surface corrosion and internal cracks; Step S106, inputting the environmental data, the vibration data of the M points and the image sequence into the trained second neural network model, and outputting the second health state of the offshore wind turbine blade.

2. The offshore wind turbine blade health monitoring method according to claim 1, characterized in that: The number of points M, the number of time points K of vibration data, the preset time period T and the preset time interval t are all custom parameters.

3. The offshore wind turbine blade health monitoring method according to claim 1, characterized in that: Preprocessing the ordinary image and infrared image of the offshore wind turbine blade to obtain an image sequence includes the following steps: Step S201, graying the ordinary image, and scaling it and the infrared image to a unified preset size, wherein the preset size is a custom parameter; Step S202, performing noise reduction processing on the ordinary image and the infrared image by Gaussian filtering; Step S203, removing the background of the ordinary image and the infrared image by using a semantic segmentation model; Step S204, normalizing the ordinary image and the infrared image by using the Min-Max method; Step S205: randomly select a data enhancement method to perform data enhancement on the ordinary image and the infrared image, wherein the data enhancement method includes: rotation, flipping and translation.

4. The offshore wind turbine blade health monitoring method according to claim 1, characterized in that: The first neural network model consists of M conversion layers, M feature extraction layers, 1 sequence construction layer and 1 time series analysis layer; M conversion layers are used to convert the vibration data of M points into Markov matrices; The M feature extraction layers are used to extract features from the M Markov matrices to obtain feature vectors, and the M feature extraction layers share weight parameters; The sequence construction layer is used to concatenate M feature vectors to obtain a feature sequence. The feature sequence includes M sequence units, and each sequence unit corresponds to a feature vector. The timing analysis layer is used to perform timing analysis on feature sequences to obtain update vectors. The timing analysis layer is built based on the Transformer model, and the number of dimensions of the update vector is a custom parameter. During the training process of the first neural network model, the update vector is input into the first classifier, the classification space of the first classifier represents the first health state of the offshore wind turbine blade, and the sample labels of the training samples used to train the first neural network model are obtained through manual labeling.

5. The offshore wind turbine blade health monitoring method according to claim 4, characterized in that: The conversion layer converts the vibration data into a Markov matrix, which includes the following steps: Step S301, normalizing the vibration data using the Min-Max method; Control the range of the acceleration values ​​at K time points between 0 and 1; Step S302, discretizing the normalized vibration data to obtain a state sequence; The range of normalized acceleration values ​​is evenly divided into Q non-overlapping state intervals, each of which corresponds to an increasing state value ranging from 1 to Q. Then the state intervals of the normalized acceleration values ​​at K time points are determined to obtain a state sequence, where Q is a user-defined positive integer. Step S303, counting the total number of occurrences of different state values ​​in the state sequence, and traversing the number of transitions of adjacent state values ​​in the state sequence to construct a Markov transition matrix; The size of the Markov transition matrix is ​​Q×Q, and the element value in the ath row and bth column represents the transition probability from the ath state value to the bth state value, where 1≤a≤Q, 1≤b≤Q; The probability that the ath state value transfers to the bth state value The calculation formula is as follows: ,in Indicates the number of times the a-th state value is transferred to the b-th state value, Indicates the total number of times the a-th state value appears; Step S304, constructing a Markov matrix according to the state sequence and the Markov transition matrix; The size of the Markov matrix is ​​K×K, and the element value in the cth row and dth column represents the probability that the cth state value of the state sequence is transferred to the dth state value, where 1≤c≤K, 1≤d≤K.

6. The offshore wind turbine blade health monitoring method according to claim 4, characterized in that: The calculation formula of the feature extraction layer is as follows: ； Where Vector represents the feature vector output by the feature extraction layer, and MTF represents the Markov matrix input by the feature extraction layer. , and represents convolution operations where the size of the convolution kernel is 3×3 and the number of convolution kernels is 32, 64, and 128 respectively. represents a convolution operation with a kernel size of 1×1 and a number of 256 kernels. Indicates the maximum pooling operation with a pooling window size of 2×2, and It indicates that the size of the convolution kernel is 3×3 and the number of convolution kernels is 64, and the dilation rates are 2 and 4 respectively. GlobalMaxPooling indicates the global maximum pooling operation, and Dense indicates the fully connected layer; Definition: The filling method of convolution operation and dilated convolution operation is SAME, and the step size of convolution kernel is 1. The activation function of convolution operation and dilated convolution operation is ReLU activation function. The step size of pooling window of maximum pooling operation is 2. The activation function of fully connected layer is Swish activation function.

7. The offshore wind turbine blade health monitoring method according to claim 4, characterized in that: Before training the first neural network model, it is pre-trained first. During the pre-training process of the first neural network model, the update vector is input into the second classifier, the third classifier, the fourth classifier, the fifth classifier and the sixth classifier. The classification spaces of the above classifiers respectively represent temperature, rainfall, snowfall, wind speed, and the angle between wind direction and blade direction.

8. The offshore wind turbine blade health monitoring method according to claim 4, characterized in that: The second neural network model is composed of the trained first neural network model, N first convolutional layers, N second convolutional layers, a concatenation layer, a hidden layer, an extraction layer and a seventh classifier; The first convolutional layer is used to extract features from the preprocessed common image to obtain a first vector, where the number of dimensions of the first vector is a custom parameter; The second convolutional layer is used to extract features from the preprocessed infrared image to obtain a second vector, where the number of dimensions of the second vector is a custom parameter; The concatenation layer is used to concatenate the update vector output by the trained first neural network model with the environmental data, the first vector and the second vector to obtain a combined vector, and convert each sequence unit of the image sequence into a combined vector representation; The hidden layer includes N hidden units, the nth hidden unit inputs the combination vector corresponding to the nth sequence unit of the image sequence, and outputs a hidden vector, where 1≤n≤N; The extraction layer is used to extract the hidden vector output by the Nth hidden unit and input it into the seventh classifier. The classification space of the seventh classifier represents the second health state of the offshore wind turbine blade. The sample labels of the training samples used to train the second neural network model are obtained through manual annotation.

9. The offshore wind turbine blade health monitoring method according to claim 8, characterized in that: The calculation formula for the nth hidden unit includes: ； ； in and Respectively represent the hidden vectors output by the nth hidden unit and the n-1th hidden unit, Assign a value of 0, Represents the combination vector corresponding to the nth sequence unit of the image sequence input by the nth hidden unit, represents the gating vector of the nth hidden unit, which has the same size as the hidden vector, and They represent the first weight parameter and the second weight parameter corresponding to the nth hidden unit respectively, and They represent the first bias parameter and the second bias parameter corresponding to the nth hidden unit, respectively. represents element-by-element multiplication, Swish represents the Swish activation function, and sigmoid represents the sigmoid activation function.

10. An offshore wind turbine blade health monitoring system, executing an offshore wind turbine blade health monitoring method as claimed in any one of claims 1 to 9, characterized in that: include: A data acquisition module, which is used to collect environmental data of the offshore wind turbine blades and collect vibration data of M points on the offshore wind turbine blades; A first training module, which is used to train a first neural network model through environmental data and vibration data of M points; An image acquisition module, which is used to acquire ordinary images and infrared images of offshore wind turbine blades within a preset time period T and at a preset time interval t; An image processing module, which is used to pre-process ordinary images and infrared images of offshore wind turbine blades to obtain an image sequence; A second training module, which is used to transfer the trained first neural network model to a second neural network model, and train the second neural network model through an image sequence; The wind turbine blade health monitoring module is used to input environmental data, vibration data of M points and image sequences into the trained second neural network model, and output the second health status of the offshore wind turbine blades.

Images