Offshore wind power blade health monitoring method and system

Abstract

The invention relates to the technical field of offshore wind power blade health monitoring, and discloses an offshore wind power blade health monitoring method and system, and the method comprises the following steps: S101, collecting environment data and vibration data; step S102, training a first neural network model; s103, acquiring a common image and an infrared image; step S104, constructing an image sequence; step S105, training a second neural network model; and step S106, completing health monitoring of the offshore wind power blade through a second neural network model. According to the invention, feature extraction is carried out on the multi-point vibration data through the first neural network model, the first neural network model is migrated to the second neural network model to carry out feature extraction on the common image and the infrared image, and the environment data is used as a pre-training target, so that the interference of the environment data on health monitoring is reduced, and the accuracy of health monitoring is improved. The monitoring precision is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of offshore wind turbine blade health monitoring, and more specifically, to an offshore wind turbine blade health monitoring method and system. Background Art

[0002] With the continuous growth of global energy demand and the improvement of environmental protection awareness, offshore wind power generation, as a clean and renewable energy form, is receiving widespread attention. Compared with land-based wind farms, offshore wind power generation has the advantages of higher annual average wind speed and more stable wind speed. However, offshore wind power generation is exposed to high humidity, strong sea breeze, salt spray corrosion and extreme temperature changes for a long time, which accelerates the aging and damage of wind turbine blades. As a key structural component in wind turbines, the operating status of wind turbine blades directly affects the power generation efficiency and equipment safety. Therefore, in order to ensure the safe operation of wind turbines, the health monitoring of offshore wind turbine blades is particularly important.

[0003] Currently, vibration sensors are used to collect vibration signals on the blade surface to analyze whether the blade is faulty. However, the vibration caused by strong winds or surges is similar to the vibration signal of blade failure. Ice or snow accumulation can also affect the vibration of the blade, which can easily lead to misjudgment. In addition, high-definition cameras are used to collect image data on the blade surface to analyze whether the blade is faulty. However, image data can only capture defect information on the blade surface, and it is difficult to detect hidden faults inside the blade (such as internal cracks, loose connections, etc.).

[0004] Therefore, there is an urgent need for an offshore wind turbine blade health monitoring method to solve the above problems. Summary of the invention

[0005] The present invention provides a method and system for monitoring the health of offshore wind turbine blades, which solve the technical problems in the above-mentioned background technology.

[0006] The present invention provides a method for monitoring the health of offshore wind turbine blades, comprising the following steps: Step S101, collecting environmental data of the offshore wind turbine blades, and collecting vibration data of M points on the offshore wind turbine blades; Environmental data include: temperature, rainfall, snowfall, wind speed, and the angle between wind direction and blade orientation; The vibration data of M points are represented by the acceleration values ​​at K time points; Step S102, training a first neural network model using environmental data and vibration data of M points; The input of the first neural network model is the vibration data of M points, and the output is the first health state of the offshore wind turbine blade, which includes: normal blade, ice accumulation on the blade and loose connection; Step S103, collecting ordinary images and infrared images of offshore wind turbine blades at preset time intervals t within a preset time period T; Step S104, preprocessing the common image and infrared image of the offshore wind turbine blade to obtain an image sequence; The image sequence includes N sequence units, each of which consists of a preprocessed ordinary image and an infrared image, where N = T / t; Step S105, migrating the trained first neural network model to a second neural network model, and training the second neural network model through the image sequence; The input of the second neural network model includes the input of the first neural network model, and also includes an image sequence and environmental data, and the output is a second health state of the offshore wind turbine blade, the second health state includes: the first health state, surface stains, surface cracks, surface corrosion and internal cracks; Step S106, inputting the environmental data, the vibration data of the M points and the image sequence into the trained second neural network model, and outputting the second health state of the offshore wind turbine blade.

[0007] Furthermore, the number of points M, the number of time points K of the vibration data, the preset time period T and the preset time interval t are all custom parameters.

[0008] Furthermore, preprocessing the ordinary image and the infrared image of the offshore wind turbine blade to obtain an image sequence includes the following steps: Step S201, graying the ordinary image, and scaling it and the infrared image to a unified preset size, wherein the preset size is a custom parameter; Step S202, performing noise reduction processing on the ordinary image and the infrared image by Gaussian filtering; Step S203, removing the background of the ordinary image and the infrared image by using a semantic segmentation model; Step S204, normalizing the ordinary image and the infrared image by using the Min-Max method; Step S205: randomly select a data enhancement method to perform data enhancement on the ordinary image and the infrared image, wherein the data enhancement method includes: rotation, flipping and translation.

[0009] Furthermore, the first neural network model consists of M conversion layers, M feature extraction layers, 1 sequence construction layer and 1 time series analysis layer; M conversion layers are used to convert the vibration data of M points into Markov matrices; The M feature extraction layers are used to extract features from the M Markov matrices to obtain feature vectors, and the M feature extraction layers share weight parameters; The sequence construction layer is used to concatenate M feature vectors to obtain a feature sequence. The feature sequence includes M sequence units, and each sequence unit corresponds to a feature vector. The timing analysis layer is used to perform timing analysis on feature sequences to obtain update vectors. The timing analysis layer is built based on the Transformer model, and the number of dimensions of the update vector is a custom parameter. During the training process of the first neural network model, the update vector is input into the first classifier, the classification space of the first classifier represents the first health state of the offshore wind turbine blade, and the sample labels of the training samples used to train the first neural network model are obtained through manual labeling.

[0010] Furthermore, the conversion layer converts the vibration data into a Markov matrix, including the following steps: Step S301, normalizing the vibration data using the Min-Max method; Control the range of the acceleration values ​​at K time points between 0 and 1; Step S302, discretizing the normalized vibration data to obtain a state sequence; The range of normalized acceleration values ​​is evenly divided into Q non-overlapping state intervals, each of which corresponds to an increasing state value ranging from 1 to Q. Then the state intervals of the normalized acceleration values ​​at K time points are determined to obtain a state sequence, where Q is a user-defined positive integer. Step S303, counting the total number of occurrences of different state values ​​in the state sequence, and traversing the number of transitions of adjacent state values ​​in the state sequence to construct a Markov transition matrix; The size of the Markov transition matrix is ​​Q×Q, and the element value in the ath row and bth column represents the transition probability from the ath state value to the bth state value, where 1≤a≤Q, 1≤b≤Q; The probability that the ath state value transfers to the bth state value The calculation formula is as follows: ,in Indicates the number of times the a-th state value is transferred to the b-th state value, Indicates the total number of times the a-th state value appears; Step S304, constructing a Markov matrix according to the state sequence and the Markov transition matrix; The size of the Markov matrix is ​​K×K, and the element value in the cth row and dth column represents the probability that the cth state value of the state sequence is transferred to the dth state value, where 1≤c≤K, 1≤d≤K.

[0011] Furthermore, the calculation formula of the feature extraction layer is as follows: ; Where Vector represents the feature vector output by the feature extraction layer, and MTF represents the Markov matrix input by the feature extraction layer. , and represents convolution operations where the size of the convolution kernel is 3×3 and the number of convolution kernels is 32, 64, and 128 respectively. represents a convolution operation with a kernel size of 1×1 and a number of 256 kernels. Indicates the maximum pooling operation with a pooling window size of 2×2, and It indicates that the size of the convolution kernel is 3×3 and the number of convolution kernels is 64, and the dilation rates are 2 and 4 respectively. GlobalMaxPooling indicates the global maximum pooling operation, and Dense indicates the fully connected layer; Definition: The filling method of convolution operation and dilated convolution operation is SAME, and the step size of convolution kernel is 1. The activation function of convolution operation and dilated convolution operation is ReLU activation function. The step size of pooling window of maximum pooling operation is 2. The activation function of fully connected layer is Swish activation function.

[0012] Furthermore, before training the first neural network model, it is pre-trained. During the pre-training process of the first neural network model, the update vector is input into the second classifier, the third classifier, the fourth classifier, the fifth classifier and the sixth classifier. The classification spaces of the above classifiers respectively represent temperature, rainfall, snowfall, wind speed, and the angle between wind direction and blade direction.

[0013] Furthermore, the second neural network model is composed of the trained first neural network model, N first convolutional layers, N second convolutional layers, a concatenation layer, a hidden layer, an extraction layer and a seventh classifier; The first convolutional layer is used to extract features from the preprocessed common image to obtain a first vector, where the number of dimensions of the first vector is a custom parameter; The second convolutional layer is used to extract features from the preprocessed infrared image to obtain a second vector, where the number of dimensions of the second vector is a custom parameter; The concatenation layer is used to concatenate the update vector output by the trained first neural network model with the environmental data, the first vector and the second vector to obtain a combined vector, and convert each sequence unit of the image sequence into a combined vector representation; The hidden layer includes N hidden units, the nth hidden unit inputs the combination vector corresponding to the nth sequence unit of the image sequence, and outputs a hidden vector, where 1≤n≤N; The extraction layer is used to extract the hidden vector output by the Nth hidden unit and input it into the seventh classifier. The classification space of the seventh classifier represents the second health state of the offshore wind turbine blade. The sample labels of the training samples used to train the second neural network model are obtained through manual annotation.

[0014] Furthermore, the calculation formula for the nth hidden unit includes: ; ; in and Respectively represent the hidden vectors output by the nth hidden unit and the n-1th hidden unit, Assign a value of 0, Represents the combination vector corresponding to the nth sequence unit of the image sequence input by the nth hidden unit, represents the gating vector of the nth hidden unit, which has the same size as the hidden vector, and They represent the first weight parameter and the second weight parameter corresponding to the nth hidden unit respectively, and They represent the first bias parameter and the second bias parameter corresponding to the nth hidden unit, respectively. represents element-by-element multiplication, Swish represents the Swish activation function, and sigmoid represents the sigmoid activation function.

[0015] The present invention provides an offshore wind turbine blade health monitoring system, comprising: A data acquisition module, which is used to collect environmental data of the offshore wind turbine blades and collect vibration data of M points on the offshore wind turbine blades; A first training module, which is used to train a first neural network model through environmental data and vibration data of M points; An image acquisition module, which is used to acquire ordinary images and infrared images of offshore wind turbine blades within a preset time period T and at a preset time interval t; An image processing module, which is used to pre-process ordinary images and infrared images of offshore wind turbine blades to obtain an image sequence; A second training module, which is used to transfer the trained first neural network model to a second neural network model, and train the second neural network model through an image sequence; The wind turbine blade health monitoring module is used to input environmental data, vibration data of M points and image sequences into the trained second neural network model, and output the second health status of the offshore wind turbine blades.

[0016] The beneficial effects of the present invention are as follows: the present invention uses a first neural network model to perform feature extraction and fusion on vibration data of multiple points, and migrates the first neural network model to a second neural network model to perform feature extraction and fusion on ordinary images and infrared images, thereby realizing health monitoring of offshore wind turbine blades, and taking environmental data as a pre-training target, thereby reducing the interference of environmental data on health monitoring and improving the monitoring accuracy of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a flow chart of a method for monitoring the health of offshore wind turbine blades according to the present invention; Figure 2 is a flow chart of obtaining an image sequence through preprocessing of the present invention; Figure 3 is a flow chart of the conversion layer of the present invention converting vibration data into a Markov matrix; Figure 4 is a schematic diagram of an offshore wind turbine blade health monitoring system of the present invention; Figure 5 is a schematic diagram of a first neural network model of the present invention; Figure 6 Schematic diagram of the second neural network model of the present invention.

[0018] In the figure: a data acquisition module 401 , a first training module 402 , an image acquisition module 403 , an image processing module 404 , a second training module 405 , and a wind turbine blade health monitoring module 406 . DETAILED DESCRIPTION

[0019] The subject matter described herein will now be discussed with reference to example embodiments. It should be understood that the discussion of these embodiments is only to enable those skilled in the art to better understand and implement the subject matter described herein, and the functions and arrangements of the elements discussed may be changed without departing from the scope of protection of the contents of this specification. Each example may omit, replace or add various processes or components as needed. In addition, the features described relative to some examples may also be combined in other examples.

[0020] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in one or more embodiments of the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The words "first", "second" and similar words used in one or more embodiments of the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0021] like Figure 1 to Figure 6 As shown, a method for monitoring the health of offshore wind turbine blades comprises the following steps: Step S101, collecting environmental data of the offshore wind turbine blades, and collecting vibration data of M points on the offshore wind turbine blades; Environmental data include: temperature, rainfall, snowfall, wind speed, and the angle between wind direction and blade orientation; The vibration data of M points are represented by the acceleration values ​​at K time points; Step S102, training a first neural network model using environmental data and vibration data of M points; The input of the first neural network model is the vibration data of M points, and the output is the first health state of the offshore wind turbine blade, which includes: normal blade, ice accumulation on the blade and loose connection; Step S103, collecting ordinary images and infrared images of offshore wind turbine blades at preset time intervals t within a preset time period T; Step S104, preprocessing the common image and infrared image of the offshore wind turbine blade to obtain an image sequence; The image sequence includes N sequence units, each of which consists of a preprocessed ordinary image and an infrared image, where N = T / t; Step S105, migrating the trained first neural network model to a second neural network model, and training the second neural network model through the image sequence; The input of the second neural network model includes the input of the first neural network model, and also includes an image sequence and environmental data, and the output is a second health state of the offshore wind turbine blade, the second health state includes: the first health state, surface stains, surface cracks, surface corrosion and internal cracks; Step S106, inputting the environmental data, the vibration data of the M points and the image sequence into the trained second neural network model, and outputting the second health state of the offshore wind turbine blade.

[0022] It should be noted that the first neural network model is mainly used to extract features related to vibration data, environmental data and mechanical failures, and to fuse features of multiple points to reflect the vibration characteristics of the entire offshore wind turbine blade; the second neural network model is based on the first neural network and has added the ability to perceive image data. Ordinary images provide blade surface information, which is mainly used to identify surface stains, surface cracks and surface corrosion. Infrared images provide internal blade information, which is mainly used to identify hidden faults inside the blade; the second neural network model can effectively reduce training costs through transfer learning and improve the generalization and robustness of the model.

[0023] In one embodiment of the present invention, the number of points M, the number of time points K of vibration data, the preset time period T and the preset time interval t are all custom parameters. Preferably, the number of points M is set to 9, the number of time points K of vibration data is set to 512, the preset time period T is set to 1 minute, and the preset time interval t is set to 5 seconds, then N is equal to 12.

[0024] It should be noted that a wind turbine generator set includes three wind turbine blades. Vibration sensors can be installed at the root, middle, main shaft or hub of each blade, for a total of nine points. The root of the blade is close to the connection between the blade and the hub, which is a stress concentration area and is prone to fatigue damage and loose connection problems. The middle of the blade can capture the bending and torsional vibrations of the entire blade. The main shaft and hub are also high-incidence areas of mechanical failure.

[0025] In one embodiment of the present invention, Figure 2 As shown, the ordinary image and infrared image of the offshore wind turbine blade are preprocessed to obtain an image sequence, including the following steps: Step S201, graying the ordinary image, and scaling it and the infrared image to a unified preset size; The preset size is a custom parameter. Preferably, the preset size is set to 512×512; Step S202, performing noise reduction processing on the ordinary image and the infrared image by Gaussian filtering; Step S203, removing the background of the ordinary image and the infrared image by using a semantic segmentation model; The semantic segmentation model can be U-Net, DeepLab series, Mask R-CNN, etc., which will not be described here; Step S204, normalizing the ordinary image and the infrared image by using the Min-Max method; Step S205, randomly selecting a data enhancement method to perform data enhancement on the ordinary image and the infrared image; Data augmentation methods include: rotation, flipping and translation.

[0026] In one embodiment of the present invention, the first neural network model is composed of M conversion layers, M feature extraction layers, 1 sequence construction layer and 1 time series analysis layer; M conversion layers are used to convert the vibration data of M points into Markov matrices; The M feature extraction layers are used to extract features from the M Markov matrices to obtain feature vectors, and the M feature extraction layers share weight parameters; The sequence construction layer is used to concatenate M feature vectors to obtain a feature sequence. The feature sequence includes M sequence units, and each sequence unit corresponds to a feature vector. The timing analysis layer is used to perform timing analysis on the feature sequence to obtain the update vector.

[0027] In one embodiment of the present invention, during the training process of the first neural network model, the update vector is input into the first classifier, the classification space of the first classifier represents the first health state of the offshore wind turbine blade, and the sample labels of the training samples for training the first neural network model are obtained by manual labeling.

[0028] In one embodiment of the present invention, Figure 3 As shown, the conversion layer converts the vibration data into a Markov matrix, which includes the following steps: Step S301, normalizing the vibration data using the Min-Max method; Control the range of the acceleration values ​​at K time points between 0 and 1; Step S302, discretizing the normalized vibration data to obtain a state sequence; The value range of the normalized acceleration value is evenly divided into Q non-overlapping state intervals, each state interval corresponds to an increasing state value ranging from 1 to Q, and then the state intervals where the normalized acceleration values ​​at K time points are located are determined to obtain a state sequence, where Q is a user-defined positive integer, preferably, Q is set to 10; Step S303, counting the total number of occurrences of different state values ​​in the state sequence, and traversing the number of transitions of adjacent state values ​​in the state sequence to construct a Markov transition matrix; The size of the Markov transition matrix is ​​Q×Q, and the element value in the ath row and bth column represents the transition probability from the ath state value to the bth state value, where 1≤a≤Q, 1≤b≤Q; The probability that the ath state value transfers to the bth state value The calculation formula is as follows: ; in Indicates the number of times the a-th state value is transferred to the b-th state value, Indicates the total number of times the a-th state value appears; Step S304, constructing a Markov matrix according to the state sequence and the Markov transition matrix; The size of the Markov matrix is ​​K×K, and the element value in the cth row and dth column represents the probability that the cth state value of the state sequence is transferred to the dth state value, where 1≤c≤K, 1≤d≤K.

[0029] For example, assuming that the normalized vibration data is represented as: {0.1, 0.3, 0.4, 0.7, 0.9}, Q is set to 3, and the value range [0, 1] is divided into {[0, 0.33), [0.33, 0.66), [0.66, 1]}, and the corresponding state values ​​are 1, 2, and 3 respectively. Then the state sequence is represented as: {1, 1, 2, 3, 3}, and the total number of times the state value of 1 appears is 2, the total number of times the state value of 2 appears is 1, and the total number of times the state value of 3 appears is 2. Then traverse the transition probabilities of adjacent state values ​​in the state sequence. The number of times the state value 1 transfers to the state value 1 is 1, the number of times the state value 1 transfers to the state value 2 is 1, the number of times the state value 2 transfers to the state value 3 is 1, and the number of times the state value 3 transfers to the state value 3 is 1. Then the Markov transfer matrix is ​​expressed as: , then the element value of the first row and first column of the Markov matrix represents the probability of the state value 1 transferring to the state value 1, which is 0.5, and the element value of the first row and second column represents the probability of the state value 1 transferring to the state value 1, which is also 0.5, and so on. The final constructed Markov matrix is ​​expressed as follows: .

[0030] It should be noted that converting the vibration data into a Markov matrix fully retains the discrete time domain information, which can more comprehensively reflect the dynamic mode of blade vibration and enhance the model's understanding of the blade vibration mode. Secondly, it is convenient for the subsequent feature extraction layer to extract features and improve the accuracy of fault analysis.

[0031] In one embodiment of the present invention, the calculation formula of the feature extraction layer is as follows: ; Where Vector represents the feature vector output by the feature extraction layer, and MTF represents the Markov matrix input by the feature extraction layer. , and represents convolution operations where the size of the convolution kernel is 3×3 and the number of convolution kernels is 32, 64, and 128 respectively. represents a convolution operation with a kernel size of 1×1 and a number of 256 kernels. Indicates the maximum pooling operation with a pooling window size of 2×2, and It indicates that the size of the convolution kernel is 3×3 and the number of convolution kernels is 64, and the dilation rates are 2 and 4 respectively. GlobalMaxPooling indicates the global maximum pooling operation, and Dense indicates the fully connected layer; Definition: The filling method of convolution operation and dilated convolution operation is SAME, and the step size of convolution kernel is 1. The activation function of convolution operation and dilated convolution operation is ReLU activation function. The step size of pooling window of maximum pooling operation is 2. The activation function of fully connected layer is Swish activation function.

[0032] It should be noted that, for example, if the size of the Markov matrix is ​​512×512, then We get a feature map of size 512×512×32, where 32 represents the number of channels in the feature map. Get a feature map of size 256×256×32, and so on. A feature map of 64×64×256 is obtained, and then a global maximum pooling operation (the maximum value is selected on the entire feature map of each channel) is performed to obtain a feature map of 64×64×1 (expanded to a vector of 1×4096). The fully connected layer is used to reduce the feature dimension and improve the feature expression ability through nonlinear transformation to reduce the computational complexity. For example, a vector of 1×4096 is reduced to 512, which will not be elaborated here.

[0033] In one embodiment of the present invention, the timing analysis layer is constructed based on the Transformer model, and can also be constructed based on the LSTM model or the GRU model, and the number of dimensions of the update vector obtained by performing timing analysis on the feature sequence is a custom parameter. Preferably, the number of dimensions of the update vector is set to 64, which is not elaborated here.

[0034] In one embodiment of the present invention, before training the first neural network model, it is pre-trained. During the pre-training process of the first neural network model, the update vector is input into the second classifier, the third classifier, the fourth classifier, the fifth classifier and the sixth classifier. The classification spaces of the above classifiers respectively represent temperature, rainfall, snowfall, wind speed, and the angle between wind direction and blade direction.

[0035] It should be noted that pre-training the first neural network model can effectively reduce the number of subsequent training samples and speed up the training of the model. Pre-training can also extract features related to environmental data in vibration data, which can reduce the interference of environmental data on offshore wind turbine blade fault monitoring to a certain extent, thereby improving the monitoring accuracy of the model.

[0036] In one embodiment of the present invention, the second neural network model is composed of the trained first neural network model, N first convolutional layers, N second convolutional layers, a concatenation layer, a hidden layer, an extraction layer and a seventh classifier; The first convolution layer is used to extract features from the preprocessed common image to obtain a first vector, where the number of dimensions of the first vector is a custom parameter. Preferably, the number of dimensions of the first vector is set to 64; The second convolution layer is used to extract features from the preprocessed infrared image to obtain a second vector, where the number of dimensions of the second vector is a custom parameter. Preferably, the number of dimensions of the second vector is set to 64; The concatenation layer is used to concatenate the update vector output by the trained first neural network model with the environmental data, the first vector and the second vector to obtain a combined vector, and convert each sequence unit of the image sequence into a combined vector representation, that is, the number of dimensions of the combined vector is 64 (update vector) + 5 (environmental data) + 64 (first vector) + 64 (second vector) = 197; The hidden layer includes N hidden units, the nth hidden unit inputs the combination vector corresponding to the nth sequence unit of the image sequence, and outputs a hidden vector, where 1≤n≤N; The extraction layer is used to extract the hidden vector output by the Nth hidden unit and input it into the seventh classifier. The classification space of the seventh classifier represents the second health state of the offshore wind turbine blade. The sample labels of the training samples used to train the second neural network model are obtained through manual annotation.

[0037] In one embodiment of the present invention, the calculation formula of the nth hidden unit includes: ; ; in and Respectively represent the hidden vectors output by the nth hidden unit and the n-1th hidden unit, Assign a value of 0, Represents the combination vector corresponding to the nth sequence unit of the image sequence input by the nth hidden unit, represents the gating vector of the nth hidden unit, which has the same size as the hidden vector, and They represent the first weight parameter and the second weight parameter corresponding to the nth hidden unit respectively, and They represent the first bias parameter and the second bias parameter corresponding to the nth hidden unit, respectively. represents element-by-element multiplication, Swish represents the Swish activation function, and sigmoid represents the sigmoid activation function.

[0038] It should be noted that the weight parameters and bias parameters in the hidden unit are all learnable hyperparameters. For example, the size of the hidden vector can be designed to be 1×R, then the size of the gate vector is also 1×R, then the second weight parameter can be designed to be a matrix of R×R, the size of the combined vector is 1×197, then the first weight parameter can be designed to be a matrix of 197×R, where R is a custom parameter. Preferably, R is set to 64.

[0039] In one embodiment of the present invention, Figure 4 As shown, an offshore wind turbine blade health monitoring system includes: A data acquisition module 401 is used to collect environmental data of the offshore wind turbine blades and collect vibration data of M points on the offshore wind turbine blades; A first training module 402, which is used to train a first neural network model using environmental data and vibration data of M points; An image acquisition module 403 is used to acquire ordinary images and infrared images of offshore wind turbine blades within a preset time period T and at a preset time interval t; An image processing module 404 is used to pre-process the common image and infrared image of the offshore wind turbine blade to obtain an image sequence; A second training module 405, which is used to transfer the trained first neural network model to a second neural network model, and train the second neural network model through an image sequence; The wind turbine blade health monitoring module 406 is used to input the environmental data, the vibration data of M points and the image sequence into the trained second neural network model, and output the second health status of the offshore wind turbine blade.

[0040] The above describes an embodiment of the present embodiment, but the present embodiment is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present embodiment, ordinary technicians in this field can also make many forms, all of which are within the protection of the present embodiment.

Claims

1. A method for monitoring the health of offshore wind turbine blades, characterized in that: The following steps are involved: Step S101, collecting environmental data of the offshore wind turbine blades, and collecting vibration data of M points on the offshore wind turbine blades; Environmental data include: temperature, rainfall, snowfall, wind speed, and the angle between wind direction and blade orientation; The vibration data of M points are represented by the acceleration values ​​at K time points; Step S102, training a first neural network model using environmental data and vibration data of M points; The input of the first neural network model is the vibration data of M points, and the output is the first health state of the offshore wind turbine blade, which includes: normal blade, ice accumulation on the blade and loose connection; Step S103, collecting ordinary images and infrared images of offshore wind turbine blades at preset time intervals t within a preset time period T; Step S104, preprocessing the common image and infrared image of the offshore wind turbine blade to obtain an image sequence; The image sequence includes N sequence units, each of which consists of a preprocessed ordinary image and an infrared image, where N = T / t; Step S105, migrating the trained first neural network model to a second neural network model, and training the second neural network model through the image sequence; The input of the second neural network model includes the input of the first neural network model, and also includes an image sequence and environmental data, and the output is a second health state of the offshore wind turbine blade, the second health state includes: the first health state, surface stains, surface cracks, surface corrosion and internal cracks; Step S106, inputting the environmental data, the vibration data of the M points and the image sequence into the trained second neural network model, and outputting the second health state of the offshore wind turbine blade.

2. The offshore wind turbine blade health monitoring method according to claim 1, characterized in that: The number of points M, the number of time points K of vibration data, the preset time period T and the preset time interval t are all custom parameters.

3. The offshore wind turbine blade health monitoring method according to claim 1, characterized in that: Preprocessing the ordinary image and infrared image of the offshore wind turbine blade to obtain an image sequence includes the following steps: Step S201, graying the ordinary image, and scaling it and the infrared image to a unified preset size, wherein the preset size is a custom parameter; Step S202, performing noise reduction processing on the ordinary image and the infrared image by Gaussian filtering; Step S203, removing the background of the ordinary image and the infrared image by using a semantic segmentation model; Step S204, normalizing the ordinary image and the infrared image by using the Min-Max method; Step S205: randomly select a data enhancement method to perform data enhancement on the ordinary image and the infrared image, wherein the data enhancement method includes: rotation, flipping and translation.

4. The offshore wind turbine blade health monitoring method according to claim 1, characterized in that: The first neural network model consists of M conversion layers, M feature extraction layers, 1 sequence construction layer and 1 time series analysis layer; M conversion layers are used to convert the vibration data of M points into Markov matrices; The M feature extraction layers are used to extract features from the M Markov matrices to obtain feature vectors, and the M feature extraction layers share weight parameters; The sequence construction layer is used to concatenate M feature vectors to obtain a feature sequence. The feature sequence includes M sequence units, and each sequence unit corresponds to a feature vector. The timing analysis layer is used to perform timing analysis on feature sequences to obtain update vectors. The timing analysis layer is built based on the Transformer model, and the number of dimensions of the update vector is a custom parameter. During the training process of the first neural network model, the update vector is input into the first classifier, the classification space of the first classifier represents the first health state of the offshore wind turbine blade, and the sample labels of the training samples used to train the first neural network model are obtained through manual labeling.

5. The offshore wind turbine blade health monitoring method according to claim 4, characterized in that: The conversion layer converts the vibration data into a Markov matrix, which includes the following steps: Step S301, normalizing the vibration data using the Min-Max method; Control the range of the acceleration values ​​at K time points between 0 and 1; Step S302, discretizing the normalized vibration data to obtain a state sequence; The range of normalized acceleration values ​​is evenly divided into Q non-overlapping state intervals, each of which corresponds to an increasing state value ranging from 1 to Q. Then the state intervals of the normalized acceleration values ​​at K time points are determined to obtain a state sequence, where Q is a user-defined positive integer. Step S303, counting the total number of occurrences of different state values ​​in the state sequence, and traversing the number of transitions of adjacent state values ​​in the state sequence to construct a Markov transition matrix; The size of the Markov transition matrix is ​​Q×Q, and the element value in the ath row and bth column represents the transition probability from the ath state value to the bth state value, where 1≤a≤Q, 1≤b≤Q; The probability that the ath state value transfers to the bth state value The calculation formula is as follows: ,in Indicates the number of times the a-th state value is transferred to the b-th state value, Indicates the total number of times the a-th state value appears; Step S304, constructing a Markov matrix according to the state sequence and the Markov transition matrix; The size of the Markov matrix is ​​K×K, and the element value in the cth row and dth column represents the probability that the cth state value of the state sequence is transferred to the dth state value, where 1≤c≤K, 1≤d≤K.

6. The offshore wind turbine blade health monitoring method according to claim 4, characterized in that: The calculation formula of the feature extraction layer is as follows: ； Where Vector represents the feature vector output by the feature extraction layer, and MTF represents the Markov matrix input by the feature extraction layer. , and represents convolution operations where the size of the convolution kernel is 3×3 and the number of convolution kernels is 32, 64, and 128 respectively. represents a convolution operation with a kernel size of 1×1 and a number of 256 kernels. Indicates the maximum pooling operation with a pooling window size of 2×2, and It indicates that the size of the convolution kernel is 3×3 and the number of convolution kernels is 64, and the dilation rates are 2 and 4 respectively. GlobalMaxPooling indicates the global maximum pooling operation, and Dense indicates the fully connected layer; Definition: The filling method of convolution operation and dilated convolution operation is SAME, and the step size of convolution kernel is 1. The activation function of convolution operation and dilated convolution operation is ReLU activation function. The step size of pooling window of maximum pooling operation is 2. The activation function of fully connected layer is Swish activation function.

7. The offshore wind turbine blade health monitoring method according to claim 1, characterized in that: Before training the first neural network model, it is pre-trained first. During the pre-training process of the first neural network model, the update vector is input into the second classifier, the third classifier, the fourth classifier, the fifth classifier and the sixth classifier. The classification spaces of the above classifiers respectively represent temperature, rainfall, snowfall, wind speed, and the angle between wind direction and blade direction.

8. The offshore wind turbine blade health monitoring method according to claim 1, characterized in that: The second neural network model is composed of the trained first neural network model, N first convolutional layers, N second convolutional layers, a concatenation layer, a hidden layer, an extraction layer and a seventh classifier; The first convolutional layer is used to extract features from the preprocessed common image to obtain a first vector, where the number of dimensions of the first vector is a custom parameter; The second convolutional layer is used to extract features from the preprocessed infrared image to obtain a second vector, where the number of dimensions of the second vector is a custom parameter; The concatenation layer is used to concatenate the update vector output by the trained first neural network model with the environmental data, the first vector and the second vector to obtain a combined vector, and convert each sequence unit of the image sequence into a combined vector representation; The hidden layer includes N hidden units, the nth hidden unit inputs the combination vector corresponding to the nth sequence unit of the image sequence, and outputs a hidden vector, where 1≤n≤N; The extraction layer is used to extract the hidden vector output by the Nth hidden unit and input it into the seventh classifier. The classification space of the seventh classifier represents the second health state of the offshore wind turbine blade. The sample labels of the training samples used to train the second neural network model are obtained through manual annotation.

9. The offshore wind turbine blade health monitoring method according to claim 8, characterized in that: The calculation formula for the nth hidden unit includes: ； ； in and Respectively represent the hidden vectors output by the nth hidden unit and the n-1th hidden unit, Assign a value of 0, Represents the combination vector corresponding to the nth sequence unit of the image sequence input by the nth hidden unit, represents the gating vector of the nth hidden unit, which has the same size as the hidden vector, and They represent the first weight parameter and the second weight parameter corresponding to the nth hidden unit respectively, and They represent the first bias parameter and the second bias parameter corresponding to the nth hidden unit, respectively. represents element-by-element multiplication, Swish represents the Swish activation function, and sigmoid represents the sigmoid activation function.

10. An offshore wind turbine blade health monitoring system, executing an offshore wind turbine blade health monitoring method as claimed in any one of claims 1 to 9, characterized in that: include: A data acquisition module, which is used to collect environmental data of the offshore wind turbine blades and collect vibration data of M points on the offshore wind turbine blades; A first training module, which is used to train a first neural network model through environmental data and vibration data of M points; An image acquisition module, which is used to acquire ordinary images and infrared images of offshore wind turbine blades within a preset time period T and at a preset time interval t; An image processing module, which is used to pre-process ordinary images and infrared images of offshore wind turbine blades to obtain an image sequence; A second training module, which is used to transfer the trained first neural network model to a second neural network model, and train the second neural network model through an image sequence; The wind turbine blade health monitoring module is used to input environmental data, vibration data of M points and image sequences into the trained second neural network model, and output the second health status of the offshore wind turbine blades.

Images