Wind turbine blade vibration monitoring method and system based on unmanned aerial vehicle and DIC

Abstract

The invention discloses a wind turbine blade vibration monitoring method and system based on an unmanned aerial vehicle and a DIC, and relates to the technical field of wind turbine blade vibration monitoring, and the method comprises the steps: employing the unmanned aerial vehicle to shoot a wind turbine blade according to a preset flight path, and obtaining the surface image data of the wind turbine blade; based on a GPS sensor and image data of the unmanned aerial vehicle, the attitude of the unmanned aerial vehicle is optimized by using an image optimization method of attitude estimation in SLAM, and a corrected image is obtained; generating normal and deformed digital speckle images based on the corrected image; constructing a three-dimensional digital image correlation measurement network model, wherein the model comprises a feature extraction module, a cost body generation module and a displacement map prediction module; and training, verifying and testing the model by using a speckle image, inputting a to-be-tested image into the final model, performing frequency analysis on a generated displacement cloud picture, and determining the current state of the blade. The problem of dynamic identification of the outdoor large-scale wind turbine blade is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of wind turbine blade vibration monitoring, and in particular to a wind turbine blade vibration monitoring method and system based on an unmanned aerial vehicle and a DIC. Background Art

[0002] Wind blades are the core equipment of wind power generation, and it is crucial to monitor their damage to avoid catastrophic consequences for wind turbines. Nondestructive testing (NDT) is widely used in structural vibration and deformation monitoring of wind turbine blades because it does not affect the physical, chemical, mechanical or dimensional properties of the structure to be tested. At present, commonly used nondestructive testing technologies mainly include strain measurement and machine vision.

[0003] Strain measurement is the most traditional damage detection method. It can generate a two-dimensional strain map and deflection shape of the surface of an object for online status assessment. Damage detection based on strain measurement has the following problems: 1. It is difficult to arrange sensors. It is very difficult to install sensors in narrow spaces, complex surfaces, or areas close to high temperature and high humidity environments. 2. The number of sensors required is large. For large structures, a large number of sensors are often required to cover them. 3. The installation process is complicated. The installation process of the sensor may require drilling holes, welding, or bonding of the structure.

[0004] Machine vision-based detection technology is the current mainstream method, which uses two-dimensional / three-dimensional image sequences from different positions and perspectives to obtain information about the target object. Its principle is similar to the stereoscopic view of human vision. Two-dimensional image sequences have the advantages of mature technology, low cost, and light computational burden, and are suitable for most applications that require planar information, such as object recognition, video surveillance, and visual inspection. However, its disadvantage is that it cannot provide depth information, which limits its application in complex three-dimensional environment analysis. Three-dimensional image sequences can provide more spatial information and depth data, and are suitable for applications with high precision requirements such as three-dimensional reconstruction, environmental perception, robot navigation, and autonomous driving. Machine vision-based methods have been widely used in damage detection and dynamic recognition of wind turbine blades. This method can not only improve the accuracy and efficiency of detection, but also is less affected by environmental factors than other detection methods.

[0005] At present, there are many methods to realize the dynamic recognition of wind turbine blades based on machine vision. Generally speaking, most dynamic recognition technologies based on machine vision collect images of the blade surface through fixed cameras and use digital image correlation technology (DIC) to detect the vibration deformation of the blade. However, these methods are usually only suitable for close-range measurements and are mostly used for fatigue tests on large wind turbine blades indoors. Outdoors, large wind turbines are often hundreds of meters high. Due to the limitation of resolution, it is difficult for fixed cameras to capture clear images, which greatly affects the development of dynamic recognition technology for large outdoor wind turbine blades. Summary of the invention

[0006] The purpose of the present invention is to solve the problem of dynamic identification of outdoor large wind turbine blades, and propose a wind turbine blade vibration monitoring method based on drone and DIC, which includes the following steps:

[0007] S1. Use a drone to photograph the wind turbine blades according to a preset flight path to obtain surface image data of the wind turbine blades;

[0008] S2. Based on the GPS sensor of the drone and the image data, the posture of the drone is optimized using the graph optimization method of posture estimation in SLAM to obtain a corrected image;

[0009] S3. Based on the corrected images, an open source stereoscopic speckle image generation algorithm is used to generate normal and deformed digital speckle images, and the normal and deformed digital speckle images are divided into a training set, a validation set, and a test set;

[0010] S4, constructing a three-dimensional digital image correlation measurement network model, the model comprising three modules: feature extraction, cost volume generation, and displacement map prediction;

[0011] The feature extraction module includes two feature extraction units, which respectively extract the features of the normal digital speckle image and the deformed digital speckle image; the cost volume generation module generates the cost volume according to the features of the normal digital speckle image and the deformed digital speckle image, and the cost volume is passed through the displacement map prediction module to obtain the predicted displacement cloud map;

[0012] S5. Use the training set, the validation set, and the test set to train, validate, and test the model to obtain a final model, input the image to be tested into the final model, perform frequency analysis on the generated displacement cloud map of the wind turbine blade, and determine the current state of the blade.

[0013] Furthermore, the objective function of drone attitude optimization is as follows:

[0014]

[0015] Among them, F(x) represents the objective function of drone posture optimization, x represents the position of the drone, C represents the index pair set between the connection nodes, and T i and T j They represent the position of the UAV at different times, e ij According to T i and T j Error items of the build, Indicates T i and T j The relative position change of ij represents the information matrix;

[0016] Set the initial point of the optimization problem to And at the initial point Add an increment to the above, and the estimated value between the state nodes of the posture graph is The error between nodes is The overall error cost function is linearized;

[0017]

[0018] in, represents the estimated value between state nodes, represents the initial point, Δx represents the increment, represents the error value between nodes, Λ k represents the information matrix, J k represents the Jacobian matrix, C k represents the constant term, b k represents the linear term of the error, H k represents the Hessian matrix.

[0019] Furthermore, the cost volume generation module includes: two multi-scale adaptive patch matching units, an attention unit, two two-dimensional convolutional layers, and a convolutional block; the convolutional block is composed of three two-dimensional convolutional layers stacked;

[0020] Characteristics of normal digital speckle images and deformed digital speckle images r and f d , f r and f d The correlation volume V is obtained by the first multi-scale adaptive patch matching unit 相关 , f r Through the first two-dimensional convolutional layer, we get f r ′ , f d The volume V is obtained by a second multi-scale adaptive patch matching unit d , f d Through the second 2D convolutional layer, we get f d′ , f r ′ 、V d and f d ′ Splicing to obtain connector V 连接 , related body V 相关 Get V through the convolution block 相关 ′ , V 相关 ′ and V 连接 The cost body is obtained through the attention unit.

[0021] Furthermore, the feature extraction unit includes six two-dimensional convolutional layers, two two-dimensional deconvolutional layers, the first and the second two-dimensional convolutional layer are stacked as a first convolutional block, the third two-dimensional convolutional layer and the first two-dimensional deconvolutional layer are stacked as a second convolutional block, the fourth and the fifth two-dimensional convolutional layer are stacked as a third convolutional block, the sixth two-dimensional convolutional layer and the second two-dimensional deconvolutional layer are stacked as a fourth convolutional block, the four convolutional blocks are connected in series, and the output of the first convolutional block is connected to the output of the second convolutional block and the output of the fourth convolutional block through a jump connection.

[0022] Furthermore, the feature extraction unit includes two 1×1 convolutional layers and four 3×3 convolutional layers connected in series. The input of the feature extraction unit passes through the first 1×1 convolutional layer and the first 3×3 convolutional layer respectively. The output of the first 1×1 convolution is spliced ​​with the output of the second 3×3 convolutional layer through a jump connection. The spliced ​​feature map is input into the third 3×3 convolutional layer and the second 1×1 convolutional layer. The output of the second 1×1 convolutional layer is spliced ​​with the output of the fourth 3×3 convolutional layer through a jump connection to obtain the output of the feature extraction unit.

[0023] Furthermore, the multi-scale adaptive patch matching unit first divides the feature map input by the multi-scale adaptive patch matching unit into patches of different scales, and concatenates the matching cost of each patch to obtain the output of the multi-scale adaptive patch matching unit:

[0024]

[0025] Among them, C correlation represents the output of the multi-scale adaptive patch matching unit, Concat{} represents the concatenation operation, represents the matching cost of group 1, represents the matching cost of group 2, represents the matching cost of group 3;

[0026] The matching cost of each group is calculated by the following formula:

[0027]

[0028] in, represents the matching cost of the kth group, N g Represents the number of feature groups, N f represents the number of feature channels, Ω k represents a set of nine-point coordinates that determine the size and dilation ratio of the patch, (i, j) represents a pixel, w i,j represents the weight of pixel (i, j), f(xi, yj) represents the feature map of the input multi-scale adaptive patch matching unit, and (x, y) represents the position of the pixel.

[0029] Furthermore, the displacement map prediction module includes four two-dimensional convolutional layers and two two-dimensional deconvolutional layers. The first two-dimensional convolutional layer, the second two-dimensional convolutional layer, and the first two-dimensional deconvolutional layer are stacked into a convolutional block CB1, and the third two-dimensional convolutional layer, the fourth two-dimensional convolutional layer, and the second two-dimensional deconvolutional layer are stacked into a convolutional block CB2. CB1 and CB2 are connected in series.

[0030] The present invention also proposes a wind turbine blade vibration monitoring system based on a drone and a DIC, comprising:

[0031] A data acquisition module is used to use a drone to photograph the wind turbine blades according to a preset flight path to obtain surface image data of the wind turbine blades;

[0032] An image correction module is used to optimize the posture of the drone based on the drone GPS sensor and the image data using a graph optimization method for posture estimation in SLAM to obtain a corrected image;

[0033] A digital speckle image generation module is used to generate normal and deformed digital speckle images based on the corrected image using an open source stereoscopic speckle image generation algorithm, and divide the normal and deformed digital speckle images into a training set, a validation set, and a test set;

[0034] A model building module is used to build a three-dimensional digital image correlation measurement network model, the model includes three modules: feature extraction, cost volume generation, and displacement map prediction;

[0035] The feature extraction module includes two feature extraction units, which respectively extract the features of the normal digital speckle image and the deformed digital speckle image; the cost volume generation module generates the cost volume according to the features of the normal digital speckle image and the deformed digital speckle image, and the cost volume is passed through the displacement map prediction module to obtain the predicted displacement cloud map;

[0036] The result acquisition module is used to train, verify and test the model using a training set, a validation set and a test set to obtain a final model, input the image to be tested into the final model, perform frequency analysis on the generated displacement cloud map of the wind turbine blade, and determine the current state of the blade.

[0037] The beneficial effects brought by the technical solution provided by the present invention are:

[0038] The present invention uses a drone to take pictures of wind turbine blades. Aiming at the problem of drone hovering and drift, the drone posture is optimized based on the sensor fusion algorithm optimized by the posture graph, which overcomes the limitations of dynamic scenes or long-distance targets of outdoor large wind turbine blades and ensures the quality of the image. A stereoscopic digital image correlation measurement network model is constructed to predict the displacement, including three modules: feature extraction, cost volume generation, and displacement map prediction. The feature extraction module uses residual layers and convolutional layers to obtain features with different receptive fields to enhance the robustness of matching features under deformations of different scales. The cost volume generation module combines the advantages of the connection body and the correlation body to construct an attention connection volume, which can provide the possibility of measuring deformations at different scales. The stacked convolutional layers of the displacement map prediction module aggregate the similarity information of the cost volumes at different scales to improve the measurement accuracy of deformations at different scales. The problem of dynamic recognition of outdoor large wind turbine blades is solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 is a flow chart of a wind turbine blade vibration monitoring method based on a drone and a DIC according to an embodiment of the present invention;

[0040] Figure 2 is a schematic diagram of drone data collection according to an embodiment of the present invention;

[0041] Figure 3 is a structural diagram of a stereo digital image correlation measurement network model according to an embodiment of the present invention;

[0042] Figure 4 It is a structural diagram of a feature extraction unit of another preferred embodiment of the embodiment of the present invention. DETAILED DESCRIPTION

[0043] To make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0044] The flow chart of the wind turbine blade vibration monitoring method based on drone and DIC in the embodiment of the present invention is as follows: Figure 1 , specifically including the following steps:

[0045] S1. Use a drone to photograph wind turbine blades according to a preset flight path to obtain surface image data of the wind turbine blades.

[0046] The drone takes off from the downwind side of the wind turbine and flies along the leading edge to the trailing edge of the wind turbine blade according to the preset flight path. During the flight, the drone equipped with a high-definition camera continuously shoots the blade surface to obtain high-resolution image data. The image data is a group of normal wind turbine blade images and deformed wind turbine blade images at the same position. Figure 2 , Figure 2 Schematic diagram of drone data collection according to an embodiment of the present invention.

[0047] S2. Based on the UAV GPS sensor and the surface image data of the wind turbine blade, the UAV posture is optimized using the graph optimization method of pose estimation in SLAM to obtain the corrected image.

[0048] The absolute position data obtained from the GPS sensor is converted into a coordinate system to obtain the position in the east, north, and up (ENU) coordinate system. The relative position of the drone is determined with the position at the first moment as the origin. Pose graph optimization is a computational method used in SLAM to optimize the robot's posture. Pose graph optimization excludes many three-dimensional spatial points considered in BA optimization and only focuses on the position and posture of the drone. This greatly reduces the optimization scale and avoids unnecessary consumption of computing power. Pose graphs are composed of nodes and edges, where: nodes correspond to the robot's posture in the environment and are also the optimization target; edges represent spatial constraints between different nodes.

[0049] The objective function of drone attitude optimization is as follows:

[0050]

[0051] Among them, F(x) represents the objective function of drone posture optimization, x represents the position of the drone, C represents the index pair set between the connection nodes, and T i and T j They represent the position of the UAV at different times, e ij According to T i and T j Error items of the build, Indicates T i and T j The relative position change of ij represents the information matrix, which is the inverse of the covariance and represents the weight of each error in the optimization function.

[0052] Set the initial point of the optimization problem to And at the initial point Add an increment to the state node, and the estimated value between the state nodes is The error between nodes is The overall error cost function is linearized.

[0053]

[0054] in, Represents the estimated value between state nodes. represents the initial point, Δx represents the increment, represents the error value between nodes, Λ k represents the information matrix, J k represents the Jacobian matrix, C k represents the constant term, b k represents the linear term of the error, H k represents the Hessian matrix.

[0055] The final result In, C k is a constant term, we will find At minimum Convert to 2b k Δx+Δx T H k Δx first-order derivative (to find 2b k Δx+Δx T H k When the first derivative of Δx with respect to Δx is 0 The value of .

[0056] That is: 2b k +2H k Δx=0 to get H k Δx=-b k ,get The optimal solution of .

[0057] S3. Based on the corrected image, an open source stereoscopic speckle image generation algorithm is used to generate normal and deformed digital speckle images, and the normal and deformed digital speckle images are divided into a training set, a validation set, and a test set.

[0058] Specifically, during the dataset generation process, the displacement amplitudes of in-plane deformation and parallax were set to 0.05 mm and 0.5 mm, respectively, the maximum in-plane deformation in the x and y directions were 1 pixel and 5 pixels, respectively, and the parallax was about 10 pixels. A total of 750 sets of data were generated, each containing 4 training samples, of which 67% of the samples were used as training sets, 22% of the samples were used as validation sets, and 11% of the samples were used as test sets.

[0059] S4. Construct a three-dimensional digital image correlation measurement network model, which includes three modules: feature extraction, cost volume generation, and displacement map prediction.

[0060] The feature extraction module includes two feature extraction units. The cost volume generation module includes: two multi-scale adaptive patch matching units, an attention unit, two 2D convolutional layers, and a convolutional block; the convolutional block is composed of three stacked 2D convolutional layers. The stride of the two 2D convolutional layers is 1 by default, and the stride of the three stacked 2D convolutional layers is 2, 3, and 2 respectively.

[0061] The structural diagram of the stereo digital image correlation measurement network model of the embodiment of the present invention is shown in FIG. Figure 3 The two feature extraction units extract the features of the normal digital speckle image and the deformed digital speckle image respectively. r and f d , f r and f d The correlation volume V is obtained by the first multi-scale adaptive patch matching unit 相关 , f r Through the first two-dimensional convolutional layer, we get f r ′ , f d The correlation volume V is obtained by the second multi-scale adaptive patch matching unit d , f d Through the second 2D convolutional layer, we get f d ′ , f r ′ 、V d and f d ′ Splicing to obtain connector V 连接 , related body V 相关 Through the convolution layer, we get V 相关 ′ , V 相关 ′ and V 连接 Through the attention unit, the attention connectome, i.e., the cost body, is obtained. The attention unit adopts the channel and spatial attention module (CBAM). The attention connectome passes through the displacement map prediction module to obtain the predicted displacement map.

[0062] Specifically, feature f r and f d is the feature map of 320 channels, f r ′ and f d ′ It is a 32-channel feature map.

[0063] In a preferred embodiment of the present invention, reference Figure 3The feature extraction unit in the feature extraction unit includes six two-dimensional convolutional layers, two two-dimensional deconvolutional layers, the first and second two-dimensional convolutional layers are stacked as the first convolutional block, the third two-dimensional convolutional layer and the first two-dimensional deconvolutional layer are stacked as the second convolutional block, the fourth and fifth two-dimensional convolutional layers are stacked as the third convolutional block, the sixth two-dimensional convolutional layer and the second two-dimensional deconvolutional layer are stacked as the fourth convolutional block, the four convolutional blocks are connected in series, and the output of the first convolutional block is connected to the output of the second convolutional block and the output of the fourth convolutional block through shotcut jump. Among them, the stride of the first two-dimensional convolutional layer, the second two-dimensional convolutional layer, the fourth two-dimensional convolutional layer, and the fifth two-dimensional convolutional layer is 2, and the stride of the third two-dimensional convolutional layer and the sixth two-dimensional convolutional layer is 1 by default.

[0064] In another preferred embodiment of the present invention, reference Figure 4 , Figure 4 This is a structural diagram of a feature extraction unit of another preferred embodiment of the present invention, wherein the feature extraction unit includes two 1×1 convolutional layers and four 3×3 convolutional layers connected in series, and the input of the feature extraction unit passes through the first 1×1 convolutional layer and the first 3×3 convolutional layer respectively, and the output of the first 1×1 convolution is spliced ​​with the output of the second 3×3 convolutional layer through a jump connection, and the spliced ​​feature map is input to the third 3×3 convolutional layer and the second 1×1 convolutional layer, and the output of the second 1×1 convolutional layer is spliced ​​with the output of the fourth 3×3 convolutional layer through a jump connection to obtain the output of the feature extraction unit. Among them, the first and third 3×3 convolutional layers include a 3×3 convolution and a ReLU activation function, the second and fourth 3×3 convolutional layers include a 3×3 convolution and a BN layer (Batch Normalization), and the two 1×1 convolutional layers each include a 1×1 convolution and a BN layer.

[0065] The feature extraction unit uses two residual layers to obtain features with different receptive fields to enhance the robustness of matching features under deformations of different scales.

[0066] The multi-scale adaptive patch matching unit first divides the feature map input by the multi-scale adaptive patch matching unit into patches of different scales, and concatenates the matching cost of each patch to obtain the output of the multi-scale adaptive patch matching unit:

[0067]

[0068] Among them, C correlation represents the output of the multi-scale adaptive patch matching unit, Concat{} represents the concatenation operation, represents the matching cost of group 1, represents the matching cost of group 2, Represents the matching cost of group 3.

[0069] The matching cost of each group is calculated by the following formula:

[0070]

[0071] in, represents the matching cost of the kth group, N g Represents the number of feature groups, N f represents the number of feature channels, Ω k represents a set of nine-point coordinates that determine the size and dilation ratio of the patch, (i, j) represents a pixel, w i,j Represents the weight of pixel (i, j), which is also the training parameter of the network, f(xi, yj) represents the feature map of the input multi-scale adaptive patch matching unit, and (x, y) represents the position of the pixel.

[0072] The displacement map prediction module includes four two-dimensional convolutional layers and two two-dimensional deconvolutional layers. The first two-dimensional convolutional layer, the second two-dimensional convolutional layer, and the first two-dimensional deconvolutional layer are stacked into a convolutional block CB1, and the third two-dimensional convolutional layer, the fourth two-dimensional convolutional layer, and the second two-dimensional deconvolutional layer are stacked into a convolutional block CB2. CB1 and CB2 are connected in series.

[0073] S5. Use the training set, validation set, and test set to train, validate, and test the above-mentioned stereoscopic digital image correlation measurement network model to obtain a final model, input the image to be measured into the final model, generate a displacement cloud map of the wind turbine blade, perform frequency analysis on the displacement cloud map, and determine the current state of the blade.

[0074] In an exemplary embodiment, a wind turbine blade vibration monitoring system based on a drone and DIC is provided, comprising:

[0075] A data acquisition module is used to use a drone to photograph the wind turbine blades according to a preset flight path to obtain surface image data of the wind turbine blades;

[0076] The image correction module is used to optimize the UAV posture based on the UAV GPS sensor and the wind turbine blade surface image data, using the graph optimization method of posture estimation in SLAM to obtain the corrected image;

[0077] A digital speckle image generation module is used to generate normal and deformed digital speckle images based on the corrected image using an open source stereoscopic speckle image generation algorithm, and divide the normal and deformed digital speckle images into a training set, a validation set, and a test set;

[0078] A model building module is used to build a three-dimensional digital image correlation measurement network model, the model includes three modules: feature extraction, cost volume generation, and displacement map prediction;

[0079] The feature extraction module includes two feature extraction units, which respectively extract the features of the normal digital speckle image and the deformed digital speckle image; the cost volume generation module generates the cost volume according to the features of the normal digital speckle image and the deformed digital speckle image, and the cost volume is passed through the displacement map prediction module to obtain the predicted displacement cloud map;

[0080] The result acquisition module is used to train, verify and test the model using a training set, a validation set and a test set to obtain a final model, input the image to be tested into the final model, perform frequency analysis on the generated displacement cloud map of the wind turbine blade, and determine the current state of the blade.

[0081] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A wind turbine blade vibration monitoring method based on drone and DIC, characterized in that: The following steps are involved: S1. Use a drone to photograph the wind turbine blades according to a preset flight path to obtain surface image data of the wind turbine blades; S2. Based on the GPS sensor of the drone and the image data, the posture of the drone is optimized using the graph optimization method of posture estimation in SLAM to obtain a corrected image; S3. Based on the corrected images, an open source stereoscopic speckle image generation algorithm is used to generate normal and deformed digital speckle images, and the normal and deformed digital speckle images are divided into a training set, a validation set, and a test set; S4, constructing a three-dimensional digital image correlation measurement network model, the model comprising three modules: feature extraction, cost volume generation, and displacement map prediction; The feature extraction module includes two feature extraction units, which respectively extract the features of the normal digital speckle image and the deformed digital speckle image; the cost volume generation module generates the cost volume according to the features of the normal digital speckle image and the deformed digital speckle image, and the cost volume is passed through the displacement map prediction module to obtain the predicted displacement cloud map; S5. Use the training set, validation set, and test set to train, validate, and test the model to obtain a final model, input the image to be tested into the final model, perform frequency analysis on the generated displacement cloud map of the wind turbine blade, and determine the current state of the blade.

2. A wind turbine blade vibration monitoring method based on drone and DIC according to claim 1, characterized in that: The objective function of drone attitude optimization is as follows: Among them, F(x) represents the objective function of drone posture optimization, x represents the position of the drone, C represents the index pair set between the connection nodes, and T i and T j They represent the postures of the drone at different times, e ij According to T i and T j Error items of the build, Indicates T i and T j The relative position change of ij represents the information matrix; Set the initial point of the optimization problem to And at the initial point Add an increment to the above, and the estimated value between the state nodes of the posture graph is The error between nodes is The overall error cost function is linearized; in, represents the estimated value between state nodes, represents the initial point, Δx represents the increment, represents the error value between nodes, Λ k represents the information matrix, J k represents the Jacobian matrix, C k represents the constant term, b k represents the linear term of the error, H k represents the Hessian matrix.

3. The wind turbine blade vibration monitoring method based on drone and DIC according to claim 1 is characterized in that: The cost volume generation module includes: two multi-scale adaptive patch matching units, an attention unit, two two-dimensional convolutional layers, and a convolutional block; the convolutional block is composed of three two-dimensional convolutional layers stacked; Characteristics of normal digital speckle images and deformed digital speckle images r and f d The correlation volume V is obtained by the first multi-scale adaptive patch matching unit 相关 , f r Through the first two-dimensional convolutional layer, we get f r ′ , f d The volume V is obtained by a second multi-scale adaptive patch matching unit d , f d Through the second two-dimensional convolutional layer, we get f d ′ , f r ′ 、V d and f d ′ Splicing to obtain connector V 连接 , related body V 相关 Get V through the convolution block 相关 ′ , V 相关 ′ and V 连接 The cost body is obtained through the attention unit.

4. The wind turbine blade vibration monitoring method based on drone and DIC according to claim 1 is characterized in that: The feature extraction unit includes six two-dimensional convolutional layers, two two-dimensional deconvolutional layers, the first and second two-dimensional convolutional layers are stacked as the first convolutional block, the third two-dimensional convolutional layer and the first two-dimensional deconvolutional layer are stacked as the second convolutional block, the fourth and fifth two-dimensional convolutional layers are stacked as the third convolutional block, the sixth two-dimensional convolutional layer and the second two-dimensional deconvolutional layer are stacked as the fourth convolutional block, the four convolutional blocks are connected in series, and the output of the first convolutional block is connected to the output of the second convolutional block and the output of the fourth convolutional block through jump connection.

5. The wind turbine blade vibration monitoring method based on drone and DIC according to claim 1 is characterized in that: The feature extraction unit includes two 1×1 convolutional layers and four 3×3 convolutional layers in series. The input of the feature extraction unit passes through the first 1×1 convolutional layer and the first 3×3 convolutional layer respectively. The output of the first 1×1 convolution is spliced ​​with the output of the second 3×3 convolutional layer through a jump connection. The spliced ​​feature map is input into the third 3×3 convolutional layer and the second 1×1 convolutional layer. The output of the second 1×1 convolutional layer is spliced ​​with the output of the fourth 3×3 convolutional layer through a jump connection to obtain the output of the feature extraction unit.

6. The wind turbine blade vibration monitoring method based on drone and DIC according to claim 1 is characterized in that: The multi-scale adaptive patch matching unit first divides the feature map input by the multi-scale adaptive patch matching unit into patches of different scales, and concatenates the matching cost of each patch to obtain the output of the multi-scale adaptive patch matching unit: Among them, C correlation represents the output of the multi-scale adaptive patch matching unit, Concat{} represents the concatenation operation, represents the matching cost of group 1, represents the matching cost of group 2, represents the matching cost of group 3; The matching cost of each group is calculated by the following formula: in, represents the matching cost of the kth group, N g Represents the number of feature groups, N f represents the number of feature channels, Ω k represents a set of nine-point coordinates that determine the size and dilation ratio of the patch, (i, j) represents a pixel, w i,j represents the weight of pixel (i, j), f(xi, yj) represents the feature map of the input multi-scale adaptive patch matching unit, and (x, y) represents the position of the pixel.

7. The wind turbine blade vibration monitoring method based on drone and DIC according to claim 1 is characterized in that: The displacement map prediction module includes four two-dimensional convolutional layers and two two-dimensional deconvolutional layers. The first two-dimensional convolutional layer, the second two-dimensional convolutional layer, and the first two-dimensional deconvolutional layer are stacked into a convolutional block CB1, and the third two-dimensional convolutional layer, the fourth two-dimensional convolutional layer, and the second two-dimensional deconvolutional layer are stacked into a convolutional block CB2. CB1 and CB2 are connected in series.

8. A wind turbine blade vibration monitoring system based on drone and DIC, characterized in that: include: A data acquisition module is used to use a drone to photograph the wind turbine blades according to a preset flight path to obtain surface image data of the wind turbine blades; An image correction module is used to optimize the posture of the drone based on the drone GPS sensor and the image data using a graph optimization method for posture estimation in SLAM to obtain a corrected image; A digital speckle image generation module is used to generate normal and deformed digital speckle images based on the corrected image using an open source stereoscopic speckle image generation algorithm, and divide the normal and deformed digital speckle images into a training set, a validation set, and a test set; A model building module is used to build a three-dimensional digital image correlation measurement network model, the model includes three modules: feature extraction, cost volume generation, and displacement map prediction; The feature extraction module includes two feature extraction units, which respectively extract the features of the normal digital speckle image and the deformed digital speckle image; the cost volume generation module generates the cost volume according to the features of the normal digital speckle image and the deformed digital speckle image, and the cost volume is passed through the displacement map prediction module to obtain the predicted displacement cloud map; The result acquisition module is used to train, verify and test the model using a training set, a validation set and a test set to obtain a final model, input the image to be tested into the final model, perform frequency analysis on the generated displacement cloud map of the wind turbine blade, and determine the current state of the blade.

Images