Wind turbine blade vibration monitoring method and system based on unmanned aerial vehicle and DIC

Abstract

The application discloses a wind turbine blade vibration monitoring method and system based on a UAV and DIC, relates to the technical field of wind turbine blade vibration monitoring, and comprises the following steps: using a UAV to shoot a wind turbine blade according to a preset flight path, and acquiring wind turbine blade surface image data; based on a UAV GPS sensor and the image data, using a graph optimization method of SLAM pose estimation to optimize the UAV attitude, and obtaining corrected images; based on the corrected images, generating normal and deformed digital speckle images; constructing a three-dimensional digital image correlation measurement network model, the model comprising three modules, namely, feature extraction, cost body generation and displacement map prediction; using the speckle images to train, verify and test the model, inputting a to-be-measured image into the final model, performing frequency analysis on a generated displacement cloud image, and determining the current state of the blade. The application solves the problem of outdoor large-scale wind turbine blade dynamic identification.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of wind turbine blade vibration monitoring, and particularly relates to a wind turbine blade vibration monitoring method and system based on a UAV and DIC. BACKGROUND

[0002] Wind blades are the core equipment of wind power generation, and it is crucial to monitor their damage to avoid catastrophic consequences for wind turbines. Non-destructive testing (NDT) is widely used in structural vibration and deformation monitoring of wind turbine blades because it does not affect the physical, chemical, mechanical or dimensional characteristics of the structure being tested. Currently, commonly used non-destructive testing techniques mainly include strain measurement and machine vision.

[0003] Strain measurement is the most traditional damage detection method, which can generate two-dimensional strain maps and deflection shapes of the object surface for online condition assessment. Damage detection based on strain measurement has the following problems: 1. Sensor placement is difficult. It is very difficult to install sensors in narrow spaces, complex surfaces or areas close to high temperature and high humidity environments. 2. Large number of sensors are required. For large structures, a large number of sensors are often required for coverage. 3. The installation process is complex. The installation process of the sensor may require drilling, welding or bonding of the structure.

[0004] Machine vision-based detection technology is the current mainstream method, which uses two-dimensional / three-dimensional image sequences from different positions and perspectives to obtain information about the target object, and its principle is similar to human visual stereopsis. Two-dimensional image sequences have the advantages of mature technology, low cost and light computing burden, and are suitable for most applications that require planar information, such as object recognition, video surveillance, visual inspection, etc. However, its disadvantage is that it cannot provide depth information, limiting its application in complex three-dimensional environment analysis. Three-dimensional image sequences can provide more spatial information and depth data, and are suitable for three-dimensional reconstruction, environment perception, robot navigation, autonomous driving and other high-precision applications. Machine vision-based methods have been widely used in damage detection and dynamic identification of wind turbine blades. This method not only improves the accuracy and efficiency of detection, but also has less impact on environmental factors compared to other detection methods.

[0005] Currently, many methods have been developed to achieve dynamic recognition of wind turbine blades based on machine vision. Generally speaking, most machine vision-based dynamic recognition technologies use fixed cameras to capture images of the blade surface and utilize digital image correlation (DIC) technology to detect blade vibration deformation. However, these methods are generally only suitable for close-range measurements and are mostly used for indoor fatigue testing of large wind turbine blades. Outdoors, large wind turbines are often hundreds of meters tall, and fixed cameras have difficulty capturing clear images due to resolution limitations. This has greatly hindered the development of dynamic recognition technology for large outdoor wind turbine blades. Summary of the Invention

[0006] The purpose of the present invention is to solve the problem of dynamic identification of outdoor large wind turbine blades and propose a wind turbine blade vibration monitoring method based on drones and DIC, which includes the following steps:

[0007] S1. Use a drone to photograph the wind turbine blades along a preset flight path to obtain surface image data of the wind turbine blades;

[0008] S2. Based on the UAV GPS sensor and the image data, the UAV posture is optimized using a graph optimization method for pose estimation in SLAM to obtain a corrected image;

[0009] S3. Based on the corrected image, use the open source stereo speckle image generation algorithm to generate normal and deformed digital speckle images, and divide the normal and deformed digital speckle images into training set, validation set, and test set;

[0010] S4, constructing a three-dimensional digital image correlation measurement network model, the model including three modules: feature extraction, cost volume generation, and displacement map prediction;

[0011] The feature extraction module includes two feature extraction units, which respectively extract the features of the normal digital speckle image and the deformed digital speckle image. The cost volume generation module generates the cost volume based on the features of the normal digital speckle image and the deformed digital speckle image. The cost volume is passed through the displacement map prediction module to obtain the predicted displacement cloud map.

[0012] S5. Use the training set, validation set, and test set to train, validate, and test the model to obtain a final model, input the image to be tested into the final model, perform frequency analysis on the generated displacement cloud map of the wind turbine blade, and determine the current state of the blade.

[0013] Furthermore, the objective function of drone attitude optimization is as follows:

[0014]

[0015] Among them, F(x) represents the objective function of the UAV posture optimization, x represents the position of the UAV, C represents the index pair set between the connection nodes, and T i and T j They represent the posture of the UAV at different times, e ij According to T i and T j Error items in the build, Indicates T i and T j The relative position change, Λ ij represents the information matrix;

[0016] Set the initial point of the optimization problem to And at the initial point Add an increment to the above, and the estimated value between the posture graph state nodes is The error between nodes is The overall error cost function is linearized;

[0017]

[0018] in, represents the estimated value between state nodes, represents the initial point, Δx represents the increment, Represents the error value between nodes, Λ k represents the information matrix, J k represents the Jacobian matrix, C k represents the constant term, b k Represents the linear term of error, H k represents the Hessian matrix.

[0019] Furthermore, the cost volume generation module includes: two multi-scale adaptive patch matching units, an attention unit, two two-dimensional convolutional layers, and a convolutional block; the convolutional block is composed of three two-dimensional convolutional layers stacked together;

[0020] Characteristics of normal digital speckle images and deformed digital speckle images r and f d , f r and f d The correlation volume V is obtained by the first multi-scale adaptive patch matching unit 相关 , f r Get f through the first two-dimensional convolution layer r ′ , f d The volume V is obtained by the second multi-scale adaptive patch matching unit d , f d Through the second two-dimensional convolution layer to get f d′ , f r ′ 、V d and f d ′ Splicing to obtain connector V 连接 , related body V 相关 Get V through the convolution block 相关 ′ , V 相关 ′ and V 连接 The cost body is obtained through the attention unit.

[0021] Furthermore, the feature extraction unit includes six two-dimensional convolutional layers, two two-dimensional deconvolutional layers, the first and second two-dimensional convolutional layers are stacked as a first convolutional block, the third two-dimensional convolutional layer and the first two-dimensional deconvolutional layer are stacked as a second convolutional block, the fourth and fifth two-dimensional convolutional layers are stacked as a third convolutional block, the sixth two-dimensional convolutional layer and the second two-dimensional deconvolutional layer are stacked as a fourth convolutional block, the four convolutional blocks are connected in series, and the output of the first convolutional block is connected to the output of the second convolutional block and the output of the fourth convolutional block through a jump connection.

[0022] Furthermore, the feature extraction unit includes two 1×1 convolutional layers and four 3×3 convolutional layers in series. The input of the feature extraction unit passes through the first 1×1 convolutional layer and the first 3×3 convolutional layer respectively. The output of the first 1×1 convolution is spliced ​​with the output of the second 3×3 convolutional layer through a jump connection. The spliced ​​feature map is input into the third 3×3 convolutional layer and the second 1×1 convolutional layer. The output of the second 1×1 convolutional layer is spliced ​​with the output of the fourth 3×3 convolutional layer through a jump connection to obtain the output of the feature extraction unit.

[0023] Furthermore, the multi-scale adaptive patch matching unit first divides the feature map input by the multi-scale adaptive patch matching unit into patches of different scales, and concatenates the matching cost of each patch to obtain the output of the multi-scale adaptive patch matching unit:

[0024]

[0025] Among them, C correlation represents the output of the multi-scale adaptive patch matching unit, Concat{} represents the concatenation operation, represents the matching cost of group 1, represents the matching cost of group 2, represents the matching cost of group 3;

[0026] The matching cost of each group is calculated using the following formula:

[0027]

[0028] in, represents the matching cost of the kth group, N g Indicates the number of feature groups, N f represents the number of feature channels, Ω k Represents a set of nine coordinates that determine the size and expansion ratio of the patch, (i, j) represents the pixel point, w i,j represents the weight of pixel (i, j), f(xi, yj) represents the feature map of the input multi-scale adaptive patch matching unit, and (x, y) represents the position of the pixel.

[0029] Furthermore, the displacement map prediction module includes four two-dimensional convolutional layers and two two-dimensional deconvolutional layers. The first two-dimensional convolutional layer, the second two-dimensional convolutional layer, and the first two-dimensional deconvolutional layer are stacked into a convolution block CB1, and the third two-dimensional convolutional layer, the fourth two-dimensional convolutional layer, and the second two-dimensional deconvolutional layer are stacked into a convolution block CB2. CB1 and CB2 are connected in series.

[0030] The present invention also proposes a wind turbine blade vibration monitoring system based on a drone and a DIC, comprising:

[0031] A data acquisition module is used to use a drone to photograph the wind turbine blades according to a preset flight path to obtain surface image data of the wind turbine blades;

[0032] An image correction module is used to optimize the UAV posture based on the UAV GPS sensor and the image data using a graph optimization method for pose estimation in SLAM to obtain a corrected image;

[0033] A digital speckle image generation module is used to generate normal and deformed digital speckle images based on the corrected image using an open source stereoscopic speckle image generation algorithm, and divide the normal and deformed digital speckle images into a training set, a validation set, and a test set;

[0034] A model building module is used to build a three-dimensional digital image correlation measurement network model, which includes three modules: feature extraction, cost volume generation, and displacement map prediction;

[0035] The feature extraction module includes two feature extraction units, which respectively extract the features of the normal digital speckle image and the deformed digital speckle image. The cost volume generation module generates the cost volume based on the features of the normal digital speckle image and the deformed digital speckle image. The cost volume is passed through the displacement map prediction module to obtain the predicted displacement cloud map.

[0036] A result obtaining module is configured to train, verify and test the model using the training set, the verification set and the test set, obtain a final model, input the to-be-tested image into the final model, perform frequency analysis on a generated displacement cloud map of the wind turbine blade, and determine a current state of the blade.

[0037] The technical solution provided by the present application has the following beneficial effects:

[0038] The present application uses a UAV to shoot pictures of a wind turbine blade, optimizes the attitude of the UAV based on a sensor fusion algorithm of attitude graph optimization, overcomes the limitations of an outdoor large wind turbine blade dynamic scene or a long-distance target, and ensures the quality of the image. A three-dimensional digital image correlation measurement network model is constructed to predict the displacement amount, including three modules of feature extraction, cost volume generation and displacement map prediction. The feature extraction module uses residual layers and convolution layers to obtain features with different receptive fields to enhance the robustness of matching features under different scale deformations. The cost volume generation module combines the advantages of connection volume and correlation volume, constructs an attention connection volume, which can provide the possibility of measuring deformation at different scales. The stacked convolution layers of the displacement map prediction module aggregate the similarity information of the cost volume at different scales, and improve the measurement accuracy of deformation at different scales. The problem of outdoor large wind turbine blade dynamic identification is solved. BRIEF DESCRIPTION OF DRAWINGS

[0039] Figure 1 is a flowchart of the wind turbine blade vibration monitoring method based on the UAV and DIC of the embodiment of the present application;

[0040] Figure 2 is a schematic diagram of the UAV data acquisition of the embodiment of the present application;

[0041] Figure 3 is a structure diagram of the three-dimensional digital image correlation measurement network model of the embodiment of the present application;

[0042] Figure 4 is a structure diagram of the feature extraction unit of another preferred embodiment of the embodiment of the present application. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical scheme and advantages of the present application clearer, the embodiments of the present application will be further described below with reference to the drawings.

[0044] The flowchart of the wind turbine blade vibration monitoring method based on the UAV and DIC of the embodiment of the present application is as shown in Figure 1 , and specifically includes the following steps:

[0045] S1, using a UAV to shoot a wind turbine blade according to a preset flight path to obtain wind turbine blade surface image data.

[0046] The UAV takes off from the downwind side of the wind turbine, and flies along the leading edge to the trailing edge of the wind turbine blade according to the preset flight path. During the flight, the UAV equipped with a high-definition camera continuously shoots the blade surface to obtain high-resolution image data. The image data is a group of normal wind turbine blade images and deformed wind turbine blade images at the same position. Reference Figure 2 , Figure 2 is a schematic diagram of UAV data acquisition according to an embodiment of the present application.

[0047] S2, based on the UAV GPS sensor and the wind turbine blade surface image data, the pose of the UAV is optimized using the graph optimization method of SLAM pose estimation, and the corrected image is obtained.

[0048] The absolute position data obtained from the GPS sensor is converted into a coordinate system to obtain the position in the East, North and Up (ENU) coordinate system. The relative position of the UAV is determined with the position at the first time as the origin. The pose graph optimization is a calculation method used to optimize the pose of the robot in SLAM. The pose graph optimization excludes the numerous three-dimensional space points considered in BA optimization, and only focuses on the position and attitude of the UAV. This greatly reduces the optimization scale and avoids unnecessary consumption of computing power. The attitude graph is composed of nodes and edges, wherein: the node corresponds to the pose of the robot in the environment, and is also the optimization target; the edge represents the spatial constraint between different nodes.

[0049] The objective function of the UAV attitude optimization is as follows:

[0050]

[0051] Wherein, F(x) represents the objective function of the UAV attitude optimization, x represents the position of the UAV, C represents the index pair set connecting the nodes, T i and T j represent the poses of the UAV at different times, e ij represents the error term constructed according to T i and T j , represents the relative pose change of T i and T j , Λ ij represents the information matrix, which is the inverse of the covariance, and represents the weight of each error in the optimization function.

[0052] The initial point of the optimization problem is set as and an increment is added to the initial point , the estimated value between the state nodes is , the error value between the nodes is The overall error cost function is linearized.

[0053]

[0054] in, Represents the estimated value between state nodes. represents the initial point, Δx represents the increment, Represents the error value between nodes, Λ k represents the information matrix, J k represents the Jacobian matrix, C k represents the constant term, b k Represents the linear term of error, H k represents the Hessian matrix.

[0055] The final result In, C k is a constant term, we will find At minimum Convert to 2b k Δx+Δx T H k Δx first-order derivative (to find 2b k Δx+Δx T H k When the first derivative of Δx with respect to Δx is 0 value.

[0056] That is: 2b k +2H k Δx=0 to get H k Δx=-b k ,get The optimal solution of .

[0057] S3. Based on the corrected image, an open source stereo speckle image generation algorithm is used to generate normal and deformed digital speckle images, and the normal and deformed digital speckle images are divided into a training set, a validation set, and a test set.

[0058] Specifically, during the dataset generation process, the displacement amplitudes of in-plane deformation and parallax were set to 0.05mm and 0.5mm, respectively. The maximum in-plane deformation in the x and y directions was 1 pixel and 5 pixels, respectively, and the parallax was approximately 10 pixels. A total of 750 sets of data were generated, each containing 4 training samples, of which 67% were used as training sets, 22% as validation sets, and 11% as test sets.

[0059] S4. Construct a three-dimensional digital image correlation measurement network model, which includes three modules: feature extraction, cost volume generation, and displacement map prediction.

[0060] The feature extraction module includes two feature extraction units. The cost volume generation module consists of two multi-scale adaptive patch matching units, an attention unit, two 2D convolutional layers, and a convolutional block. The convolutional block is composed of three stacked 2D convolutional layers. The stride of the two 2D convolutional layers defaults to 1, and the strides of the three stacked 2D convolutional layers are 2, 3, and 2, respectively.

[0061] The structural diagram of the stereo digital image correlation measurement network model of the embodiment of the present invention is shown in FIG. Figure 3 The two feature extraction units extract the features of normal digital speckle image and deformed digital speckle image respectively. r and f d , f r and f d The correlation volume V is obtained by the first multi-scale adaptive patch matching unit 相关 , f r Get f through the first two-dimensional convolution layer r ′ , f d The correlation volume V is obtained by the second multi-scale adaptive patch matching unit d , f d Through the second two-dimensional convolution layer to get f d ′ , f r ′ 、V d and f d ′ Splicing to obtain connector V 连接 , related body V 相关 Get V through the convolution layer 相关 ′ , V 相关 ′ and V 连接 The attention unit is used to obtain the attention connectome, i.e., the cost body. The attention unit adopts the channel and spatial attention module (CBAM). The attention connectome passes through the displacement map prediction module to obtain the predicted displacement map.

[0062] Specifically, feature f r and f d is a 320-channel feature map, f r ′ and f d ′ It is a 32-channel feature map.

[0063] In a preferred embodiment of the present invention, referring to Figure 3The feature extraction unit in

[15] consists of six 2D convolutional layers and two 2D deconvolutional layers. The first and second 2D convolutional layers are stacked as the first convolutional block, the third 2D convolutional layer and the first 2D deconvolutional layer are stacked as the second convolutional block, the fourth and fifth 2D convolutional layers are stacked as the third convolutional block, and the sixth 2D convolutional layer and the second 2D deconvolutional layer are stacked as the fourth convolutional block. The four convolutional blocks are connected in series, and the output of the first convolutional block is connected to the output of the second and fourth convolutional blocks via shotcut jumps. The stride of the first, second, fourth, and fifth 2D convolutional layers is 2, and the stride of the third and sixth 2D convolutional layers is 1 by default.

[0064] In another preferred embodiment of the present invention, referring to Figure 4 , Figure 4 This is a structural diagram of a feature extraction unit of another preferred embodiment of the present invention. The feature extraction unit includes two 1×1 convolutional layers and four 3×3 convolutional layers in series. The input of the feature extraction unit passes through the first 1×1 convolutional layer and the first 3×3 convolutional layer respectively. The output of the first 1×1 convolution is spliced ​​with the output of the second 3×3 convolutional layer via a jump connection. The spliced ​​feature map is input to the third 3×3 convolutional layer and the second 1×1 convolutional layer. The output of the second 1×1 convolutional layer is spliced ​​with the output of the fourth 3×3 convolutional layer via a jump connection to obtain the output of the feature extraction unit. Among them, the first and third 3×3 convolutional layers include a 3×3 convolution and a ReLU activation function, the second and fourth 3×3 convolutional layers include a 3×3 convolution and a BN layer (Batch Normalization), and the two 1×1 convolutional layers each include a 1×1 convolution and a BN layer.

[0065] The feature extraction unit uses two residual layers to obtain features with different receptive fields to enhance the robustness of matching features under deformations of different scales.

[0066] The multi-scale adaptive patch matching unit first divides the feature map input by the multi-scale adaptive patch matching unit into patches of different scales, and concatenates the matching cost of each patch to obtain the output of the multi-scale adaptive patch matching unit:

[0067]

[0068] Among them, C correlation represents the output of the multi-scale adaptive patch matching unit, Concat{} represents the concatenation operation, represents the matching cost of group 1, represents the matching cost of group 2, represents the matching cost of group 3.

[0069] The matching cost of each group is calculated using the following formula:

[0070]

[0071] in, represents the matching cost of the kth group, N g Indicates the number of feature groups, N f represents the number of feature channels, Ω k Represents a set of nine coordinates that determine the size and expansion ratio of the patch, (i, j) represents the pixel point, w i,j Represents the weight of pixel (i, j), which is also the training parameter of the network, f(xi, yj) represents the feature map of the input multi-scale adaptive patch matching unit, and (x, y) represents the position of the pixel.

[0072] The displacement map prediction module includes four two-dimensional convolutional layers and two two-dimensional deconvolutional layers. The first two-dimensional convolutional layer, the second two-dimensional convolutional layer, and the first two-dimensional deconvolutional layer are stacked into a convolution block CB1, and the third two-dimensional convolutional layer, the fourth two-dimensional convolutional layer, and the second two-dimensional deconvolutional layer are stacked into a convolution block CB2. CB1 and CB2 are connected in series.

[0073] S5. Use the training set, validation set, and test set to train, validate, and test the above-mentioned stereo digital image correlation measurement network model to obtain a final model. Input the image to be measured into the final model to generate a displacement cloud map of the wind turbine blade. Perform frequency analysis on the displacement cloud map to determine the current state of the blade.

[0074] In an exemplary embodiment, a wind turbine blade vibration monitoring system based on a drone and a DIC is provided, comprising:

[0075] A data acquisition module is used to use a drone to photograph the wind turbine blades according to a preset flight path to obtain surface image data of the wind turbine blades;

[0076] The image correction module is used to optimize the UAV's posture based on the UAV GPS sensor and the wind turbine blade surface image data, using the graph optimization method of pose estimation in SLAM to obtain the corrected image;

[0077] A digital speckle image generation module is used to generate normal and deformed digital speckle images based on the corrected image using an open source stereoscopic speckle image generation algorithm, and divide the normal and deformed digital speckle images into a training set, a validation set, and a test set;

[0078] A model building module is used to build a three-dimensional digital image correlation measurement network model, which includes three modules: feature extraction, cost volume generation, and displacement map prediction;

[0079] The feature extraction module includes two feature extraction units, which respectively extract the features of the normal digital speckle image and the deformed digital speckle image. The cost volume generation module generates the cost volume based on the features of the normal digital speckle image and the deformed digital speckle image. The cost volume is passed through the displacement map prediction module to obtain the predicted displacement cloud map.

[0080] The result acquisition module is used to train, verify and test the model using the training set, verification set and test set to obtain the final model, input the image to be tested into the final model, perform frequency analysis on the generated displacement cloud map of the wind turbine blade, and determine the current state of the blade.

[0081] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A wind turbine blade vibration monitoring method based on drone and DIC, characterized in that: The following steps are involved: S1. Use a drone to photograph the wind turbine blades along a preset flight path to obtain surface image data of the wind turbine blades; S2. Based on the UAV GPS sensor and the image data, the UAV posture is optimized using a graph optimization method for pose estimation in SLAM to obtain a corrected image; S3. Based on the corrected image, use the open source stereo speckle image generation algorithm to generate normal and deformed digital speckle images, and divide the normal and deformed digital speckle images into training set, validation set, and test set; S4, constructing a three-dimensional digital image correlation measurement network model, the model including three modules: feature extraction, cost volume generation, and displacement map prediction; The feature extraction module includes two feature extraction units, which respectively extract the features of the normal digital speckle image and the deformed digital speckle image. The cost volume generation module generates the cost volume based on the features of the normal digital speckle image and the deformed digital speckle image. The cost volume is passed through the displacement map prediction module to obtain the predicted displacement cloud map. S5. Use the training set, validation set, and test set to train, validate, and test the model to obtain a final model, input the image to be tested into the final model, perform frequency analysis on the generated displacement cloud map of the wind turbine blade, and determine the current state of the blade.

2. The wind turbine blade vibration monitoring method based on drone and DIC according to claim 1 is characterized in that: The objective function of drone attitude optimization is as follows: Among them, F(x) represents the objective function of the UAV posture optimization, x represents the position of the UAV, C represents the index pair set between the connection nodes, and T i and T j They represent the posture of the UAV at different times, e ij According to T i and T j Error items in the build, Indicates T i and T j The relative position change of ij represents the information matrix; Set the initial point of the optimization problem to And at the initial point Add an increment to the above, and the estimated value between the posture graph state nodes is The error between nodes is The overall error cost function is linearized; in, represents the estimated value between state nodes, represents the initial point, Δx represents the increment, Represents the error value between nodes, Λ k represents the information matrix, J k represents the Jacobian matrix, C k represents the constant term, b k Represents the linear term of error, H k represents the Hessian matrix.

3. The wind turbine blade vibration monitoring method based on drone and DIC according to claim 1 is characterized in that: The cost volume generation module includes: two multi-scale adaptive patch matching units, an attention unit, two 2D convolutional layers, and a convolutional block; the convolutional block is composed of three stacked 2D convolutional layers; Characteristics of normal digital speckle images and deformed digital speckle images r and f d The correlation volume V is obtained by the first multi-scale adaptive patch matching unit 相关 , f r Get f through the first two-dimensional convolution layer r ′ , f d The volume V is obtained by the second multi-scale adaptive patch matching unit d , f d Through the second two-dimensional convolution layer to get f d ′ , f r ′ 、V d and f d ′ Splicing to obtain connector V 连接 , related body V 相关 Get V through the convolution block 相关 ′ , V 相关 ′ and V 连接 The cost body is obtained through the attention unit.

4. The wind turbine blade vibration monitoring method based on drone and DIC according to claim 1 is characterized in that: The feature extraction unit includes six two-dimensional convolutional layers, two two-dimensional deconvolutional layers, the first and second two-dimensional convolutional layers are stacked as the first convolutional block, the third two-dimensional convolutional layer and the first two-dimensional deconvolutional layer are stacked as the second convolutional block, the fourth and fifth two-dimensional convolutional layers are stacked as the third convolutional block, the sixth two-dimensional convolutional layer and the second two-dimensional deconvolutional layer are stacked as the fourth convolutional block, the four convolutional blocks are connected in series, and the output of the first convolutional block is connected to the output of the second convolutional block and the output of the fourth convolutional block through jump connection.

5. The wind turbine blade vibration monitoring method based on drone and DIC according to claim 1 is characterized in that: The feature extraction unit consists of two 1×1 convolutional layers and four 3×3 convolutional layers in series. The input of the feature extraction unit passes through the first 1×1 convolutional layer and the first 3×3 convolutional layer respectively. The output of the first 1×1 convolution is spliced ​​with the output of the second 3×3 convolutional layer through a jump connection. The spliced ​​feature map is input into the third 3×3 convolutional layer and the second 1×1 convolutional layer. The output of the second 1×1 convolutional layer is spliced ​​with the output of the fourth 3×3 convolutional layer through a jump connection to obtain the output of the feature extraction unit.

6. The wind turbine blade vibration monitoring method based on drone and DIC according to claim 1 is characterized in that: The multi-scale adaptive patch matching unit first divides the feature map input by the multi-scale adaptive patch matching unit into patches of different scales, and concatenates the matching cost of each patch to obtain the output of the multi-scale adaptive patch matching unit: Among them, C correlation represents the output of the multi-scale adaptive patch matching unit, Concat{} represents the concatenation operation, represents the matching cost of group 1, represents the matching cost of group 2, represents the matching cost of group 3; The matching cost of each group is calculated using the following formula: in, represents the matching cost of the kth group, N g Indicates the number of feature groups, N f represents the number of feature channels, Ω k Represents a set of nine coordinates that determine the size and expansion ratio of the patch, (i, j) represents the pixel point, w i,j represents the weight of pixel (i, j), f(xi, yj) represents the feature map of the input multi-scale adaptive patch matching unit, and (x, y) represents the position of the pixel.

7. The wind turbine blade vibration monitoring method based on drone and DIC according to claim 1 is characterized in that: The displacement map prediction module includes four two-dimensional convolutional layers and two two-dimensional deconvolutional layers. The first two-dimensional convolutional layer, the second two-dimensional convolutional layer, and the first two-dimensional deconvolutional layer are stacked into a convolution block CB1, and the third two-dimensional convolutional layer, the fourth two-dimensional convolutional layer, and the second two-dimensional deconvolutional layer are stacked into a convolution block CB2. CB1 and CB2 are connected in series.

8. A wind turbine blade vibration monitoring system based on drone and DIC, characterized in that: include: A data acquisition module is used to use a drone to photograph the wind turbine blades according to a preset flight path to obtain surface image data of the wind turbine blades; An image correction module is used to optimize the UAV posture based on the UAV GPS sensor and the image data using a graph optimization method for pose estimation in SLAM to obtain a corrected image; A digital speckle image generation module is used to generate normal and deformed digital speckle images based on the corrected image using an open source stereoscopic speckle image generation algorithm, and divide the normal and deformed digital speckle images into a training set, a validation set, and a test set; A model building module is used to build a three-dimensional digital image correlation measurement network model, which includes three modules: feature extraction, cost volume generation, and displacement map prediction; The feature extraction module includes two feature extraction units, which respectively extract the features of the normal digital speckle image and the deformed digital speckle image. The cost volume generation module generates the cost volume based on the features of the normal digital speckle image and the deformed digital speckle image. The cost volume is passed through the displacement map prediction module to obtain the predicted displacement cloud map. The result acquisition module is used to train, verify and test the model using the training set, verification set and test set to obtain the final model, input the image to be tested into the final model, perform frequency analysis on the generated displacement cloud map of the wind turbine blade, and determine the current state of the blade.

Images