A method for detecting defects in offshore wind turbine blades

Abstract

This invention discloses a method for detecting defects in offshore wind turbine blades, belonging to the field of wind turbine blade inspection technology. The detection method includes the following steps: S01: Offshore wind turbine blade defect detection: Identifying the location of blade defects, outputting a location marker box for the defect on the original image, and providing the coordinates of the center position of the marker box and its corresponding length and width; S02: Blade image stitching: Through step-by-step shooting; using image stitching technology, stitching together each detected image to form a complete wind turbine blade image; S03: Determining the actual physical location of blade defects in a single image: In the stitched overall image, determining the actual physical coordinates of the blade defects in the original single images corresponding to the wind turbine blade. This invention's detection method integrates multiple detection techniques such as computer vision, feature extraction, image stitching, and edge detection, enabling comprehensive acquisition of blade surface defect information and significantly improving detection accuracy and precision.

Description

Technical Field

[0001] This invention belongs to the field of wind turbine blade inspection technology, and particularly relates to a method for detecting defects in offshore wind turbine blades. Background Technology

[0002] In recent years, my country's wind power industry has been booming. As the scale of wind power continues to expand, the demand for operation and maintenance is also increasing.

[0003] In wind turbine units, blades play a crucial role. However, during long-term operation, various defects easily appear on the blade surface. To prevent these defects from developing into irreversible damage, timely blade maintenance is essential. Compared to onshore wind power, offshore wind power faces more severe challenges, such as salt spray corrosion, typhoons, and lightning strikes, leading to more frequent blade surface defects and more challenging maintenance tasks. Currently, offshore wind turbine blade inspection still relies primarily on traditional manual checks, with specialized blade inspection systems being particularly scarce. This makes large-scale aerial operations difficult, resulting in numerous blind spots and extremely poor accessibility. Conventional target detection algorithms suffer from low accuracy, high latency, limited classification capabilities, and insensitivity to small target defects. Therefore, achieving efficient real-time detection and safety early warning of wind turbine blades in marine environments has become an urgent problem to solve. Summary of the Invention

[0004] This invention provides a method for detecting defects in offshore wind turbine blades, aiming to address the problems of low accuracy, high latency, limited classification, and insensitivity to small defects encountered when using conventional target detection algorithms. The method employs a DW feature extraction network structure. Within this architecture, a C2f_CBAM (C2f_CBAM: C2f_Convolutional Block Attention Module) attention module is incorporated, enabling the network to not only focus on information interaction between channels but also to delve deeper into the distribution of features in the image space, thereby more accurately capturing the local details and global information of blade defects. Furthermore, the concept of grid segmentation is introduced to classify and locate defects in the stitched image, improving the accuracy and efficiency of the entire recognition process.

[0005] The technical solution of this invention is implemented as follows:

[0006] A method for detecting defects in offshore wind turbine blades includes the following steps:

[0007] S01: Offshore wind turbine blade defect detection: Identify the location of blade defects, output the defect location marker box on the original image, and provide the coordinates of the center position of the marker box and its corresponding length and width;

[0008] S02: Blade Image Stitching: By taking pictures of different parts of the blade step by step, the image data of the blade is gradually accumulated; with the help of image stitching technology, each detected image is stitched together to form a complete image of the wind turbine blade.

[0009] S03: Determine the actual physical location of blade defects in a single image: In the stitched overall image, determine the actual physical coordinates of the blade defects in the original single image corresponding to the wind turbine blades.

[0010] Preferably, in step S01:

[0011] The location of blade defects is identified using a DW feature extraction network structure, which incorporates a C2f_CBAM module.

[0012] The C2f_CBAM module adds the original image features to the enhanced features through residual connections to ensure feature continuity. Since offshore wind turbine blades often have small defects, the C2f_CBAM module introduces a BEAM module to enhance the DW network's ability to perceive important features, further suppress useless and redundant information in the original image, and further improve the network's accuracy in detecting small blade defects.

[0013] The BEAM module consists of multiple BottleNeck_CBAM modules.

[0014] The process of outputting defect location bounding boxes on the original image and providing the coordinates of the center position of the bounding box and its corresponding length and width is as follows: The input original image is divided into n×n small grid images. Each small grid image is input into the DW network structure. Each small grid predicts S anchors with a predetermined aspect ratio (the number of anchor boxes used in each grid unit). The center of the bounding box must be within the corresponding network region. Through the DW network structure, the length, width, and center point coordinates of the predicted bounding box are output. Simultaneously, the classifier in the network structure outputs the probability of predicting each category. Scores are set as the confidence score for each predicted box. The formula for calculating Scores is shown in Equation I.

[0015]

[0016] Where, P(Class) iThe value represents the probability of prediction for different blade defect categories. PIoU is an improvement on IoU (Intersection over Union) and is used to evaluate the degree of overlap between the predicted bounding box and the true bounding box, thereby accurately measuring the accuracy of localization. A higher PIoU value means a larger overlap area between the predicted and true bounding boxes, indicating higher accuracy in the prediction results. The formula for calculating PIoU is shown in Equation II:

[0017]

[0018] Where A represents the predicted bounding box, B represents the ground truth bounding box, and i takes values ​​from 1 to 4. This represents the distance from the midpoint of the line connecting the predicted bounding box and the ground truth bounding box to the coordinates of each point in the predicted bounding box. This represents the distance from the midpoint of the line connecting the predicted bounding box and the ground truth bounding box to the coordinates of each point in the ground truth bounding box. This indicates taking the maximum distance.

[0019] The improved PIOU loss function enhances the model's convergence efficiency. By combining the corresponding classification probabilities, the parameters in the classifier can be better adjusted, thereby generating predicted boxes that are closer to the true bounding boxes and further improving the accuracy of defect localization.

[0020] Since each grid generates S bounding boxes, there may be cases where a single target has multiple bounding boxes. Therefore, it is necessary to reasonably filter the bounding boxes. Preferably, the filtering of multiple bounding boxes specifically includes: 1) discarding bounding box confidence scores below a threshold, and 2) using matrix nonmaximum suppression (Matrix NMS) to remove redundant bounding boxes for each class, until all redundant bounding boxes have been processed, at which point the entire process ends.

[0021] Thus far, the model has demonstrated good performance in detecting defects in offshore wind turbine blades, accurately predicting the location and corresponding category of blade defects, and exhibiting low false detection and false negative rates.

[0022] In actual offshore wind turbine blade defect identification tasks, due to the large area of ​​the blades, a single drone shot is insufficient to cover the entire blade. Therefore, step S02 adopts a step-by-step shooting strategy, gradually accumulating image data of the blade by capturing different parts of it one by one; then, image stitching technology is used to fuse the scattered image fragments together, ultimately constructing a complete image of the wind turbine blade for further defect analysis and identification.

[0023] Preferably, in step S02,

[0024] The image stitching technique is used to stitch together each detected image. The specific steps are as follows:

[0025] S21. Extract features from the original single image and obtain feature point information for each image through supervised learning to facilitate subsequent image stitching operations;

[0026] S22. Based on the feature points matched in step S21, estimate the geometric transformation relationship between different images and complete the stitching.

[0027] Preferably, in step S21,

[0028] The feature extraction of the original single image adopts the QX network structure, utilizes FRConv convolutional layers to reduce the number of model parameters, introduces the BEAM module, and uses the idea of ​​residual connections to ensure the performance of deep learning. It focuses on inter-channel information and emphasizes the distribution of features in the image space, thereby capturing more local and global information about blade defects.

[0029] By utilizing FRConv convolutional layers, the accuracy of extracting blade defect features is improved while reducing the number of model parameters, and the robustness of the model to input images is enhanced. The BEAM module is introduced, employing the idea of ​​residual connections to ensure the performance of deep learning, focusing on inter-channel information and emphasizing the distribution of features in the image space, thereby capturing more local and global information about blade defects. Through the assembly of modules, a QX network structure is formed. This structure can deeply learn and extract effective information closely related to blade defects, identify feature points, and assign descriptive information to these feature points, forming a series of feature vectors, providing strong support for the accurate identification and localization of blade defects.

[0030] To extract more comprehensive feature information from offshore wind turbine blades, the FRConv module is used. By integrating contextual and regional feature information, it helps the network better understand the relationship between defects and the surrounding environment, thus improving the network's accuracy in extracting blade defect features. The FRConv module consists of a regional feature extractor, a contextual feature extractor, and an integrated feature extractor. The regional and contextual feature extractors are connected to the integrated feature extractor.

[0031] The FRConv module performs feature extraction, including the following steps: the region feature extractor is a 3×3 standard convolutional layer (Conv) that learns local features from the eight surrounding feature vectors; the context feature extractor is a 3×3 dilated convolution (AConv), which has a relatively large receptive field and can effectively learn surrounding contextual information; the ensemble feature extractor concatenates the output features of the region feature extractor and the context feature extractor (Concat), and processes them through a batch normalization (BatchNorm2d) function and a FReLU activation function (BN+FReLU), finally inputting the extracted feature map. The overall output of the FRConv module is represented as follows:

[0032] FRConv(x)=FReLU(BatchNorm2d(Concat(Conv 3×3 (x),AConv 3×3 (x))

[0033] Where FRConv is the output of the FRConv module, FReLU is the FReLU activation function, BatchNorm2d is the batch normalization BatchNorm2d function, Concat is the connection, and Conv... 3×3 AConv is a standard 3×3 convolutional layer. 3×3 It is a 3×3 dilated convolution, where x is the input feature map.

[0034] The BEAM module adds the original image information to the image information enhanced by the CBAM module through residual connection to ensure the continuity of features and the flow of information, thereby enhancing the perception and learning ability of blade defect features.

[0035] The CBAM module employs two module components: the Channel Attention Module (CAM) and the Spatial Attention Module (SAM).

[0036] The operation flow of the CBAM module is as follows: input the original image, multiply the output of the channel attention module (CAM) with the original image to extract features; at the same time, use the output of the channel attention module (CAM) as input to enter the two-dimensional convolution of the spatial attention module (SAM), and then multiply the output with the original image to enhance the model's ability to perceive features, so as to better identify defects in small targets.

[0037] The Channel Attention Module (CAM) focuses on meaningful information in the input image while compressing spatial dimensions, keeping the channel dimension unchanged, and enhancing the feature representation of each channel. The input feature map is passed through two parallel max-pooling layers (MaxPool) and a global average pooling layer (AvgPool), reducing the feature map from C*H*W to C*1*1. Then, it passes through a Shared Fully Connected Layer (Share MLP) module. In the Shared MLP module, the number of channels is first compressed to 1 / r of the original number, then expanded to the original number. The FReLU activation function is then applied to obtain two activated results. These two outputs are added element-wise, and then the channel attention weight α is obtained through a Sigmoid activation function. Finally, this output is multiplied by each channel of the original feature map, reducing it back to C*H*W.

[0038] The calculation formula in the Shared Fully Connected Layer (Share MLP) module is shown in Equation III:

[0039]

[0040] Where x is the input feature map, This represents a 1x1 convolution operation with the following output channels: Where C is the number of channels in the input feature map, and r is a reduction ratio used to reduce parameters and computational cost.

[0041] The formula for calculating the channel attention weight α is shown in Equation IV:

[0042] α = Sigmoid(x) Sum );

[0043] Formula IV;

[0044] Where α is the channel attention weight, Sigmoid is the Sigmoid activation function, and x Sum This is the result of adding the two feature maps element-wise after activation by the FReLU activation function.

[0045] The Spatial Attention Module (SAM) focuses on more target location information by compressing the channel dimension while maintaining the spatial dimension. The SAM takes the output of the Channel Attention Module (CAM) as its input feature, and obtains two 1*H*W feature maps through max pooling and average pooling. These two feature maps are then concatenated using a concat operation to obtain a 2*H*W feature map x. ConcatThe feature map is processed by a convolutional layer (Conv layer), and then a sigmoid activation function is used to obtain the feature map of spatial attention. Finally, the output is multiplied by the original image to restore it to the size of C*H*W, and the features at each spatial location are weighted.

[0046] The feature map x Concat The formula is shown in equation V:

[0047] x Concat =Concat(x) MaxPool ,x AvgPool );

[0048] Formula V;

[0049] The BEAM model structure adds the original image information to the enhanced image information through residual connections to ensure the continuity of features and the flow of information, thereby enhancing the perception and learning ability of blade defect features. A Fosine similarity method is established to calculate the similarity between two images. Based on the calculation results, feature points are matched, and the prediction accuracy is evaluated using the feature point detection network loss function.

[0050] The formula for calculating the Fosine similarity is shown in Equation VI:

[0051]

[0052] a and b are two-dimensional feature vectors of two images, respectively.

[0053] The loss function of the feature point detection network is shown in Equation VII:

[0054] Loss=1-Fosine_Similarity(FRConv(prea),Conv(t));

[0055] Equation VII;

[0056] prea represents the feature points of the predicted value, and t represents the labeled feature points.

[0057] The higher the value of the loss function of the feature point detection network, the more inaccurate the prediction; the lower the value of the loss function, the closer the prediction is to the true value.

[0058] Preferably, in step S22, estimating the geometric transformation relationship between different images specifically involves: estimating transformation parameters based on minimizing the reprojection error, thereby determining the positional relationship between matching feature points, and fitting translation, rotation, and scaling transformations through least squares reprojection to provide support for subsequent image stitching.

[0059] The geometric transformation based on minimizing reprojection error includes the following steps:

[0060] The model takes the feature vectors of any feature point matching pair in the matching image as input, and aims to find a geometric transformation model that makes one feature vector in the matching pair as close as possible to another feature vector after transformation, so as to better determine the transformation relationship. An affine transformation model is used, which includes three transformation parameters: translation, rotation and scaling. The reprojection error is used as the objective function to measure the error between the model's predicted value and the actual value.

[0061] The affine transformation model is shown in Equation VIII:

[0062]

[0063] Among them, (x i ,y i (x) represents a feature vector in a matching pair. i ′,y i ′) represents the eigenvector after geometric transformation, e, f, g, and h represent the rotation and scaling parameters, respectively, and p and q represent the translation parameters.

[0064] The reprojection error, used as the objective function, measures the error between the model's predicted value and the actual value. For a two-dimensional image, assuming there are N matching pairs, the i-th point on the source image is (x... i ,y i The corresponding point on the target image is (x i ′,y i The reprojection error is used to minimize the objective function through gradient descent to solve for the optimal parameters of the corresponding geometric transformation, thereby estimating the geometric transformation relationship between two images. Based on the estimated geometric transformation relationship, the different images are stitched together. For each pixel in the overlapping area, the weighting coefficient is calculated based on the gradient difference of its surrounding pixels. The gradient of the image is calculated using a two-dimensional second-order difference method.

[0065] The formula for calculating the reprojection error is shown in Equation IX:

[0066]

[0067] The operation of stitching different images based on the estimated geometric transformation relationship specifically involves: using the geometric transformation relationship between images to perform geometric transformation on the images, and then combining the relevant results of feature matching to identify the overlapping areas between images, and achieving seamless stitching by merging the pixels in the overlapping areas.

[0068] For each pixel within the overlapping region, a weighting coefficient is calculated based on the gradient difference between its surrounding pixels. Specifically, assuming the gradients of the two images are G1 and G2 respectively, the weighted fusion formula Blend(x,y) for each pixel within the overlapping region is as described in equation X:

[0069]

[0070] Where I1(x,y) and I2(x,y) are the pixel values ​​of the two images in the overlapping region, and G1(x,y) and G2(x,y) are the gradient values ​​of the two images in the overlapping region.

[0071] The gradient calculation of the image is performed using a two-dimensional second-order difference method. By adding the second-order differences in the x and y directions, the gradient image is obtained. The weighted fusion formula and the difference method are applied to each pixel in the overlapping area to effectively eliminate the discontinuity at the image stitching boundary, thereby achieving a smooth transition and continuity of the image.

[0072] The formula for the two-dimensional second-order difference method is shown in Equation XI:

[0073]

[0074] The above operation process is performed on all input images. At this point, a relatively complete and clear image of the blade is output. The entire operation is closely connected and shows a good image stitching effect, laying a solid foundation for the subsequent identification and localization of blade defects.

[0075] Preferably, in step S03:

[0076] The process of determining the actual physical coordinates of the blade defect in the original single image corresponding to the wind turbine blade is as follows: the defect is physically located on the stitched image, and the defect coordinates are determined with the lower left corner of the wind turbine blade as the origin; before the location, the invalid area of ​​the image needs to be cropped to remove the non-wind turbine area, and the lower left corner of the removed image is taken as the origin of the coordinate system.

[0077] In the process of determining the defect coordinates, a search algorithm based on 8-neighborhood relationship is used to extract the pixel position of each edge and its extension order on the edge. This algorithm can overcome the problems of pixel stacking or discontinuity.

[0078] In a binary image F(i,j), non-zero points represent edges. Ideally, these non-zero points should be continuous and distributed as single pixels in the image space. However, some edges may exhibit pixel stacking or pixel discontinuity.

[0079] The search algorithm based on 8-neighborhood relationships designs an edge detection operator Q(i,j) within the 8-neighborhood of a pixel. Its functions are: first, to determine whether the current pixel is an edge point based on the Q(i,j) value; and second, to determine the coordinates of the next edge point. The specific application method of the edge detection operator Q(i,j) is shown in Equation XII:

[0080]

[0081] Here, the gray value of edge points is 1, the gray value of background points is 0, and m and n are the indices used to traverse the 8 neighboring pixels of the current pixel (i,j), which can be -1, 0, or 1.

[0082] Using an 8-neighborhood search algorithm, the upper and lower boundaries of the wind turbine blades are found. This involves identifying the maximum (or minimum) Y value on the left and the minimum (or maximum) Y value on the right, i.e., the two points at the outermost boundaries on both sides. The equation of the upper boundary line YS is determined by these two points as YS = KS*X1 + BS. Similarly, the two points at the lower boundary line are found using the search algorithm, and the equation of the lower boundary line YU is determined as YU = KU*X2 + BU.

[0083] Find the lowest point of the lower boundary line, and then correct the original equations of the upper and lower boundaries so that the lowest point of the lower boundary line is the origin. The corrected equation of the upper boundary line is YS=KS*X1+BS-C1, and the equation of the lower boundary line is YU=KU*X2+BU-C2.

[0084] Where YS is the Y-coordinate of any point on the upper boundary line, KS is the slope of the upper boundary line, X1 is the X-coordinate of any point on the upper boundary line, BS is the intercept of the upper boundary line, and C1 is a constant subtracted from the equation of the upper boundary line.

[0085] Similarly, YU is the Y-coordinate of any point on the lower boundary line, KU is the slope of the lower boundary line, X2 is the X-coordinate of any point on the lower boundary line, BU is the intercept of the lower boundary line, and C2 is a constant subtracted from the equation of the lower boundary line.

[0086] To smooth image features, better handle edge detection, and avoid insufficient data points, the IGP model is used to optimize the boundary equation. Taking the upper boundary line equation as an example, it is assumed that KS follows KS~N(μ0,∑0), and BS follows BS~N(0,σ). 2 I), where μ0 is the mean vector of KS, ∑0 is the prior covariance matrix, and σ 2 I is the variance of the error term, and I is the identity matrix.

[0087] The posterior distribution of KS is calculated using the formula shown in Equation XIII:

[0088] p(KS|X,YS)∝p(YS|X,KS)p(KS);

[0089] Formula XIII;

[0090] Where KS is the slope of the upper boundary line, X is the X coordinate of any point on the upper boundary line, and YS is the Y coordinate of any point on the upper boundary line.

[0091] The mean and covariance of YS are calculated using the formula shown in Equation XIV:

[0092] μ post =∑0M T (M∑0M T +σ 2 I) -1 YS;

[0093] ∑ post =∑0-∑0M T (M∑0M T +σ 2 I) -1 M∑0;

[0094] Formula XIV;

[0095] Where, μ post Let YS be the mean vector, ∑ post Let YS be the covariance matrix, ∑0 be the prior covariance matrix; M is a matrix, each row representing the X-coordinate of any point on the upper bound, and T is the matrix transpose sign; σ 2 I is the variance of the error term, and I is the identity matrix.

[0096] It can be implemented given a new input x * Predicted output y * The formula is shown in equation XV:

[0097] p(y * |x * ,X,YS)=∫p(y * |x * ,KS)p(KS|X,YS)dKS;

[0098] Formula XV;

[0099] Where X is the X-coordinate of any point on the upper boundary line, YS is the Y-coordinate of any point on the upper boundary line, and KS is the slope of the upper boundary line.

[0100] Based on the above operations, the final original image will output the location bounding boxes of defects, including the coordinates of the center position of the bounding box and its corresponding length and width. After the image stitching operation, the positions of the stitching lines of different images in the original image can be recorded (the leftmost position is 0), and the order of the images is known. Combining the above two pieces of information, the position information of each stitching line in the entire image can be calculated. By performing certain digital transformations based on the positional distribution of the stitching lines, the position of the leaf defect corresponding to the original image before stitching can be obtained.

[0101] The center coordinates (x, y) of the new image defect localization marker box are known. i ,y i Given a length w and a width h, the coordinates of the top right, bottom right, top left, and bottom left corners of the marker frame can be easily calculated. The coordinates of the defect are represented by the coordinates of the midpoint of the marker frame, i.e., X. q =(x i +w) / 2,Y q =(y i +h) / 2.

[0102] Based on the upper and lower boundaries, first determine the defect coordinate Y. q Perform a relative transformation, if Y q If the defect is detected above the upper boundary line or below the lower boundary line, it is considered abnormal and discarded. If Y q Between the upper and lower boundaries, a transformation is performed, i.e., Y. q The Manhattan distance to the origin of the lower boundary, i.e.

[0103] For the defect coordinate X q The transformation takes into account the horizontal overlap that occurs during image stitching. In this graphic stitching algorithm, the search range of keypoints can be dynamically adjusted to find the optimal matching point pairs. The FRConv module and Fosine_Similarity are used to find corresponding keypoints in two images, and the width of the overlapping portion is determined by calculating the distance between the keypoints. The n closest keypoints are selected, and the Manhattan average distance D between them is calculated. i D i =SUM(n i xn j x) / n,n i x and n j x represents n i n j The x-coordinate. Obtain the D-coordinate between adjacent images. i Each sub-image has a width of w. i Then the horizontal position L of the dividing line of the i-th sub-image in the stitched image is... i Represented as:

[0104] Then the transformed X qe =L i -D.

[0105] At this point, the physical localization step of the blade defect in a single image is completed. The above-mentioned network structure has high defect recognition accuracy. At the same time, the calculation process of obtaining the defect coordinate information of a single image from the defect information of the whole image is simple and fast, and it can effectively select defect information with a high accuracy rate.

[0106] Compared with the prior art, the technical solution of the present invention has the following advantages:

[0107] 1. The detection method of the present invention realizes fully automatic intelligent detection of defects in offshore wind turbine blades, which greatly improves detection efficiency and coverage, and helps to detect and deal with hidden dangers in a timely manner, and avoid the expansion of faults.

[0108] 2. The detection method of the present invention integrates multiple detection techniques such as computer vision, feature extraction, image stitching, and edge detection, which can comprehensively acquire information on surface defects of blades and greatly improve detection accuracy and precision.

[0109] 3. The detection method of the present invention is seamlessly integrated with the existing operation and maintenance management system, realizing real-time high-precision detection and significantly improving the operational safety of wind farms and the service life of equipment.

[0110] 4. The detection method of the present invention can be widely applied to the intelligent detection and health management of offshore wind farm blades, which is of great significance for reducing operation and maintenance costs and promoting the sustainable development of the wind power industry. Attached Figure Description

[0111] Figure 1 This is a flowchart of the offshore wind turbine blade defect detection method in Embodiment 1 of the present invention;

[0112] Figure 2 This is a schematic diagram of the DW network structure in Embodiment 1 of the present invention;

[0113] Figure 3 This is a schematic diagram of the C2f_CBAM module structure in Embodiment 1 of the present invention;

[0114] Figure 4 This is a schematic diagram of the QX network structure in Embodiment 1 of the present invention;

[0115] Figure 5 This is a schematic diagram of the FRConv module structure in Embodiment 1 of the present invention;

[0116] Figure 6 This is a structural diagram of the CAM module in Embodiment 1 of the present invention;

[0117] Figure 7 This is a structural diagram of the SAM module in Embodiment 1 of the present invention;

[0118] Figure 8 This is a structural diagram of the BEAM in Embodiment 1 of the present invention.

[0119] Figure 9 This is a diagram of the upper and lower boundary lines before modification in Embodiment 1 of the present invention.

[0120] Figure 10 This is a diagram of the modified upper and lower boundary lines in Embodiment 1 of the present invention. Detailed Implementation

[0121] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0122] It should be noted that if the embodiments of the present invention involve directional indicators (such as up, down, left, right, front, back, top, bottom, etc.), the directional indicators are only used to explain the relative positional relationship and movement of the components in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indicators will also change accordingly.

[0123] In this application, unless otherwise expressly specified and limited, the terms "installation," "connection," "joining," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise expressly limited. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0124] It should be noted that when a component is described as "fixed to" or "set on" another component, it can be directly on the other component or there may be an intervening component. When a component is described as "connected to" another component, it can be directly connected to the other component or there may be an intervening component.

[0125] Furthermore, if the embodiments of this invention involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by this invention.

[0126] Currently, conventional target detection algorithms suffer from drawbacks such as low accuracy, high latency, limited classification, and insensitivity to small target defects. To address these issues, this invention proposes a method for detecting defects in offshore wind turbine blades. This method employs a DW feature extraction network structure. Within this architecture, a C2f_CBAM (C2f_Convolutional Block Attention Module) is incorporated, enabling the network to not only focus on inter-channel information interaction but also delve deeper into the distribution of features in the image space, thereby more accurately capturing both local details and global information about blade defects. Furthermore, the concept of grid segmentation is introduced to classify and locate defects in the stitched image, improving the accuracy and efficiency of the entire recognition process.

[0127] Example 1

[0128] like Figure 1 As shown, a method for detecting defects in offshore wind turbine blades includes the following steps:

[0129] S01: Offshore wind turbine blade defect detection: Identify the location of blade defects, output the defect location marker box on the original image, and provide the coordinates of the center position of the marker box and its corresponding length and width;

[0130] S02: Blade Image Stitching: By taking pictures of different parts of the blade step by step, the image data of the blade is gradually accumulated; with the help of image stitching technology, each detected image is stitched together to form a complete image of the wind turbine blade.

[0131] S03: Determine the actual physical location of blade defects in a single image: In the stitched overall image, determine the actual physical coordinates of the blade defects in the original single image corresponding to the wind turbine blades.

[0132] Preferred, such as Figure 2 As shown, in step S01,

[0133] To effectively address the problems of low accuracy, high latency, and insufficient sensitivity to small target defects in offshore wind turbine blade defect identification, a DW feature extraction network structure is proposed. The DW feature extraction network structure includes:

[0134] Input layer: Receives the image data to be processed;

[0135] Feature extraction convolutional layer (FRConv): Used to initially extract features from the input image;

[0136] C2f_CBAM: Used to enhance the network's attention to important features by focusing on inter-channel and spatial information interactions to improve the accuracy of feature extraction;

[0137] Spatial Pyramid Feature Fusion Module (SPFF): Used to integrate feature information at different scales to improve the feature representation capability of the network;

[0138] Upsampling operation (Up): Used to restore the feature map to its original size or the desired size;

[0139] Channel Fusion Layer (CBS): Used to fuse feature information from different channels to generate richer feature representations;

[0140] Concat operation: used to stitch multiple feature maps along the channel dimension to integrate feature information from different sources;

[0141] Feature Clarification Layer: Used to further process and optimize the fused features to improve their clarity and interpretability;

[0142] Output layer: Generates the final feature representation or prediction result.

[0143] This network structure, through multi-layer convolutional operations, attention mechanisms, and feature fusion techniques, can effectively extract and integrate feature information from leaf images, thereby achieving accurate detection and analysis of leaf defects.

[0144] This architecture incorporates a C2f_CBAM attention module, enabling the network to not only focus on information interaction between channels but also delve deeper into the distribution of features in the image space, thereby capturing more accurate local details and global information about blade defects. Building upon this, the concept of grid segmentation is further introduced to classify and locate defects in the stitched image, improving the accuracy and efficiency of the entire recognition process.

[0145] The location of blade defects is identified using a DW feature extraction network structure, which incorporates a C2f_CBAM module.

[0146] like Figure 3 As shown, the C2f_CBAM module adds the original image features to the enhanced features through residual connections to ensure feature continuity. Since offshore wind turbine blades often have small defects, the C2f_CBAM module introduces a BEAM module to enhance the DW network's ability to perceive important features, further suppress useless and redundant information in the original image, and further improve the network's accuracy in detecting small blade defects.

[0147] The BEAM module consists of multiple BottleNeck_CBAM modules.

[0148] The process of outputting defect localization bounding boxes on the original image, providing the coordinates of the center position of the bounding box and its corresponding length and width, specifically involves: dividing the input image into n×n small grids (n is usually a multiple of 8, such as 8×8, 16×16, or n can take specific values ​​such as 7, 14, 19, etc., applicable to YOLO and R-CNN series algorithms); inputting each small grid image into the DW network structure; specifying that each small grid predicts S anchors with a predetermined aspect ratio (the number of anchor boxes used in each grid unit), wherein the center of the bounding box must be within the corresponding network region; through the DW network structure, the length, width, and center point coordinates of the predicted bounding box are output; simultaneously, the classifier in the network structure outputs the probability of predicting each category. Scores are set as the confidence score corresponding to each predicted box: the calculation formula for Scores is shown in Equation I.

[0149]

[0150] Wherein, P(Class) i The value represents the probability of prediction for different blade defect categories. PIoU is an improvement on IoU (Intersection over Union) and is used to evaluate the degree of overlap between the predicted bounding box and the true bounding box, thereby accurately measuring the accuracy of localization. `pred` is the predicted value, and `truth` is the true value. A higher PIoU value means a larger overlap area between the predicted and true bounding boxes, indicating higher accuracy in the prediction result. The formula for calculating PIoU is shown in Equation II:

[0151]

[0152] Where A represents the predicted bounding box, B represents the ground truth bounding box, and i takes values ​​from 1 to 4. This represents the distance from the midpoint of the line connecting the predicted bounding box and the ground truth bounding box to the coordinates of each point in the predicted bounding box. This represents the distance from the midpoint of the line connecting the predicted bounding box and the ground truth bounding box to the coordinates of each point in the ground truth bounding box. This indicates taking the maximum distance.

[0153] The improved PIOU loss function enhances the model's convergence efficiency. By combining the corresponding classification probabilities, the parameters in the classifier can be better adjusted, thereby generating predicted boxes that are closer to the true bounding boxes and further improving the accuracy of defect localization.

[0154] Since each grid generates S bounding boxes, there may be cases where a single target has multiple bounding boxes. Therefore, it is necessary to reasonably filter the bounding boxes. Preferably, the filtering of multiple bounding boxes specifically includes: 1) discarding bounding box confidence scores below a threshold, and 2) using matrix nonmaximum suppression (Matrix NMS) to remove redundant bounding boxes for each class, until all redundant bounding boxes have been processed, at which point the entire process ends.

[0155] Thus far, the model has demonstrated good performance in detecting defects in offshore wind turbine blades, accurately predicting the location and corresponding category of blade defects, and exhibiting low false detection and false negative rates.

[0156] In actual offshore wind turbine blade defect identification tasks, due to the large area of ​​the blades, a single drone shot is insufficient to cover the entire blade. Therefore, step S02 adopts a step-by-step shooting strategy, gradually accumulating image data of the blade by capturing different parts of it sequentially. Then, image stitching technology is used to fuse the scattered image fragments together, ultimately constructing a complete image of the wind turbine blade for further defect analysis and identification.

[0157] Preferably, in step S02,

[0158] By taking photos of different parts of the blade in stages, image data of the blade is gradually accumulated; then, using image stitching technology, each detected image is stitched together to form a complete image of the wind turbine blade.

[0159] The image stitching technique is used to stitch together each detected image. The specific steps are as follows:

[0160] S21. Extract features from the original single image and obtain feature point information for each image through supervised learning to facilitate subsequent image stitching operations;

[0161] S22. Based on the feature points matched in step S21, estimate the geometric transformation relationship between different images and complete the stitching.

[0162] Preferably, in step S21,

[0163] The feature extraction of the original single image adopts the QX network structure, utilizes FRConv convolutional layers to reduce the number of model parameters, introduces the BEAM module, and uses the idea of ​​residual connections to ensure the performance of deep learning. It focuses on inter-channel information and emphasizes the distribution of features in the image space, thereby capturing more local and global information about blade defects.

[0164] By utilizing FRConv convolutional layers, the accuracy of extracting blade defect features is improved while reducing the number of model parameters, and the robustness of the model to input images is enhanced. The BEAM module is introduced, employing the idea of ​​residual connections to ensure the performance of deep learning, focusing on inter-channel information and emphasizing the distribution of features in the image space, thereby capturing more local and global information about blade defects. Through the assembly of modules, a QX network structure is formed. This structure can deeply learn and extract effective information closely related to blade defects, identify feature points, and assign descriptive information to these feature points, forming a series of feature vectors, providing strong support for the accurate identification and localization of blade defects.

[0165] The QX network structure is as follows: Figure 4 As shown, it includes:

[0166] Convolutional layer (Conv): It is a neural network layer used to extract image features;

[0167] Average Pooling: It is a neural network layer used to reduce the size of feature maps. It downsamples the feature maps output by the Conv layer to reduce the size of the feature maps, thereby reducing the computational complexity of the network.

[0168] The QX network first extracts features from the input image using a Conv layer, and then further extracts blade defect features using an FRConv layer. Next, an Average Pooling layer is used to downsample the feature map to reduce its size. Then, a BEAM module is introduced to capture more local and global information about the blade defects. Finally, Conv and Average Pooling layers are used to process the features again for accurate identification and localization of blade defects.

[0169] To extract more comprehensive feature information from offshore wind turbine blades, the FRConv module is used. By integrating contextual and regional feature information, it helps the network better understand the relationship between defects and the surrounding environment, thus improving the network's accuracy in extracting blade defect features. Figure 5As shown, the structure of the FRConv module includes a region feature extractor, a context feature extractor, and an integrated feature extractor, wherein the region feature extractor and the context feature extractor are both connected to the integrated feature extractor.

[0170] The FRConv module performs feature extraction, including the following steps: the region feature extractor is a 3×3 standard convolutional layer (Conv) that learns local features from the eight surrounding feature vectors; the context feature extractor is a 3×3 dilated convolution (AConv), which has a relatively large receptive field and can effectively learn surrounding contextual information; the ensemble feature extractor concatenates the output features of the region feature extractor and the context feature extractor (Concat), and processes them through a batch normalization (BatchNorm2d) function and a FReLU activation function (BN+FReLU), finally inputting the extracted feature map. The overall output of the FRConv module is represented as follows:

[0171] FRConv(x)=FReLU(BatchNorm2d(Concat(Conv 3×3 (x),AConv 3×3 (x));

[0172] Where FRConv is the output of the FRConv module, FReLU is the FReLU activation function, BatchNorm2d is the batch normalization BatchNorm2d function, Concat is the connection, and Conv... 3×3 AConv is a standard 3×3 convolutional layer. 3×3 It is a 3×3 dilated convolution, where x is the input feature map.

[0173] The BEAM module adds the original image information to the image information enhanced by the CBAM module through residual connection to ensure the continuity of features and the flow of information, thereby enhancing the perception and learning ability of blade defect features.

[0174] The CBAM module employs two module components: the Channel Attention Module (CAM) and the Spatial Attention Module (SAM).

[0175] The operation flow of the CBAM module is as follows: input the original image, multiply the output of the channel attention module (CAM) with the original image to extract features; at the same time, use the output of the channel attention module (CAM) as input to enter the two-dimensional convolution of the spatial attention module (SAM), and then multiply the output with the original image to enhance the model's ability to perceive features, so as to better identify defects in small targets.

[0176] The Channel Attention Module (CAM) focuses on meaningful information in the input image and enhances the feature representation of each channel by compressing spatial dimensions while maintaining channel dimensions. For example... Figure 6 As shown, the input feature map is passed through two parallel max pooling layers (MaxPool) and a global average pooling layer (AvgPool) to reduce the feature map from C*H*W to C*1*1 size. Then, it passes through a shared fully connected layer (Share MLP) module. In the Share MLP module, the number of channels is first compressed to 1 / r of the original number, and then expanded to the original number of channels. After passing through the FReLU activation function, two activated results are obtained. These two output results are added element-wise, and then the channel attention weight α is obtained by passing through the Sigmoid activation function. Finally, this output result is multiplied by each channel of the original feature map to reduce it back to the size of C*H*W.

[0177] The calculation formula in the Shared Fully Connected Layer (Share MLP) module is shown in Equation III:

[0178]

[0179] Where x is the input feature map, This represents a 1x1 convolution operation with the following output channels: Where C is the number of channels in the input feature map, and r is a reduction ratio used to reduce parameters and computational cost.

[0180] The formula for calculating the channel attention weight α is shown in Equation IV:

[0181] α = Sigmoid(x) Sum );

[0182] Formula IV;

[0183] Where α is the channel attention weight, Sigmoid is the Sigmoid activation function, and x Sum This is the result of adding the two feature maps element-wise after activation by the FReLU activation function.

[0184] like Figure 7As shown, the Spatial Attention Module (SAM) focuses on more target location information by compressing the channel dimension while maintaining the spatial dimension. The SAM takes the output of the Channel Attention Module (CAM) as its input feature, and obtains two 1*H*W feature maps through max pooling and average pooling. Then, the two feature maps are concatenated using a concat operation to obtain a 2*H*W feature map x. Concat The feature map is processed by a convolutional layer (Conv layer), and then a sigmoid activation function is used to obtain the feature map of spatial attention. Finally, the output is multiplied by the original image to restore it to the size of C*H*W, and the features at each spatial location are weighted.

[0185] The feature map x Concat The formula is shown in equation V:

[0186] x Concat =Concat(x) MaxPool ,x AvgPool );

[0187] Formula V;

[0188] like Figure 8 As shown, the BEAM model structure adds the original image information and the enhanced image information through residual connections to ensure the continuity of features and the flow of information, thereby enhancing the perception and learning ability of blade defect features. The Fosine similarity method is established to calculate the similarity between two images. Based on the calculation results, feature points are matched, and the feature point detection network loss function is used to evaluate the prediction accuracy.

[0189] The formula for calculating the Fosine similarity is shown in Equation VI:

[0190]

[0191] a and b are two-dimensional feature vectors of two images, respectively.

[0192] The loss function of the feature point detection network is shown in Equation VII:

[0193] Loss=1-Fosine_Similarity(FRConv(prea),Conv(t));

[0194] Equation VII;

[0195] prea represents the feature points of the predicted value, and t represents the labeled feature points.

[0196] The higher the value of the loss function of the feature point detection network, the more inaccurate the prediction; the lower the value of the loss function, the closer the prediction is to the true value.

[0197] Preferably, in step S22, estimating the geometric transformation relationship between different images specifically involves: estimating transformation parameters based on minimizing the reprojection error, thereby determining the positional relationship between matching feature points, and fitting translation, rotation, and scaling transformations through least squares reprojection to provide support for subsequent image stitching.

[0198] The geometric transformation based on minimizing reprojection error includes the following steps:

[0199] The model takes the feature vectors of any feature point matching pair in the matching image as input, and aims to find a geometric transformation model that makes one feature vector in the matching pair as close as possible to another feature vector after transformation, so as to better determine the transformation relationship. An affine transformation model is used, which includes three transformation parameters: translation, rotation and scaling. The reprojection error is used as the objective function to measure the error between the model's predicted value and the actual value.

[0200] The affine transformation model is shown in Equation VIII:

[0201]

[0202] Among them, (x i ,y i (x) represents a feature vector in a matching pair. i ′,y i ′) represents the eigenvector after geometric transformation, e, f, g, and h represent the rotation and scaling parameters, respectively, and p and q represent the translation parameters.

[0203] The reprojection error, used as the objective function, measures the error between the model's predicted value and the actual value. For a two-dimensional image, assuming there are N matching pairs, the i-th point on the source image is (x... i ,y i The corresponding point on the target image is (x i ′,y i The reprojection error is used to minimize the objective function through gradient descent to solve for the optimal parameters of the corresponding geometric transformation, thereby estimating the geometric transformation relationship between two images. Based on the estimated geometric transformation relationship, the different images are stitched together. For each pixel in the overlapping area, the weighting coefficient is calculated based on the gradient difference of its surrounding pixels. The gradient of the image is calculated using a two-dimensional second-order difference method.

[0204] The formula for calculating the reprojection error is shown in Equation IX:

[0205]

[0206] The operation of stitching different images based on the estimated geometric transformation relationship specifically involves: using the geometric transformation relationship between images to perform geometric transformation on the images, and then combining the relevant results of feature matching to identify the overlapping areas between images, and achieving seamless stitching by merging the pixels in the overlapping areas.

[0207] For each pixel within the overlapping region, a weighting coefficient is calculated based on the gradient difference between its surrounding pixels. Specifically, assuming the gradients of the two images are G1 and G2 respectively, the weighted fusion formula Blend(x,y) for each pixel within the overlapping region is as described in equation X:

[0208]

[0209] Where I1(x,y) and I2(x,y) are the pixel values ​​of the two images in the overlapping region, and G1(x,y) and G2(x,y) are the gradient values ​​of the two images in the overlapping region.

[0210] The gradient calculation of the image is performed using a two-dimensional second-order difference method. By adding the second-order differences in the x and y directions, the gradient image is obtained. The weighted fusion formula and the difference method are applied to each pixel in the overlapping area to effectively eliminate the discontinuity at the image stitching boundary, thereby achieving a smooth transition and continuity of the image.

[0211] The formula for the two-dimensional second-order difference method is shown in Equation XI:

[0212]

[0213]

[0214] The above operation process is performed on all input images. At this point, a relatively complete and clear image of the blade is output. The entire operation is closely connected and shows a good image stitching effect, laying a solid foundation for the subsequent identification and localization of blade defects.

[0215] Preferably, in step S03:

[0216] The process of determining the actual physical coordinates of the blade defect in the original single image corresponding to the wind turbine blade is as follows: the defect is physically located on the stitched image, and the defect coordinates are determined with the lower left corner of the wind turbine blade as the origin; before the location, the invalid area of ​​the image needs to be cropped to remove the non-wind turbine area, and the lower left corner of the removed image is taken as the origin of the coordinate system.

[0217] In the process of determining the defect coordinates, a search algorithm based on 8-neighborhood relationship is used to extract the pixel position of each edge and its extension order on the edge. This algorithm can overcome the problems of pixel stacking or discontinuity.

[0218] In a binary image F(i,j), non-zero points represent edges. Ideally, these non-zero points should be continuous and distributed as single pixels in the image space. However, some edges may exhibit pixel stacking or pixel discontinuity.

[0219] The search algorithm based on 8-neighborhood relationships designs an edge detection operator Q(i,j) within the 8-neighborhood of a pixel. Its functions are: first, to determine whether the current pixel is an edge point based on the Q(i,j) value; and second, to determine the coordinates of the next edge point. The specific application method of the edge detection operator Q(i,j) is shown in Equation XII:

[0220]

[0221] Here, the gray value of edge points is 1, the gray value of background points is 0, and m and n are the indices used to traverse the 8 neighboring pixels of the current pixel (i,j), which can be -1, 0, or 1.

[0222] Using an 8-neighborhood search algorithm, the upper and lower boundaries of the wind turbine blades are found. This involves identifying the maximum (or minimum) Y value on the left and the minimum (or maximum) Y value on the right, i.e., the two points at the outermost boundaries on both sides. The equation of the upper boundary line YS is determined by these two points as YS = KS*X1 + BS. Similarly, the two points at the lower boundary line are found using the search algorithm, and the equation of the lower boundary line YU is determined as YU = KU*X2 + BU.

[0223] Find the lowest point of the lower boundary line, and then correct the original equations of the upper and lower boundaries so that the lowest point of the lower boundary line is the origin. The corrected equation of the upper boundary line is YS = KS*X1 + BS - C1, and the equation of the lower boundary line is YU = KU*X2 + BU - C2. See the schematic diagram below. Figure 10 As shown.

[0224] Where YS is the Y-coordinate of any point on the upper boundary line, KS is the slope of the upper boundary line, X1 is the X-coordinate of any point on the upper boundary line, BS is the intercept of the upper boundary line, and C1 is a constant subtracted from the equation of the upper boundary line.

[0225] Similarly, YU is the Y-coordinate of any point on the lower boundary line, KU is the slope of the lower boundary line, X2 is the X-coordinate of any point on the lower boundary line, BU is the intercept of the lower boundary line, and C2 is a constant subtracted from the equation of the lower boundary line.

[0226] To smooth image features, better handle edge detection, and avoid insufficient data points, the IGP model is used to optimize the boundary equation. Taking the upper boundary line equation as an example, it is assumed that KS follows KS~N(μ0,∑0), and BS follows BS~N(0,σ). 2 I), where μ0 is the mean vector of KS, ∑0 is the prior covariance matrix, and σ 2 Let I be the variance of the error term, and I be the identity matrix. The posterior distribution of KS is calculated using the formula shown in Equation XIII:

[0227] p(KS|X,YS)∝p(YS|X,KS)p(KS);

[0228] Formula XIII;

[0229] Where KS is the slope of the upper boundary line, X is the X coordinate of any point on the upper boundary line, and YS is the Y coordinate of any point on the upper boundary line.

[0230] p(KS|X,YS) is the posterior distribution of the slope KS, i.e., the probability distribution of the slope KS given the data X and YS; p(YS|X,KS) is the likelihood function of observed YS, which describes the probability of data YS given the data X and KS; p(KS) is the prior distribution of the slope KS, i.e., the prior belief about the slope KS without any data.

[0231] The mean and covariance of YS are calculated using the formula shown in Equation XIV:

[0232] μ post =∑0M T (M∑0M T +σ 2 I) -1 YS;

[0233] ∑ post =∑0-∑0M T (M∑0M T +σ 2 I) -1 M∑0;

[0234] Formula XIV;

[0235] Where, μ post Let YS be the mean vector, ∑ post Let YS be the covariance matrix, ∑0 be the prior covariance matrix; M is a matrix, each row representing the X-coordinate of any point on the upper bound, and T is the matrix transpose sign; σ 2 I is the variance of the error term, and I is the identity matrix.

[0236] It can be implemented given a new input x * Predicted output y *The formula is shown in equation XV:

[0237] p(y * |x * ,X,YS)=∫p(y * |x * ,KS)p(KS|X,YS)dKS;

[0238] Formula XV;

[0239] Where X is the X-coordinate of any point on the upper boundary line, YS is the Y-coordinate of any point on the upper boundary line, and KS is the slope of the upper boundary line.

[0240] p(y * |x * (X, YS) represents the input x given a new input x. * Given input X and output YS, the new output y is predicted by integrating over all possible values ​​of KS. * The probability distribution of p(y); * |x * KS) is the expression for a given new input x * In the case of KS, the new output y * The probability distribution of KS; p(KS|X,YS) is the probability distribution of KS given the known input X and output YS.

[0241] Based on the above operations, the final original image will output the location bounding boxes of defects, including the coordinates of the center position of the bounding box and its corresponding length and width. After the image stitching operation, the positions of the stitching lines of different images in the original image can be recorded (the leftmost position is 0), and the order of the images is known. Combining the above two pieces of information, the position information of each stitching line in the entire image can be calculated. By performing certain digital transformations based on the positional distribution of the stitching lines, the position of the leaf defect corresponding to the original image before stitching can be obtained.

[0242] The center coordinates (x, y) of the new image defect localization marker box are known. i ,y i Given a length w and a width h, the coordinates of the top right, bottom right, top left, and bottom left corners of the marker frame can be easily calculated. The coordinates of the defect are represented by the coordinates of the midpoint of the marker frame, i.e., X. q =(x i +w) / 2,Y q =(y i +h) / 2.

[0243] Based on the upper and lower boundaries, first determine the defect coordinate Y. q Perform a relative transformation, if Y qIf the defect is detected above the upper boundary line or below the lower boundary line, it is considered abnormal and discarded. If Y q Between the upper and lower boundaries, a transformation is performed, i.e., Y. q The Manhattan distance to the origin of the lower boundary, i.e.

[0244] For the defect coordinate X q The transformation considers the horizontal overlap that can occur during image stitching. In this image stitching algorithm, the search range of keypoints can be dynamically adjusted to find the optimal matching point pairs. The FRConv module and Fosine_Similarity are used to find corresponding keypoints in two images, and the width of the overlapping portion is determined by calculating the distance between the keypoints. The n closest keypoints are selected, and the Manhattan average distance D between them is calculated. i D i =SUM(n i xn j x) / n,n i x and n j x represents n i n j The x-coordinate. Obtain the D-coordinate between adjacent images. i Each sub-image has a width of w. i Then the horizontal position L of the dividing line of the i-th sub-image in the stitched image is... i Represented as:

[0245]

[0246] Then the transformed X qe =L i -D.

[0247] At this point, the physical localization step of the blade defect in a single image is completed. The above-mentioned network structure has high defect recognition accuracy. At the same time, the calculation process of obtaining the defect coordinate information of a single image from the defect information of the whole image is simple and fast, and it can effectively select defect information with a high accuracy rate.

[0248] Compared with the prior art, the technical solution of the present invention has the following advantages:

[0249] 1. The detection method of the present invention realizes fully automatic intelligent detection of defects in offshore wind turbine blades, which greatly improves detection efficiency and coverage, and helps to detect and deal with hidden dangers in a timely manner, and avoid the expansion of faults.

[0250] 2. The detection method of the present invention integrates multiple detection techniques such as computer vision, feature extraction, image stitching, and edge detection, which can comprehensively acquire information on surface defects of blades and greatly improve detection accuracy and precision.

[0251] 3. The detection method of the present invention is seamlessly integrated with the existing operation and maintenance management system, realizing real-time high-precision detection and significantly improving the operational safety of wind farms and the service life of equipment.

[0252] 4. The detection method of the present invention can be widely applied to the intelligent detection and health management of offshore wind farm blades, which is of great significance for reducing operation and maintenance costs and promoting the sustainable development of the wind power industry.

[0253] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting defects in offshore wind turbine blades, characterized in that: Includes the following steps: S01: Offshore wind turbine blade defect detection: Identify the location of blade defects, output the defect location marker box on the original image, and provide the coordinates of the center position of the marker box and its corresponding length and width; S02: Blade Image Stitching: By taking pictures of different parts of the blade step by step, the image data of the blade is gradually accumulated; with the help of image stitching technology, each detected image is stitched together to form a complete image of the wind turbine blade. S03: Determine the actual physical location of blade defects in a single image: In the stitched overall image, determine the actual physical coordinates of the blade defects in the original single image corresponding to the wind turbine blades; In step S01: the location of the blade defect is identified using a DW feature extraction network structure, which includes: Input layer: Receives the image data to be processed; Feature extraction convolutional layer: used to initially extract features from the input image; C2f_CBAM: Used to enhance the network's attention to important features by focusing on inter-channel and spatial information interactions to improve the accuracy of feature extraction; Spatial pyramid feature fusion module: used to integrate feature information at different scales to improve the feature representation capability of the network; Upsampling operation: used to restore the feature map to its original size or the desired size; Channel fusion layer: Used to fuse feature information from different channels to generate richer feature representations; Connect operation: used to concatenate multiple feature maps along the channel dimension to integrate feature information from different sources; Feature clarification layer: Used to further process and optimize the fused features to improve their clarity and interpretability; Output layer: Generates the final feature representation or prediction result.

2. The method for detecting defects in offshore wind turbine blades according to claim 1, characterized in that: In step S01: The C2f_CBAM module is introduced into the DW feature extraction network structure; The C2f_CBAM module adds the original image features to the enhanced features through residual connections to ensure the continuity of the features; The C2f_CBAM module introduces the BEAM module to enhance the DW network's ability to perceive important features, further suppress useless and redundant information in the original image, and further improve the network's accuracy in detecting small targets such as blade defects. The BEAM module consists of multiple BottleNeck_CBAM modules.

3. The method for detecting defects in offshore wind turbine blades according to claim 1, characterized in that: In step S01: The step of outputting defect location bounding boxes on the original image and providing the coordinates of the center position of the bounding box and its corresponding length and width specifically involves: dividing the input original image into n×n small grid images; inputting each small grid image into the DW network structure; specifying that each small grid predicts S anchors with pre-determined aspect ratios, where the center of the bounding box must be within the corresponding network region; outputting the length, width, and center point coordinates of the predicted bounding box through the DW network structure; simultaneously, the classifier in the network structure outputs the probability of prediction for each category; and setting Scores as the confidence score for each predicted box; the calculation formula for Scores is shown in Equation I: ； Formula I; in, PIoU represents the probability value predicted for different blade defect categories. It is an improvement on IoU and is used to evaluate the degree of overlap between the predicted bounding box and the true bounding box, thereby accurately measuring the accuracy of localization. The higher the PIoU value, the greater the overlap area between the predicted bounding box and the true bounding box, which indicates a higher accuracy of the prediction result. The formula for calculating PIoU is shown in Equation II: ； ； Formula II; Where A represents the predicted bounding box, B represents the ground truth bounding box, and i takes values ​​from 1 to 4. This represents the distance from the midpoint of the line connecting the predicted bounding box and the ground truth bounding box to the coordinates of each point in the predicted bounding box. This represents the distance from the midpoint of the line connecting the predicted bounding box and the ground truth bounding box to the coordinates of each point in the ground truth bounding box. , This indicates taking the maximum distance.

4. The method for detecting defects in offshore wind turbine blades according to claim 1, characterized in that: In step S01: the filtering of multiple bounding boxes specifically includes: 1) discarding bounding box confidence scores below a threshold, 2) using matrix nonmaximum suppression to remove redundant bounding boxes for each class, until all redundant bounding boxes are processed, and the entire process ends.

5. The method for detecting defects in offshore wind turbine blades according to claim 1, characterized in that: In step S02, each detected image is stitched together using image stitching technology. The specific steps are as follows: S21. Extract features from the original single image. Through supervised learning, obtain feature point information for each image to facilitate subsequent image stitching operations and match feature points. S22. Based on the feature points matched in step S21, estimate the geometric transformation relationship between different images and complete the stitching.

6. The method for detecting defects in offshore wind turbine blades according to claim 5, characterized in that: In step S21, feature extraction is performed on the original single image using a QX network structure. The FRConv convolutional layer is used to reduce the number of model parameters. The BEAM module is introduced, and the idea of ​​residual connection is used to ensure the performance of deep learning. The focus is on inter-channel information and the distribution of features in the image space, thereby capturing more local and global information about the blade defects. The structure of the FRConv module includes a region feature extractor, a context feature extractor, and an integrated feature extractor, wherein the region feature extractor and the context feature extractor are both connected to the integrated feature extractor. The FRConv module performs feature extraction, including the following steps: the region feature extractor is a standard 3×3 convolutional layer that learns local features from the eight surrounding feature vectors; the context feature extractor is a 3×3 dilated convolution, which has a relatively large receptive field and effectively learns surrounding contextual information; the ensemble feature extractor concatenates the output features of the region feature extractor and the context feature extractor, and processes them through a batch normalization (BatchNorm2d) function and an FReLU activation function, finally inputting the extracted feature map; the overall output of the FRConv module is represented as: ； in, For the output of the FRConv module, For FReLU activation function, The BatchNorm2d function is used for batch normalization. For connection, It is a standard 3×3 convolutional layer. It is a 3×3 dilated convolution, where x is the input feature map.

7. The method for detecting defects in offshore wind turbine blades according to claim 6, characterized in that: In step S21, the BEAM module adds the original image information to the image information enhanced by the CBAM module through residual connection to ensure the continuity of features and the flow of information, thereby enhancing the perception and learning ability of blade defect features.

8. The method for detecting defects in offshore wind turbine blades according to claim 5, characterized in that: In step S22, estimating the geometric transformation relationship between different images specifically involves: estimating transformation parameters based on minimizing reprojection error, thereby determining the positional relationship between matching feature points, and fitting translation, rotation, and scaling transformations through least squares reprojection to provide support for subsequent image stitching. The geometric transformation based on minimizing reprojection error includes the following steps: The model takes the feature vectors of any feature point matching pair in the matching image as input, and aims to find a geometric transformation model that makes one feature vector in the matching pair as close as possible to another feature vector after transformation, so as to better determine the transformation relationship. An affine transformation model is used, which includes three transformation parameters: translation, rotation and scaling. The reprojection error is used as the objective function to measure the error between the model's predicted value and the actual value. The affine transformation model is shown in Equation VIII: = + ； Formula VIII; in, Represents a feature vector in a matching pair, ( ′, ′) represents the eigenvector after geometric transformation, e, f, g, h represent the rotation and scaling parameters, and p and q represent the translation parameters; The reprojection error, used as the objective function, measures the error between the model's predicted value and the actual value. For a two-dimensional image, assuming there are N matching pairs, the i-th point on the source image is ( , The corresponding point on the target image is ( ′, The reprojection error is used to minimize the objective function through gradient descent to solve for the optimal parameters of the corresponding geometric transformation, thereby estimating the geometric transformation relationship between two images. Based on the estimated geometric transformation relationship, the different images are stitched together. For each pixel in the overlapping region, the weighting coefficient is calculated based on the gradient difference of its surrounding pixels. The gradient of the image is calculated using a two-dimensional second-order difference method. The formula for calculating the reprojection error is shown in Equation IX: ； Formula IX; The operation of stitching different images according to the estimated geometric transformation relationship is as follows: using the geometric transformation relationship between the images, the images are geometrically transformed, and then combined with the relevant results of feature matching, the overlapping areas between the images are identified, and seamless stitching is achieved by pixel fusion of the overlapping areas. For each pixel within the overlapping region, a weighting coefficient is calculated based on the gradient difference between its surrounding pixels. Specifically, assuming the gradients of the two images are G1 and G2 respectively, the weighted fusion formula Blend(x,y) for each pixel within the overlapping region is as shown in equation X: ； Formula X; in, and These are the pixel values ​​of the two images within the overlapping area. and These are the gradient values ​​of the two images within the overlapping region; The gradient calculation of the image is performed using a two-dimensional second-order difference method. By adding the second-order differences in the x and y directions, the gradient image is obtained. The weighted fusion formula and the difference method are applied to each pixel in the overlapping area to effectively eliminate the discontinuity at the image stitching boundary and achieve smooth transition and continuity of the image. The formula for the two-dimensional second-order difference method is shown in Equation XI: ； ； ； Formula XI.

9. The method for detecting defects in offshore wind turbine blades according to claim 1, characterized in that: In step S03: The process of determining the actual physical coordinates of the blade defect in the original single image corresponding to the wind turbine blade is as follows: the defect is physically located on the stitched image, and the defect coordinates are determined with the lower left corner of the wind turbine blade as the origin; before the location, the invalid area of ​​the image needs to be cropped to remove the non-wind turbine area, and the lower left corner of the removed image is taken as the origin of the coordinate system. In the process of determining the defect coordinates, a search algorithm based on 8-neighborhood relationship is used to extract the pixel position of each edge and its extension order on the edge. This algorithm can overcome the problems of pixel stacking or discontinuity. In a binary image F(i,j), non-zero points represent edges. Ideally, these non-zero points should be continuous and distributed as single pixels in the image space. However, some edges may exhibit pixel stacking or pixel discontinuity. The search algorithm based on 8-neighborhood relationships designs an edge detection operator Q(i,j) within the 8-neighborhood of a pixel. Its functions are: first, to determine whether the current pixel is an edge point based on the Q(i,j) value; and second, to determine the coordinates of the next edge point. The specific application method of the edge detection operator Q(i,j) is shown in Equation XII: ； Formula XII; Where the gray value of edge points is 1, the gray value of background points is 0, and m and n are the indices of the 8 neighboring pixels of the current pixel (i,j), which are -1, 0, and 1 respectively. Using an 8-neighborhood relation search algorithm, the upper and lower boundaries of the wind turbine blades are found. This involves identifying the maximum or minimum Y-value on the left and the minimum or maximum Y-value on the right, i.e., the two points representing the outermost boundaries on both sides. The equation of the straight line representing the upper boundary YS is then determined using these two points. Similarly, by using a search algorithm to find two points on the lower boundary line, the equation of the lower boundary line YU is determined as follows: ; Find the lowest point of the lower boundary line, and correct the original equations of the upper and lower boundaries so that the lowest point of the lower boundary line is the origin; the corrected equation of the upper boundary line is: The equation of the lower boundary is ; Where YS is the Y-coordinate of any point on the upper boundary line, KS is the slope of the upper boundary line, X1 is the X-coordinate of any point on the upper boundary line, BS is the intercept of the upper boundary line, and C1 is a constant subtracted from the equation of the upper boundary line. Similarly, YU is the Y-coordinate of any point on the lower boundary line, KU is the slope of the lower boundary line, X2 is the X-coordinate of any point on the lower boundary line, BU is the intercept of the lower boundary line, and C2 is a constant subtracted from the equation of the lower boundary line. To smooth image features, better handle edge detection, and avoid insufficient data points, the IGP model is used to optimize the boundary equation. Taking the boundary equation YS as an example, it is assumed that KS follows... BS obey ,in, Let KS be the mean vector. The prior covariance matrix, is the variance of the error term, and I is the identity matrix; The posterior distribution of KS is calculated using the formula shown in Equation XIII: ； Formula XIII; Where KS is the slope of the upper boundary line, X is the X coordinate of any point on the upper boundary line, and YS is the Y coordinate of any point on the upper boundary line. The mean and covariance of YS are calculated using the formula shown in Equation XIV: ； ； Formula XIV; in, Let YS be the mean vector. Let YS be the covariance matrix. Let M be the prior covariance matrix; T is the matrix transpose symbol, and each row of matrix M represents the X coordinate of any point on the upper bound. is the variance of the error term, and I is the identity matrix; Implement given a new input Predicted output The formula is shown in equation XV: ； Formula XV; Where X is the X coordinate of any point on the upper boundary line, YS is the Y coordinate of any point on the upper boundary line, and KS is the slope of the upper boundary line. To know the new input Given input X and output YS, the new output is predicted by integrating over all possible values ​​of KS. The probability distribution; To give a new input In the case of KS, the new output The probability distribution; Let KS be the probability distribution given a known input X and output YS. Based on the above operations, the final original image will output the location marker box of the defect, including the coordinates of the center position of the marker box and the corresponding length and width. After the image stitching operation, the position of the segmentation line of different image stitching in the original image is recorded, with the leftmost position being 0. The order between the images is known. Combining the above two pieces of information, the position information of each segmentation line in the whole image is calculated. Based on the position distribution of the segmentation line, a certain digital conversion is performed to obtain the position of the leaf defect corresponding to the original image before stitching.

10. The method for detecting defects in offshore wind turbine blades according to claim 9, characterized in that: In step S03: The center coordinates of the new image defect location marker box are known ( , Given a length w and a width h, the coordinates of the top right, bottom right, top left, and bottom left corners of the marker frame can be easily calculated. The coordinates of the defect can then be represented by the coordinates of the midpoint of the marker frame. =( +w) / 2, =( +h) / 2; Based on the upper and lower boundaries, first determine the defect coordinates. Perform a relative transformation, if If the defect is detected abnormally at the upper boundary or lower boundary, it is discarded; if Between the upper and lower boundaries, a transformation is performed, i.e. The Manhattan distance to the origin of the lower boundary, i.e. ; For defect coordinates The transformation takes into account the horizontal overlap effect of image stitching. In this graphic stitching algorithm, the search range of key points is dynamically adjusted to find the best matching point pair. The FRConv module and Fosine_Similarity are used to find corresponding key points in two images, and the width of the overlapping part is determined by calculating the distance between the key points. The n closest key points are selected, and the Manhattan average distance between them is calculated. ,Right now =SUM( x- x) / n, x and x are respectively , The x-coordinate; to obtain the x-coordinate between adjacent images The width of each sub-image is Then the horizontal position of the dividing line of the i-th sub-image in the stitched image is... Represented as: ； ； Then X after transformation q coordinates .

Images