Fan blade damage identification method and system based on improved convolutional neural network

Abstract

The application discloses a fan blade damage identification method and system based on an improved convolutional neural network, and particularly relates to the technical field of digital image processing and image defect detection. An original fan blade image is collected, and a sample pair set of standard fan blade labels is output. An image defect detection processing model is constructed, and a multi-scale feature map for identifying fine and small damage of the fan blade is extracted. Network parameters are updated based on the sample pair set through back propagation. A standard fan blade image to be detected is input into the trained image defect detection processing model, and an image defect detection result of the fan blade is output. Through the image defect detection processing model construction step, the application is combined with multi-layer lightweight convolution processing, multi-layer attention processing and multi-layer pyramid processing, and the extraction capability of long and thin crack strip features is considered, so that the missing rate of fine and small damage and the false defect detection rate caused by texture shadow are reduced.

Description

Technical Field

[0001] This invention relates to the field of digital image processing and image defect detection technology, specifically to a method and system for identifying wind turbine blade damage based on an improved convolutional neural network. Background Technology

[0002] As a key component of wind turbine generators, wind turbine blades are prone to damage such as long cracks, micropores, and surface peeling during long-term outdoor operation. Hidden micro-damage can continue to expand and cause blade breakage, seriously threatening the operational safety of wind farms.

[0003] Current mainstream damage detection methods are divided into two categories: manual high-altitude inspection and traditional CNN image recognition. Manual high-altitude inspection suffers from high operational risks, strong subjectivity, high rate of missed detection of minor damage, and extremely low inspection efficiency. Existing image processing solutions for wind turbine blade aerial image defect detection have many shortcomings: image preprocessing only uses single filtering and noise reduction, lacks a foreground mask segmentation image processing step, and irrelevant background pixels such as the sky and mountains consume a large amount of image processing computing power; traditional 3×3 convolution image operators have a single receptive field, making it difficult to process images of fine, elongated crack textures, unable to distinguish between inherent blade textures and real minor defects, and prone to generating a large number of image pseudo-defects due to light and shadow. The pyramid image processing method lacks ultra-high resolution shallow image information. Small defects smaller than 4×4 pixels lose features after image downsampling, resulting in significant defect under-detection problems. Existing CNNs are only used for simple feature extraction and lack a complete process for image denoising, segmentation, and defect geometric parameter calculation, resulting in poor defect detection accuracy. The loss functions commonly used are IoU and SIoU boundary losses, which are only suitable for rectangular block targets and have a large error in recognizing images of slender cracks in wind turbines. The use of only cross-entropy loss (Focal) cannot constrain the damage geometry, resulting in a large deviation between the detected damage size and the actual shape. There is no multi-task dynamic weight balancing mechanism, and the classification and boundary regression gradients interfere with each other.

[0004] In summary, existing technologies lack a wind turbine blade recognition solution based on an image defect detection and processing model that is suitable for drone aerial photography scenarios, can identify small and slender damage, and can be deployed in a lightweight airborne manner. Summary of the Invention

[0005] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide a wind turbine blade damage identification method and system based on an improved convolutional neural network to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a wind turbine blade damage identification method based on an improved convolutional neural network, comprising: S1: Acquire the original aerial images of wind turbine blades, perform wavelet morphological denoising and blade foreground mask semantic segmentation image preprocessing on the original images to obtain standard wind turbine blade images and label them, and output a set of sample pairs of standard wind turbine blade labels; S2: Based on standard wind turbine blade images, multi-level image feature processing is performed. Spatial compression and channel dimensionality enhancement are completed through the first-layer downsampling convolution. Then, through multi-layer lightweight convolution processing, multi-layer attention processing, and multi-layer pyramid processing, multi-scale feature maps for identifying slender and minute damage to wind turbine blades are extracted. Combined with the non-maximum suppression algorithm, the wind turbine blade image detection results are obtained, and an image defect detection and processing model is constructed. S3: Based on the sample pair set of standard wind turbine blade labels, perform forward image feature inference to output defect detection results. Train the image feature processing model by constraining the three-class weighted composite loss of boundary, classification and morphology, adaptively adjust the loss weight, backpropagate to update the parameters of the image feature processing model, and obtain the trained image defect detection processing model. S4: Input the preprocessed image of the standard wind turbine blade to be tested into the trained image defect detection and processing model, perform image feature extraction forward, and output the wind turbine blade image defect detection result.

[0007] Preferably, the wind turbine blade damage identification system based on an improved convolutional neural network includes: Wind turbine blade image acquisition module: Acquires original aerial images of wind turbine blades, performs wavelet morphological denoising and blade foreground mask semantic segmentation image preprocessing operations on the original images to obtain standard wind turbine blade images and label them, and outputs a set of sample pairs of standard wind turbine blade labels; Image Defect Detection and Processing Model Construction Module: Based on standard wind turbine blade images, multi-level image feature processing is performed. Spatial compression and channel dimensionality enhancement are completed through the first-layer downsampling convolution. Then, through multi-layer lightweight convolution processing, multi-layer attention processing, and multi-layer pyramid processing, multi-scale feature maps for identifying slender and minute damage to wind turbine blades are extracted. Combined with the non-maximum suppression algorithm, the wind turbine blade image detection results are obtained, and the image defect detection and processing model is constructed. Offline model weighted composite loss training module: Based on the sample pair set of standard wind turbine blade labels, it performs forward image feature inference to output defect detection results. It trains the image feature processing model by constraining the three categories of boundary, classification, and morphology weighted composite loss, adaptively adjusts the loss weights, and backpropagates to update the parameters of the image feature processing model to obtain the trained image defect detection and processing model. Online wind turbine blade damage output module: Input the preprocessed image of the standard wind turbine blade to be tested into the trained image defect detection and processing model, perform image feature extraction forward, and output the wind turbine blade image defect detection result.

[0008] The technical effects and advantages of this invention are as follows: 1. This invention utilizes a complete image processing workflow for an image defect detection and processing model. By segmenting the image through a mask, irrelevant background pixels are removed, reducing the computational power consumption of ineffective image processing. At the same time, lightweight convolution, attention enhancement, and multi-layer pyramid image feature processing operators are used to enhance the image processing effect of fine crack textures, significantly reducing the false detection rate of small defects and the false detection rate of image pseudo-defects caused by leaf textures and light and shadow. 2. This invention designs three types of weighted composite loss functions based on image defect detection for training image feature processing models. It adds interval constraint processing to the defect geometric shape parameters of the image output to make up for the distortion problem of defect size detection in traditional image processing models. Combined with iterative round adaptive weight adjustment, it improves the defect detection accuracy of aerial blade images. 3. This invention uses drone aerial photography for automated identification, avoiding the personal safety risks of high-altitude operations, and batch image parallel inference significantly improves inspection efficiency; at the same time, it addresses a series of inherent defects in outdoor aerial blade detection from the entire process of preprocessing, network structure, loss training, and post-processing classification, forming an automated solution from image input to wind turbine blade damage. Attached Figure Description

[0009] Figure 1 This is a schematic diagram of the overall process of the present invention.

[0010] Figure 2 This is a schematic diagram of the method flow of the present invention.

[0011] Figure 3 This is a schematic diagram illustrating the construction process of the image defect detection and processing model of the present invention. Detailed Implementation

[0012] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0013] Please see Figure 1 As shown, the present invention provides a wind turbine blade damage recognition system based on an improved convolutional neural network, including a wind turbine blade image acquisition module, an image defect detection and processing model construction module, an offline model weighted composite loss training module, and an online wind turbine blade damage output module.

[0014] The wind turbine blade image acquisition module is connected to the image defect detection and processing model construction module and the offline model weighted composite loss training module, respectively, and the online wind turbine blade damage output module is connected to the offline model weighted composite loss training module.

[0015] Wind turbine blade image acquisition module: Acquires original aerial images of wind turbine blades, performs wavelet morphological denoising and blade foreground mask semantic segmentation image preprocessing operations on the original images to obtain standard wind turbine blade images and label them, and outputs a set of sample pairs of standard wind turbine blade labels; Image Defect Detection and Processing Model Construction Module: Based on standard wind turbine blade images, multi-level image feature processing is performed. Spatial compression and channel dimensionality enhancement are completed through the first-layer downsampling convolution. Then, through multi-layer lightweight convolution processing, multi-layer attention processing, and multi-layer pyramid processing, multi-scale feature maps for identifying slender and minute damage to wind turbine blades are extracted. Combined with the non-maximum suppression algorithm, the wind turbine blade image detection results are obtained, and the image defect detection and processing model is constructed. Offline model weighted composite loss training module: Based on the sample pair set of standard wind turbine blade labels, it performs forward image feature inference to output defect detection results. It trains the image feature processing model by constraining the three categories of boundary, classification, and morphology weighted composite loss, adaptively adjusts the loss weights, and backpropagates to update the parameters of the image feature processing model to obtain the trained image defect detection and processing model. Online wind turbine blade damage output module: Input the preprocessed image of the standard wind turbine blade to be tested into the trained image defect detection and processing model, perform image feature extraction forward, and output the wind turbine blade image defect detection result.

[0016] Please see Figure 2As shown, the wind turbine blade damage identification method based on an improved convolutional neural network includes: S1: Acquiring original aerial images of wind turbine blades, performing wavelet morphological denoising and blade foreground mask semantic segmentation image preprocessing operations on the original images to obtain standard wind turbine blade images and labeling them, outputting a set of sample pairs of standard wind turbine blade labels; S2: Performing multi-level image feature processing based on the standard wind turbine blade images, completing spatial compression and channel dimensionality enhancement through the first-layer downsampling convolution, followed by multi-layer lightweight convolution processing, multi-layer attention processing, and multi-layer pyramid processing to extract multi-scale feature maps for identifying slender and minute damage to wind turbine blades, combined with non-polar The algorithm for large value suppression is used to obtain the image detection results of the wind turbine blades, and an image defect detection and processing model is constructed. S3: Based on the sample pair set of standard wind turbine blade labels, forward image feature inference is performed to output the defect detection results. The image feature processing model is trained by constraining the loss through a three-class weighted composite loss of boundary, classification, and morphology, adaptively adjusting the loss weights, and backpropagating to update the parameters of the image feature processing model, resulting in a trained image defect detection and processing model. S4: The preprocessed standard wind turbine blade image to be tested is input into the trained image defect detection and processing model, and forward image feature extraction is performed to output the wind turbine blade image defect detection results. S1: Aerial images of the wind turbine blades are acquired, and wavelet morphological denoising and blade foreground mask semantic segmentation image preprocessing operations are performed on the original images to obtain standard wind turbine blade images and label them, outputting a sample pair set of standard wind turbine blade labels, including: S1.1: Collection of original aerial images of wind turbine blades (I) raw ={I raw,1 ,I raw,2 ,...,I raw,M}, M is the total number of images acquired; for each original image I raw,m First, perform DB4 wavelet combined soft thresholding noise reduction transform F. wave (I raw,m ,j max ,T wave ), j max T wave The wavelet decomposition level and the noise reduction threshold are used to obtain the denoised image I. raw,m qz ; Redefine the morphological structural element S∈R k×k (e.g., a square structuring element with k=3), combined with a denoised image I raw,m qz Perform morphological opening and closing joint transformation F morph (I raw,m qz First, an opening operation is performed to isolate salt-and-pepper noise, then a closing operation is performed to repair damaged edges of the blades and fill in tiny holes, outputting an image I that has undergone joint wavelet and morphological denoising. de,m Then based on image Ide,m and adaptive threshold T mask (Value range [0.4, 0.6]), a lightweight binary classification semantic segmentation network is used to generate a binary mask for the leaf foreground. m At pixel coordinates (x, y), a mask value of 1 indicates a leaf foreground, and pixel 0 indicates redundant background (sky, mountains, etc.). The lightweight semantic segmentation network used is such as MobileNetV2 or Lightweight ResNet. The mask is based on... m For the denoised image I de,m Pixel-by-pixel mask multiplication is performed to remove the background region, resulting in the mask-constrained image I. crop,m (x,y,c)=I de,m (x,y,c)×Mask m (x,y), c∈{0,1,2} represents the RGB three channels. Traverse all pixels of the mask to extract all Masks. m Find the extreme values ​​of the coordinates (x, y) = 1, and crop the rectangular region [x min ,x max ]×[y min ,y max ], thus obtaining an initial image I containing only the leaves. ROI,m Set the network input size to a fixed length H. net =Width W net (e.g. H) net =W net =640 pixels), and the image I is obtained by bilinear interpolation scaling. re,m =ResizeBilinear(I ROI,m H net W net ResizeBilinear(·) is a bilinear interpolation scaling function. Then, based on the mean and standard deviation of the c-th channel pixels in the wind turbine blade image set, it is applied to I... re,m Perform channel mean and variance standardization to obtain RGB channel standard image I. norm,m (x,y,c), finally passing the standard global minimum pixel I. norm,min Maximum value I norm,max , to I norm,m Normalize (x,y,c) to obtain the standard image I. std,m ∈R H×W×3 ; S1.2: For each image I std,m Generate structured labels m ={C m,J B m,J G m,J|J=1,2,...,N2}, where N2 is the total number of damages in each image, and the damage category label is C. m A total of n sample labels are defined, with label category numbers 0, 1, ..., n-1, where n is a positive integer greater than or equal to 3 (label categories include, but are not limited to, normal blades, crack damage, surface flaking damage, lightning strikes, impacts, etc., and damage categories can be flexibly added or removed according to the actual types of blade failures in the wind farm). The damage bounding box label is B. m =[x1,y1,x2,y2], where (x1,y1) is the top-left pixel of the damage bounding box, and (x2,y2) is the bottom-right pixel of the bounding box. The damage geometric parameter label feature set G is... m =[L m W m ,S m D m ], L m W m S m D m These are the damage pixel length (the number of pixels on the longer side of the smallest bounding rectangle of the damage region), the damage pixel width (the number of pixels on the shorter side of the same smallest bounding rectangle), the total pixel area of ​​the damage region, and the equivalent diameter of the point-like pore defects (when the damage number is 0, all fields in the damage bounding box and geometric parameter labels are labeled as null values); after all labels are completed, a set D of sample pairs with complete truth labels is obtained. all ={(I std,m Label m |m=1,2,...,M};Based on the total number of samples M and the set of sample pairs D all The training subsets D are divided according to a set ratio (e.g., 7:1:2). tr , Verification subset D va and test subset D te This invention is not limited to the four types of geometric parameters mentioned above; for other wind turbine damage such as lightning strike bifurcation scratches, impact dents, and surface corrosion, geometric output parameters such as contour curvature, number of bifurcations, equivalent perimeter, and concavity / convexity can be added, requiring only the addition of parameters to the G-axis. m The vector is expanded with corresponding dimensions in sequence, and corresponding constraint terms are added synchronously in the damage morphology regularization loss. It only needs to follow the core logic of "applying interval penalty based on defect geometric parameters". Please see Figure 3 As shown, S2: Based on standard wind turbine blade images, multi-level image feature processing is performed. Spatial compression and channel dimensionality enhancement are achieved through the first-layer downsampling convolution. Then, through multi-layer lightweight convolution processing, multi-layer attention processing, and multi-layer pyramid processing, multi-scale feature maps for identifying slender and minute damage to wind turbine blades are extracted. Combined with a non-maximum suppression algorithm, the wind turbine blade image detection results are obtained, and an image defect detection and processing model is constructed, including: S2.1: Initial convolution of the first layer: A single standard image I... std,m Input first convolutional layer Conv k1,s Perform feature upsampling and downsampling to generate the first-layer basic feature F0=Conv k1,s (I std,m k1 is the kernel size, and s is the sliding step (the present invention preferably uses a 3×3 kernel and a sliding step s=2 standard convolution operation. The kernel can be 5×3 or 3×5, and the step can be adjusted to 1 or 3. Only the number of subsequent network channels and the total downsampling factor need to be adjusted to achieve the basic feature extraction function). S2.2: Multi-layer lightweight convolution processing: Then, N layers of LASC lightweight asymmetric sparse convolution modules are executed cyclically. Each module evenly divides the input feature channels and uses long horizontal and vertical asymmetric convolutions to extract thin, elongated damaged textures. After fusion, sparse channel cropping is performed to reduce the number of parameters. The mapping relationship of a single-layer LASC lightweight asymmetric sparse convolution block is: F t =LASC(F t-1 ,r sp ), F t-1 r sp These are the input features from the previous layer and the sparse channel pruning ratio (e.g., r). sp =0.25 (can be adjusted according to dataset and operation and maintenance standards), F t To obtain the output features of this layer, perform LASC block operations N times (e.g., N=4) in a loop, obtaining F1, F2, ..., F... N The basic feature map is downsampled layer by layer; the internal operation process of LASC is as follows: First, the input feature channel is divided in half to obtain two paths of features F that are adapted to the horizontal and vertical long cracks, respectively. sp1 and F sp2 The two feature streams are input into a long strip asymmetric convolution to extract the horizontal and vertical crack texture features F, respectively. longH and F longV (For long strip asymmetric convolutions, 1×3 and 3×1 kernels can be used in parallel, or 1×5 and 5×1 kernels can be used as equivalent replacements, i.e., F...) longH =Conv 1×3 (F sp1 ), F longV =Conv3×1(F sp2 Then, the channel dimensions are concatenated using Concat, which outputs texture features from two convolutional paths, and the two directional strip features F are fused together. cat =Concat(F longH ,F longV Next, the sparse channel pruning operator SparsePrune is used to prune the channels according to the sparse channel pruning ratio r. sp Eliminating redundant channels completes the sparse compression feature F spa=SparsePrune(F cat ,r sp Finally, after batch normalization (BN) and the nonlinear activation function ReLU, the single-layer basic feature F is output. t =ReLU(BN(F spa )); S2.3: Multi-layer attention processing module: outputs feature F for each LASC layer. t Independent input to the MECA multi-layer cross-scale attention operator MECA(·), output enhanced features F t zq =MECA(F t N layers output features F t Perform the calculations separately and output F1. zq_1 F2 zq_2 ,...,F N zq_N Layer enhancement features; the original MECA process is as follows: First, AvgPool is used with different sizes of average pooling operations with a set number of layers N1. K_i×K_i Parallel extraction of multi-scale features P K_1 1 ,P K_2 2 ,...,P K_N1 N1 i=1,2,...,N1 (matching features such as micropores, medium cracks, and large-area detachment, e.g., N1=3, K_i=1, 5, 9, etc., P1 1 =AvgPool 1×1 (F t ), P5 2 =AvgPool 5×5 (F t ), P9 3 =AvgPool 9×9 (F t Then, the pooling results are concatenated using feature channels (Concat) to fuse multi-scale global information (P). all =Concat(P K_1 1 ,P K_2 2 ,...,P K_N1 N1 Next, the multi-scale global information P... all Global average pooling (GAP) is performed, followed by two fully connected layers FC1 and FC2 and a sigmoid activation function σ to generate cross-scale channel attention weights A. c Then, with the output feature F t Channel-by-channel multiplication suppresses uniform base texture of the blades, yielding a channel-dimensional feature weighted F.ch , A c =σ(FC2(ReLU(FC1(GAP(P all Secondly, the weighted feature F ch The channel mean and maximum values ​​are calculated separately and concatenated. Then, a spatial attention weight A is generated by a 3×3 convolution and a sigmoid function. s =σ(Conv 3×3 (Concat(Mean(F ch ),Max(F ch Finally, the weighted feature F ch Spatial attention weight A s Pixel-wise weighting, outputting enhanced feature F t zq , ; S2.4: Multi-layer pyramid processing: A feature pyramid is constructed from the enhanced features obtained through multi-layer attention processing. A new ultra-high resolution P2 layer is added to capture pixel-level micro-defects within the image (e.g., micro-defects smaller than 4×4 pixels). Cross-scale feature fusion is achieved by combining top-down upsampling with lateral residual addition, resulting in the fused feature P for each pyramid layer. i =F2 zq_i +ResizeUp(P i+1 size(F2) zq_i ResizeUp is bilinear interpolation upsampling (scaling high-level low-resolution features to the same size as the bottom layer). Each enhanced feature corresponds one-to-one with each level of the pyramid. (Those skilled in the art can adjust the number of LASC convolutional block stacking layers, MECA enhanced output feature layers, and feature pyramid layers through feature merging, virtual upsampling expansion, etc., only needing to retain the ultra-high resolution shallow layer for minor damage extraction, all of which can achieve the basic recognition effect of this invention. If the number of LASC and MECA output layers is inconsistent with the number of pyramid layers, multi-layer feature merging dimensionality reduction or top-level feature upsampling expansion can be used to complete the level matching.) The top layer of the pyramid is equal to F2. zq_N Without upper-layer input, a preferred approach in this embodiment is as follows: the multi-resolution feature pyramid has a four-layer structure: P2, P3, P4, and P5. P2 is a shallow, ultra-high-resolution layer used to capture minute early damage of 4×4 pixels or less. Features are categorized as small-scale (P3), medium-scale (P4), and large-scale (P5). Channel compression is applied to the fused features of each pyramid layer to ultimately obtain the multi-scale fused feature F for each pyramid layer. py,i =DynamicPrune(P i ,r py DynamicPrune(·) is the pyramid channel clipping operator, r py The compression ratio (e.g., r)py =0.25, indicating that 25% of redundant channels are removed from each layer), finally yielding the multi-scale fusion feature F of the N-layer pyramid. py,i ; S2.5: A pyramid multi-task detection head is set as the output of the MSA-CNN image defect detection processing model. N layers of pyramid multi-scale fusion features are input in parallel into the detection head module. The detection head has three independent convolutional regression branches: the first classification branch outputs the detection probability of the damage category, the second boundary regression branch outputs the pixel coordinates of the damage bounding box, and the third geometric regression branch outputs the geometric parameters of the damage. The mapping relationship between the input of a single-layer pyramid multi-scale fusion feature to the detection head is: {C i B i G i}=Head(F py,i ), {C i B i G i} represents the damage category, damage bounding box label, and damage geometric parameter label; Head is the pyramid multi-task detection head operator; after the multi-scale fusion features of the N-layer pyramid independently output the hierarchical loss detection results, the non-maximum suppression (NMS) algorithm (based on the intersection-union ratio of the detection boxes to remove the same damage repeatedly detected across the N-layer pyramid) is used to remove the same damage repeatedly detected across the scales, thus obtaining all independent damages that do not overlap within the image. Finally, the wind turbine blade image detection result with damage number J is output {C}. J B J G J} J=1 N2 N2 represents the total number of all detected independent defects, thus constructing an image defect detection and processing model; In this embodiment, it should be specifically noted that existing feature pyramid networks generally adopt a fusion method of top-down upsampling and residual addition. This basic fusion operator is a well-known technology in the field. Upsampling can be replaced by nearest neighbor interpolation, and fusion addition can increase the weight coefficient to adjust the contribution of multi-layer features, all of which can achieve multi-scale feature fusion effect.

[0017] S3: Based on a set of sample pairs labeled with standard wind turbine blades, forward image feature inference is performed to output defect detection results. The image feature processing model is trained using a weighted composite loss of boundary, classification, and morphology, adaptively adjusting the loss weights, and backpropagating to update the parameters of the image feature processing model, resulting in a trained image defect detection processing model, including: S3.1: Sample pair set D based on standard wind turbine blade labels all For each set of standard images I in the training subset std,m Structured Label m Sample pairs (I)std,m Label m Image-by-image forward feature extraction, extracting features from a single standard image I. std,m The input image defect detection and processing model is fed forward to perform image feature extraction, and the output is an N-layer pyramid multi-scale fusion feature F. py,i The single-sample detection result is output based on the pyramid multi-task detection head operator, including the detected damage category C. m y Detect damage bounding box label B m y and damage geometry parameter label G m y ; S3.2: Boundary regression loss L box Multi-class Focal loss L cls and damage morphology regularized loss L reg Three categories constitute the weighted composite loss L tot =∑ J=1 N2 [λ box (epoch)×L box,J +λ cls (epoch)×L cls,J +λ reg (epoch)×L reg,J}, J is a single image I std,m Damage number at position J, N2 is the number of image I std,m The total number of independent damages detected within the interior, L tot For the single-sample weighted composite total loss, λ box (epoch), λ cls (epoch), λ reg The weights are dynamically balanced adaptively with each training iteration (epoch), and automatically updated in each iteration, balancing the three optimization objectives of boundary, classification, and morphology preset. The damage morphology regularization loss sets a reasonable morphology range for each damage geometric detection parameter in the network output. When the damage detection geometric parameter exceeds the true defect morphology range, a penalty loss is applied to that damage separately to filter out false defect detections caused by leaf texture and light and shadow. The single-point damage morphology regularization loss is composed of at least the sum of aspect ratio constraint terms, single defect size constraint terms, and sheet defect area constraint terms. The aspect ratio constraint term is used to constrain long crack-type damage, the single defect size constraint term is used to constrain point-type pore-type damage, and the sheet defect area constraint term is used to constrain gel coat peeling-type damage. The single-point boundary regression loss L box,J Based on Wassstein distance, construct the boundary regression loss L box,J =1-WIoU(B m y Bm ), WIoU=IoU(B m y B m )×exp(-W WD (B m y B m ) / C diag IoU is the detection bounding box B. m y The truth bounding box B of the annotation m The intersection and union ratio, W WD To detect the 2D Wasserstein distance between the bounding box and the ground truth bounding box (adapting to the irregular boundaries of slender cracks and addressing the large regression bias of traditional IoU for narrow cracks), C diag The length of the diagonal pixels of the bounding rectangle of the image; the single-point multi-class Focal loss L cls,J , C represents the damage label category number, α C To balance the weights for the pre-defined damage categories (e.g., setting a low weight for the undamaged background category C=0 to reduce interference from a large number of blank leaf samples during training), p m,C Let y be the probability that the m-th sample belongs to class C in the network detection. m,C The damage category is represented by a one-hot label, with the label corresponding to the true category being 1 and the rest being 0. γ is a modulation factor (e.g., γ=2, reducing the gradient weight of easily classified samples and focusing training on difficult-to-identify small damage samples); the single-damage morphology regularization loss L... reg,J =L asp +L siz +L are L asp For the aspect ratio constraint term of strip damage, L asp =max(0,|L m y / W m y -R min |,|L m y / W m y -R max |), [R min ,R max [R] represents the reasonable range of length and width of a real wind turbine crack (such as a common crack). min ,R max = [0.8, 20], the aspect ratio constraint term for the lightning strike scratch is [R]. min ,R max ]=[1,40]), L siz For point-like single-unit defect size constraints, DC y The equivalent diameter of the dot-shaped pores (based on the equivalent circle diameter converted from the total pixel area of ​​the pore detection region output by the network), [D min D max [D] represents the reasonable range of equivalent diameters for point-like pore defects (e.g., [D]). min D max [3,30] pixels), L are For the sheet-like damage area constraint term, L are =max(0,(S m y -S max ) / (C diag ) 2 ,(S min -S m y ) / (C diag ) 2 ), [S min ,S max [S] represents a reasonable range for the area of ​​patchy damage (e.g., [S]). min ,S max [10, 1200] pixels, the network outputs a binary mask of peeling damage, where a pixel value of 1 in the mask represents the peeling area. The total number of foreground pixels is directly counted to obtain S. m y The above-mentioned single-site damage morphology regularization loss is applicable to cracks, pores, and gel coat peeling damage, and is a preferred defect embodiment for wind turbine blades. The damage morphology regularization loss of this invention is not limited to these three types of damage: for other wind turbine damage such as lightning strike bifurcation scratches, impact dents, and surface corrosion, it is only necessary to add corresponding exclusive constraint terms and match the true morphology range of such defects to achieve the same type of morphology constraint function; the constraint terms only need to follow the core logic of "applying penalty when the detected geometric parameters exceed the true defect range"; S3.3: After each training iteration, based on the mean L of the three types of loss in the current training subset... box avg L cls avg L reg avg Dynamically update the balance weight λ box (epoch+1), λ cls (epoch+1) and λ reg (epoch+1), λ box (epoch+1)=λ box (epoch), λ cls (epoch+1)=λ cls (epoch)×L box avg / Lcls avg , λ reg (epoch+1)=λ reg (epoch)×L box avg / L reg avg Using the boundary regression loss weights as a fixed benchmark, the classification and morphological regularization weights are adaptively scaled by the ratio of the two loss means to balance the differences in the gradient magnitudes of the three loss classes. S3.4: Total composite damage L of a single sample tot For the objective function, construct the gradient descent update rule, and the single-sample parameter update formula is as follows: θ represents all trainable weight parameters of the network (LASC convolution, MECA attention, all parameters of the pyramid fusion layer, etc.), and η represents the training learning rate. The gradient of the total loss per sample with respect to the network parameters is used. The entire training subset is iterated through repeatedly, performing forward inference, loss calculation, and gradient update until the loss converges, resulting in the trained image defect detection and processing model MSA-CNN. The loss convergence criterion is as follows: Multiple complete training cycles are executed, and the global average total loss L of the training subset is calculated after each epoch. tot avg Meanwhile, the average total loss L of the validation set is calculated separately. va avg The training iteration is terminated if the following two conditions are met: (1) The global average total loss of the training set decreases by less than the threshold ε (value range 10) for TT consecutive rounds (value range 5 to 20, such as TT=10, which can be adjusted according to the dataset and operation and maintenance standards). -3 ~10 -5 For example, ε=10 -4 (2) The average total loss of the validation set no longer decreases for TT consecutive rounds (all are greater than or equal to the validation loss of the previous round), and there is no overfitting phenomenon. In this embodiment, it is necessary to specifically explain that after the model training converges, the test subset D, which has been partitioned by S1, is read. te The S4 inference process is executed on each image in the test subset. Based on the detection results of the test set, the damage recognition accuracy, false negative rate, and false positive rate are calculated to complete the offline evaluation of the overall recognition performance of the model. The accuracy rate is the proportion of samples that are correctly detected out of all detected defect samples. The false negative rate is the proportion of samples that are not detected out of all samples with real damage. The false positive rate is the proportion of samples with no damage background that are incorrectly identified as defects out of all samples with no damage. The offline evaluation of the overall recognition performance of the model is completed.

[0018] S4: Input the preprocessed image of the standard wind turbine blade to be tested into the trained image defect detection and processing model, perform image feature extraction, and output the identification results of all independent damages detected on the wind turbine blade; first, read the optimal network weights θ after training. * The preprocessed image of the standard wind turbine blade to be tested is I std te Input the trained image defect detection and processing model MSA-CNN, perform image feature extraction forward, and output the identification results of all independent damages detected on the wind turbine blades, including damage category, bounding box coordinates of lost pixels, and damage geometric parameter feature set {C}. std,J y B std,J y G std,J y |} J=1 N2 =MSA-CNN(I std te ,θ * J is the damage number at the Jth location, and N2 is the total number of all independent damages detected, thus completing the damage identification of the standard wind turbine blade under test. This embodiment provides a preferred embodiment and a comparative experiment with existing traditional wind turbine blade damage identification schemes; Comparison object 1: Manual high-altitude inspection; Comparison object 2: General YOLOv5 (traditional CNN / FPN scheme); Comparison object 3: The MSA-CNN wind turbine blade damage identification scheme of this invention; Test dataset: 12,000 640×640 drone aerial images of wind turbine blades, including 5 types of damage: cracks, lightning strike scratches, gel coat peeling, and micropores, including complex working conditions such as backlighting, clouds and fog, mountain backgrounds, and blade shadows; Core evaluation indicators: micro-damage false negative rate, defect false positive rate, single image inference time, average recognition accuracy, percentage of invalid alarms during maintenance downtime, and duration of a single inspection operation; The comparative experimental data are as follows:

[0019] Secondly: The accompanying drawings of the embodiments disclosed in this invention only involve the structures involved in the embodiments disclosed in this invention. Other structures can refer to the general design. In the absence of conflict, the same embodiment and different embodiments of this invention can be combined with each other. In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A wind turbine blade damage identification method based on an improved convolutional neural network, characterized in that: include: S1: Acquire the original aerial images of wind turbine blades, perform wavelet morphological denoising and blade foreground mask semantic segmentation image preprocessing on the original images to obtain standard wind turbine blade images and label them, and output a set of sample pairs of standard wind turbine blade labels; S2: Based on standard wind turbine blade images, perform multi-level image feature processing. Through the first-layer downsampling convolution, followed by multi-layer lightweight convolution processing, multi-layer attention processing, and multi-layer pyramid processing, extract multi-scale feature maps for identifying slender and minute damage to wind turbine blades. Combined with the non-maximum suppression algorithm, obtain the wind turbine blade image detection results and construct an image defect detection and processing model. S3: Based on the sample pair set of standard wind turbine blade labels, perform forward image feature inference to output defect detection results. Train the image feature processing model by constraining the three-class weighted composite loss of boundary, classification and morphology, adaptively adjust the loss weight, backpropagate to update the parameters of the image feature processing model, and obtain the trained image defect detection processing model. S4: Input the preprocessed image of the standard wind turbine blade to be tested into the trained image defect detection and processing model, perform image feature extraction forward, and output the wind turbine blade image defect detection result.

2. The wind turbine blade damage identification method based on an improved convolutional neural network according to claim 1, characterized in that: The implementation of S2 includes: S2.1: Initial convolution of the first layer: converting a single standard image I... std,m Input first convolutional layer Conv k1,s Perform feature upsampling and downsampling to generate the first-layer basic feature F0=Conv k1,s (I std,m k1 is the kernel size, and s is the sliding step. S2.2: Multi-layer lightweight convolution processing: Then, N layers of LASC lightweight asymmetric sparse convolution modules are executed cyclically. Each module evenly divides the input feature channels and uses long horizontal and vertical asymmetric convolutions to extract thin, elongated damaged textures. After fusion, sparse channel cropping is performed to reduce the number of parameters. The mapping relationship of a single-layer LASC lightweight asymmetric sparse convolution block is: F t =LASC(F t-1 ,r sp ), F t-1 r sp These represent the input features of the previous layer, the sparse channel pruning ratio, and F. t To obtain the output features of this layer, the LASC block operation is performed N times in a loop, resulting in F1, F2, ..., F... N Layer-by-layer downsampling of basic feature maps.

3. The wind turbine blade damage identification method based on an improved convolutional neural network according to claim 1, characterized in that: The implementation of S2 also includes: S2.3: Multi-layer attention processing module: outputting feature F for each LASC layer. t Independent input to the MECA multi-layer cross-scale attention operator MECA(·), output enhanced features F t zq =MECA(F t N layers output features F t Perform the calculations separately and output F1. zq_1 F2 zq_2 ,...,F N zq_N Layer enhancement features; S2.4: Multi-layer pyramid processing: A feature pyramid is constructed from the enhanced features obtained through multi-layer attention processing. A new ultra-high resolution P2 layer is added to capture pixel-level minute defects within the image. Cross-scale feature fusion is achieved by combining top-down upsampling with lateral residual addition, resulting in the fused feature P for each pyramid layer. i =F2 zq_i +ResizeUp(P i+1 size(F2) zq_i ResizeUp is a bilinear interpolation upsampling algorithm. Each enhanced feature corresponds one-to-one with each level of the pyramid, and the top level of the pyramid equals F2. zq_N No upper-level input.

4. The wind turbine blade damage identification method based on an improved convolutional neural network according to claim 1, characterized in that: The S3 implementation includes: a sample pair set D based on standard wind turbine blade labels. all For each set of standard images I in the training subset std,m Structured Label m Sample pairs (I) std,m Label m Image-by-image forward feature extraction, extracting features from a single standard image I. std,m The input image defect detection and processing model is fed forward to perform image feature extraction, and the output is an N-layer pyramid multi-scale fusion feature F. py,i The single-sample detection result is output based on the pyramid multi-task detection head operator, including the detected damage category C. m y Detect damage bounding box label B m y and damage geometry parameter label G m y .

5. The wind turbine blade damage identification method based on an improved convolutional neural network according to claim 1, characterized in that: The implementation of S3 also includes: S3.2: the boundary regression loss L box Multi-class Focal loss L cls and damage morphology regularized loss L reg Three categories constitute the weighted composite loss L tot The damage morphology regularization loss sets a reasonable morphology range for the damage geometry detection parameters output by the network, and applies a penalty loss when the detection geometry parameters exceed the actual defect morphology range.

6. The wind turbine blade damage identification method based on an improved convolutional neural network according to claim 1, characterized in that: The implementation of S3 also includes: S3.3: After each training iteration, based on the mean L of the three types of losses in the current training subset... box avg L cls avg L reg avg Dynamically update the balance weight λ box (epoch+1), λ cls (epoch+1) and λ reg (epoch+1); S3.4: Total composite damage L of a single sample tot For the objective function, construct the gradient descent update rule, and the single-sample parameter update formula is as follows: θ represents all trainable weights in the network, and η represents the training learning rate. The gradient of the total loss for a single sample is given by the network parameters. The entire training subset is iterated through repeatedly, and the forward inference, loss calculation, and gradient update processes are repeated until the loss converges, resulting in the trained image defect detection and processing model MSA-CNN.

7. The wind turbine blade damage identification method based on an improved convolutional neural network according to claim 1, characterized in that: The S4 implementation includes: first, reading the optimal network weights θ after offline training is completed. * The preprocessed image of the standard wind turbine blade to be tested is I std te The trained image defect detection and processing model MSA-CNN is input, performs image feature extraction forward, and outputs the defect detection results of the wind turbine blade image, including the damage category, the coordinates of the lost pixel bounding box, and the damage geometric parameter feature set {C}. std,J y B std,J y G std,J y |} J=1 N2 =MSA-CNN(I std te ,θ * J is the damage number at the Jth location, and N2 is the total number of independent damages detected, thus completing the damage identification of the standard wind turbine blade under test.

8. A wind turbine blade damage identification system based on an improved convolutional neural network, used in the wind turbine blade damage identification method based on an improved convolutional neural network as described in any one of claims 1-7, characterized in that, include: Wind turbine blade image acquisition module: Acquires original aerial images of wind turbine blades, performs wavelet morphological denoising and blade foreground mask semantic segmentation image preprocessing operations on the original images to obtain standard wind turbine blade images and label them, and outputs a set of sample pairs of standard wind turbine blade labels; Image Defect Detection and Processing Model Construction Module: Based on standard wind turbine blade images, multi-level image feature processing is performed. Spatial compression and channel dimensionality enhancement are completed through the first-layer downsampling convolution. Then, through multi-layer lightweight convolution processing, multi-layer attention processing, and multi-layer pyramid processing, multi-scale feature maps for identifying slender and minute damage to wind turbine blades are extracted. Combined with the non-maximum suppression algorithm, the wind turbine blade image detection results are obtained, and the image defect detection and processing model is constructed. Offline model weighted composite loss training module: Based on the sample pair set of standard wind turbine blade labels, it performs forward image feature inference to output defect detection results. It trains the image feature processing model by constraining the three categories of boundary, classification, and morphology weighted composite loss, adaptively adjusts the loss weights, and backpropagates to update the parameters of the image feature processing model to obtain the trained image defect detection and processing model. Online wind turbine blade damage output module: Input the preprocessed image of the standard wind turbine blade to be tested into the trained image defect detection and processing model, perform image feature extraction forward, and output the wind turbine blade image defect detection result.

Images