Fan blade inner cavity defect detection method based on image recognition and deep learning

Abstract

The application discloses a fan blade inner cavity defect detection method based on image recognition and deep learning, and particularly relates to the technical field of fan blade inner cavities. First, an original image set of a fan blade inner cavity is collected and preprocessed to obtain an original inner cavity standard image set. A layered labeling training data set is constructed based on a conditional generative adversarial network. A defect detection network is constructed to obtain a defect detection result. A three-stage progressive differentiated training strategy is used to train the defect detection network, and finally a trained fan inner cavity defect detection network is output. An image of a fan blade inner cavity to be detected is input into the trained defect detection network after preprocessing to obtain a defect detection result. Through the defect detection network construction step, the application combines layered feature fusion and prior parameter embedding and an adaptive noise suppression mechanism to effectively overcome the problems of inner cavity dust noise interference and weak micro-defect features, and reduce the missed detection rate and the false detection rate.

Description

Technical Field

[0001] This invention relates to the field of wind turbine blade internal cavity technology, specifically a method for detecting defects in wind turbine blade internal cavities based on image recognition and deep learning. Background Technology

[0002] During long-term operation, the internal structure of wind turbine blades is prone to internal defects such as cracks, delamination, bubbles, and degumming. If the internal defects are characterized by strong concealment, small size, irregular distribution, complex background texture, and serious dust interference, they will gradually expand and cause blade cracking and breakage if they cannot be detected and repaired in time, which will greatly reduce the safety and service life of the wind turbine.

[0003] Existing detection technologies include manual endoscopic inspection and deep learning detection methods. Manual endoscopic inspection relies on operator experience, which is highly subjective, has a high false negative rate, and low detection efficiency. Furthermore, it poses safety hazards in high-risk operation scenarios. Conventional deep learning detection methods rely on massive amounts of real defect samples. However, collecting real defect samples from the inner cavity of wind turbine blades is difficult, the number of samples is scarce, and the sample balance is poor. Small defects are easily covered by background noise. At the same time, existing detection networks have single feature extraction methods and cannot take into account both shallow detail features and deep semantic features. They have low recognition accuracy for low-contrast, small-sized, and difficult defects. There are distributional differences between defect images synthesized by generative adversarial networks and real images. Directly using these images for training will lead to a decrease in the generalization performance of the model in actual detection. Moreover, they lack targeted noise suppression and sample layering training strategies, making it difficult to adapt to the complex detection environment of the inner cavity of wind turbine blades.

[0004] Therefore, in the existing methods for detecting defects in the internal cavity of wind turbine blades, there are problems such as harsh internal cavity imaging environment, limited image quality, difficulty in obtaining defect samples, severe data scarcity, and insufficient adaptability to the internal cavity environment. There is an urgent need for a wind turbine blade internal cavity monitoring method that can effectively cope with the complex internal cavity imaging environment, solve the problem of sample scarcity, and improve the detection accuracy of small defects. Summary of the Invention

[0005] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide a method for detecting defects in the internal cavity of wind turbine blades based on image recognition and deep learning, in order to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a method for detecting internal defects in wind turbine blades based on image recognition and deep learning, comprising: S1: Collect the original image set of the inner cavity of the wind turbine blade and perform preprocessing to obtain the original standard image set of the inner cavity composed of preprocessed defect images and preprocessed defect-free images; S2: Based on conditional generative adversarial networks, a set of internal cavity defect samples is generated from the original standard internal cavity image set, and all image data are labeled, managed and divided to construct a hierarchical labeled training dataset; S3: Construct a defect detection network, which consists of a shallow feature extraction branch, a deep semantic feature extraction branch, a hierarchical feature fusion module, an adaptive noise suppression module, and a detection head cascaded in sequence to obtain the defect detection results; S4: A three-stage progressive differential training strategy is adopted to train the defect detection network. Differentiated sample ratios, loss functions and parameter training strategies are used in each stage. Finally, the trained wind turbine cavity defect detection network is output. S5: The image of the inner cavity of the wind turbine blade to be detected is preprocessed and then input into the trained defect detection network to obtain the defect detection result.

[0007] The technical effects and advantages of this invention are as follows: 1. This invention, through the defect sample generation and management steps, adds a conditional generative adversarial network with defect morphology regular constraints to achieve controllable generation of defect morphology, size, and location, accurately expands scarce micro-defect samples, solves the industry pain points of difficult, insufficient, and unbalanced collection of real defect samples in the inner cavity of wind turbine blades, and greatly improves the model's generalization ability. 2. This invention, through the defect detection network construction steps, takes into account both shallow detailed features and deep multi-scale semantic features. It combines hierarchical feature fusion and prior parameter embedding with an adaptive noise suppression mechanism to effectively overcome the problems of internal dust noise interference and weak features of minute defects. This significantly improves the detection accuracy of cracks, micropores, and minor peeling defects, and reduces the false detection rate and false detection rate. 3. The present invention adopts a three-stage progressive differential training strategy through the defect detection network training steps. By adjusting the sample ratio, learning rate and loss function in stages, the model parameters are optimized step by step. First, basic features are learned, then the recognition of difficult samples is optimized, and finally the high-order detection capability is fine-tuned. This adapts to the distribution characteristics of easy and difficult samples, and significantly improves the adaptability and stability of the model to complex internal cavity scenarios. 4. This invention fully automates image preprocessing, sample generation, model training, and defect detection, improving detection accuracy and anti-interference capabilities. It requires no manual intervention, avoiding the problem of low efficiency in manual detection. Its detection efficiency is higher than that of manual detection and traditional machine vision detection methods, making it suitable for batch and routine operation and maintenance detection scenarios of wind turbine blades. Attached Figure Description

[0008] Figure 1 This is a schematic diagram of the overall process of the present invention.

[0009] Figure 2 This is a schematic diagram of the method flow of the present invention.

[0010] Figure 3 This is a schematic diagram illustrating the process of constructing the hierarchical labeled training dataset of the present invention.

[0011] Figure 4 This is a schematic diagram of the defect detection network construction process of the present invention. Detailed Implementation

[0012] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0013] Please see Figure 1 As shown, the present invention provides a wind turbine blade internal cavity defect detection system based on image recognition and deep learning, including an image acquisition and preprocessing module, a defect sample generation and management module, a defect detection network construction module, a defect detection network training module, and a defect detection output module.

[0014] The image acquisition and preprocessing module is connected to the defect sample generation and management module, and the defect detection network training module is connected to the defect sample generation and management module, the defect detection network construction module, and the defect detection output module, respectively.

[0015] Image acquisition and preprocessing module: Acquires original images of the inner cavity of the wind turbine blades and performs preprocessing to obtain a set of original standard images of the inner cavity composed of preprocessed defective images and preprocessed defect-free images; Defect Sample Generation and Management Module: Based on conditional generative adversarial networks, it generates a set of cavity defect samples from the original standard cavity image set, and performs annotation management and division on all image data to construct a hierarchical annotation training dataset; Defect detection network construction module: Construct a defect detection network, which consists of shallow feature extraction branch, deep semantic feature extraction branch, hierarchical feature fusion module, adaptive noise suppression module, and detection head cascaded in sequence to obtain defect detection results; Defect Detection Network Training Module: A three-stage progressive differential training strategy is adopted to train the defect detection network. Differentiated sample ratios, loss functions and parameter training strategies are used in each stage, and the trained wind turbine cavity defect detection network is finally output. Defect detection output module: The image of the inner cavity of the wind turbine blade to be detected is preprocessed and then input into the trained defect detection network to obtain the defect detection result; Please see Figure 2As shown, the wind turbine blade internal cavity defect detection method based on image recognition and deep learning includes: S1: Acquiring a set of original images of the wind turbine blade internal cavity and preprocessing them to obtain a set of original standard internal cavity images composed of preprocessed defective images and preprocessed defect-free images; S2: Generating a set of internal cavity defect samples based on the original standard internal cavity image set using a conditional generative adversarial network, and labeling, managing, and classifying all image data to construct a hierarchical labeled training dataset; S3: Constructing a defect detection network, which consists of a shallow feature extraction branch, a deep semantic feature extraction branch, a hierarchical feature fusion module, an adaptive noise suppression module, and a detection head cascaded sequentially to obtain defect detection results; S4: Training the defect detection network using a three-stage progressive differential training strategy, with each stage employing a differential sample ratio, loss function, and parameter training strategy, and finally outputting the trained wind turbine internal cavity defect detection network; S5: Inputting the preprocessed image of the wind turbine blade internal cavity to be detected into the trained defect detection network to obtain defect detection results.

[0016] S1: Acquire the original image set of the wind turbine blade's internal cavity and preprocess it to obtain the original standard image set of the internal cavity, which consists of preprocessed defect images and preprocessed defect-free images; convert the original image set of the wind turbine blade's internal cavity into a fixed-size grayscale internal cavity grayscale defect image subset U. def Set and grayscale defect-free image subset U good The images are sequentially processed using adaptive median filtering for noise reduction, CLAHE contrast enhancement, and gradient edge enhancement to obtain the preprocessed original standard intracavitary image set S=S def ∪S good S def and S goodThe images are divided into a pre-processed defect image subset and a defect-free image subset. First, using an imaging device (industrial endoscope camera), images of the wind turbine blade's internal cavity at different angles and positions are acquired. These images include defective images (cracks, pores, material spalling) and defect-free images. The defects include three typical types: wind turbine blade internal cavity cracks, pores, and material spalling. It can also be extended to identify secondary internal cavity defects such as insufficient glue, internal delamination, and localized resin defects. In a preferred embodiment, images are acquired according to the blade's internal cavity structure, divided into four main acquisition locations: the leading edge cavity section, the web bonding section, the trailing edge cavity section, and the irregularly shaped area at the internal cavity corner. Each region is spaced 1 unit apart along the internal cavity axis. A data acquisition point is set up every 50-300mm. Each point acquires images of good products without defects and images of products with defects such as cracks, pores, or material peeling. Multi-angle acquisition specifications: Each acquisition point is set with three shooting angles: including the frontal view (camera optical axis perpendicular to the inner wall of the cavity (0° frontal view)), the tilt / slant view (optical axis tilted 25°-40° relative to the normal of the inner wall), and the lateral deflection angle (horizontal lateral deflection 20°-35°). This eliminates surface reflection and inner wall shadow occlusion, and acquires images of the same type of defects under different perspective deformations. It simulates the imaging distortion of a random shot taken by an on-site inspection robot inserted into the inner cavity, allowing GAN to generate samples and the detection network to adapt to the actual multi-angle images to be tested. In this embodiment, the adaptive median filtering window range is 3×3 to 7×7, and the local variance discrimination threshold is 20 to 40. CLAHE uses 16×16 image blocks and histogram cropping is limited to 30 to 50. The edge gradient enhancement coefficient is 0.2 to 0.5. After three levels of processing, salt-and-pepper noise is eliminated, the defect edge recognition is improved, and a preprocessed standard image set is output as the cavity background input for the subsequent conditional generative adversarial network.

[0017] Please see Figure 3 As shown, S2: Based on a conditional generative adversarial network, a set of intracavitary defect samples is generated from the original standard intracavitary image set. All image data is labeled, managed, and partitioned to construct a hierarchical labeled training dataset, including: S2.1: From the preprocessed defect-free image set S good Selecting single intracavitary images I in sequence bg,i As a background base for generation, for a fixed background image I bg,i Three types of defects were selected: cracks, pores, and material spalling. Multiple sets of two-dimensional spatial coordinates Pos were assigned to each defect with different center pixel positions. k =(x0,y0) and the differential defect morphology parameter P def,k Each set of parameters corresponds one-to-one with a single sample generation task. The preset defect morphology parameters include the crack aspect ratio R. l Crack overlap coefficient O l Pore ​​equivalent diameter D s(Unit: pixels) and material peeling area threshold S t (Unit: pixels); Each group (Pos) k ,P def,k A conditional generative adversarial network (CGAN) is input to a fixed background image. The CGAN network consists of a generator G and a discriminator D, and the ternary combination {I} is used as input. bg,i Pos k ,P def,k Input generator G, generator mapping relationship satisfies: I bg,k h =G(I bg,i Pos k ,P def,k Output a synthetic defect image I bg,k h For the preprocessed defect image subset S def Real defect image I re,i Its corresponding condition vector is c i Including background image I re,i The spatial location of the defect and the manually annotated morphological parameters of the defect are used for the synthetic defect image I. bg,k h Its corresponding condition vector is c k Including background image I bg,i Defect spatial coordinates Pos k and preset defect morphology parameter P def,k The input to discriminator D is an "image-condition" pair, i.e., {I re,i ,c i} and {I bg,k h ,c k The output is the true / false probability D(I,c) of the "image-condition" pair (the condition vector is aligned with the image input using channel concatenation: the condition vector is mapped to a feature vector with the same number of image channels through a multilayer perceptron, then expanded to the same spatial size as the image, and finally concatenated with the image in the channel dimension to form the complete input of the discriminator); a defect morphology regularization constraint term is added to the basic adversarial loss of the discriminator to obtain the total loss L of the discriminator. D =L adv D +λ×L reg L adv D The discriminator's basic adversarial loss (a well-known basic loss, used to distinguish between real cavity defect images and generated defect images) is used. D() represents the true / false probability of the "image-condition" pair output by the discriminator, p def p represents the distribution of real defect images and their annotation conditions. denThe distribution of the synthesized defect image and its generation conditions is given by λ, which is the morphological regularization constraint weight coefficient (typically λ takes a value of 0.8 to 1.2). reg For defect morphological regularization constraint loss, L reg =w1×|R l -R l,s | / (R l,max + ε)+w2×|O l -O l,s | / (O l,max + ε)+w3×|D s -D s,s | / (D s,max + ε)+w4×|S t -S t,s | / (S t,max + ε), R l,s O l,s D s,s and S t,s Synthetic defect image I bg,i h Corresponding to the measured parameters of the wind turbine, R l,max O l,max D s,max and S t,max These represent the maximum values ​​of the corresponding parameters (obtained from the preset / real defect parameters of all samples), and ε is a very small positive number (e.g., 10). -6 ), avoid a denominator of 0 (for synthetic defect images I) bg,k h Defect region binary mask segmentation is performed to obtain defect region mask binary image Mask: defect pixel = 1, inner cavity background pixel = 0, obtained by statistical calculation of pixel coordinates and pixel count of defect mask; wherein: the actual length-to-width ratio of crack is calculated based on the pixel size of the minimum bounding rectangle of crack; the crack overlap coefficient is calculated by the ratio of the pixel area of ​​the overlapping area of ​​multiple cracks to that of a single crack; the equivalent diameter of pores is converted from the total pixels of the pore area; the total number of pixels of the peeling area is directly used as the peeling measured area), w1, w2, w3 and w4 are the corresponding constraint weights (in this implementation, w1=0.4, w2=0.2, w3=0.2 and w4=0.2), the greater the deviation between the generated defect parameters and the preset parameters, the L reg The higher the loss value, the more automatically the network iterates to correct the generated defect morphology and size, thus achieving controllable defect generation. S2.2: According to the total loss L of the discriminator D Total generator loss L G Alternately iterate the training of the generator and discriminator, L G =L advG +λ×L reg L adv G To help the generator combat loss, Record the total discriminant loss L in each iteration. D t And generator loss L G t t represents the iteration round. Convergence is determined by the relative rate of change of loss over t0 consecutive rounds, where the relative rate of change η(L) = |L t -L t-t0 | / (|L t-t0 |+ε), L t For L in round t D or L G In this embodiment, the number of consecutive statistical rounds t0 = 20, and the convergence threshold η(L) is used. th 10 -3 When the relative rates of change corresponding to the discriminator loss and the generator loss are simultaneously less than the convergence threshold η(L) for t0 consecutive rounds. th Once the network converges, the iteration stops. After convergence, without changing the background image, the defect generation coordinates and morphological parameters such as crack aspect ratio, crack overlap coefficient, pore equivalent diameter, and material spalling area threshold are continuously adjusted to obtain batch simulation generation I of defects of different locations and specifications on the same internal cavity background image. bg,k h =G(I bg,i Pos k ,P def,k ), k=1,2,...,K, where K is the total number of parameter configuration groups corresponding to a single background image, I bg,k h The synthesized defect image is generated for the k-th set of parameters; after traversing all parameter sets corresponding to a single background, a new defect-free background image is generated cyclically, and finally, all background images are collected into a set S. good The defect image sample set S is obtained. gen , ; S2.3: For real defect images I re,i ∈S def Labels are generated manually, including the defect type (crack / porosity / material spalling), the outer boundary of the defect pixels, and the defect geometry, forming a set of labeled real defect samples S. def y ; Targeting the generated defect image sample set S gen The images in the dataset are generated based on the input spatial coordinates Pos corresponding to each group of generated images. k With the preset defect morphology parameter P def,kThe defect bounding box pixel coordinates are converted, and preset parameters are directly used as geometric dimension annotations. The defect type is determined by the currently enabled defect morphology parameters (crack aspect ratio R is enabled simultaneously). l Crack overlap coefficient O l The defect was initially identified as a crack; the equivalent diameter D of the pores was then used separately. s The defect is identified as porosity; the material peeling area threshold S is activated separately. t When a defect is identified as material spalling, unused parameters are set to empty; images of defect-free cavities are uniformly labeled as defect-free samples, and all labeled real defect samples S are... def b Generate defect sample S gen b Defect-free sample S good b Merging them to form the full original labeled dataset S all For generating defective image samples, the outer boundary of the defective pixels is: generated from the center point Pos of the defective image. k =(x0,y0), the outer boundary of the defect is calculated based on the preset defect morphology parameters. First, the crack is formed by the center (x0,y0) and the aspect ratio R. l Based on the preset baseline pixel width W0, the actual crack length L=R is calculated. l ×W0, automatically calculates the circumscribed boundary (x min ,y min ,x max ,y max ), x min y min These are the pixel coordinates of the top-left corner of the defect bounding box, x max y max These are the pixel coordinates of the bottom right corner, x min =x0-L / 2, x max =x0+L / 2, y min =y0-W0 / 2, y max =y0+W0 / 2; then the pore size, based on the equivalent diameter D s With (x0, y0) as the center, the side length of the circumscribed square is D. s The coordinates of the pore frame are automatically obtained; finally, the material peels off, based on the preset area S. t The equivalent circumscribed side length is calculated, and the bounding box of the spalling area is generated by combining it with the center coordinates; the defect geometry is defined as follows: the crack dimension is the crack aspect ratio R. l Crack overlap coefficient O l The pore size is indicated by the equivalent pore diameter D. s The dimension of the material peeling is the preset area S. t ; S2.4: Let the total number of pixels in a single image be N.all (Obtained by multiplying the image width pixels by the height pixels), the total number of defective pixels N in a single image def (Defect pixels refer to foreground pixels that belong only to the main defect areas of cracks, pores, and material spalling, i.e., defect pixels with a value of 1 in the binary mask of the defect area), according to the proportion of defect pixels R pix =N def / N all The full original labeled dataset S all Divide into simple sample subsets S easy and difficult sample subset S hard If R pix >Pixel percentage threshold R pix,th (e.g., 0.005), then the single image I bg,i h Belongs to a simple sample subset S easy (Defects account for a large proportion and have obvious characteristics, such as large-area peeling and wide cracks), if R pix ≤R pix,th Then the single image I bg,i h Belonging to the difficult sample subset S hard (Minor defects, with very few defect pixels, easily masked by dust and noise, such as micropores and fine cracks); finally, a hierarchical labeled training dataset S1 is constructed. all =S easy ∪S hard The dataset is divided into training, validation and test sets according to a set ratio (usually 7:1:2). Please see Figure 4 As shown, S3: Construct a defect detection network, which consists of a shallow feature extraction branch, a deep semantic feature extraction branch, a hierarchical feature fusion module, an adaptive noise suppression module, and a detection head cascaded sequentially, to obtain the defect detection results, including: S3.1: Shallow feature extraction branch: Extracts the preprocessed standard intracavitary image I... pre Each feature is fed into a shallow feature extraction branch, and then a series of 3×3 convolutional layers are stacked sequentially. 3×3 Extracting shallow detail feature maps F sh (Focusing on crack edges, pore contours, material spalling boundaries, etc.), F sh =Conv 3×3 (I pre )={F sh 1 ,F sh 2 ,...,F sh n}, where n is the total number of feature levels in the shallow branches (a hyperparameter pre-defined in the network), and the preset level of shallow features F is output through short-connection bypasses across branches. de The deep semantic feature extraction branch compensates for the loss of details caused by deep feature downsampling. de =Select({F sh i}), Select(·) is a hierarchical filtering operation, discarding some shallow detail layers and using only medium-scale shallow features as the starting input for deep branches to reduce redundant computation; the preset hierarchical filtering rules are as follows: 1. The shallow branches have a total of n convolutional feature layers, sorted from high to low according to feature map resolution: the first n / 3 layers are micro-scale detail layers (too high resolution, too much noise redundancy, directly discarded), the last n / 3 layers are large-scale coarse contour layers (too low resolution, severe loss of detail, directly discarded), and the middle n / 3 layers are retained as medium-scale shallow features; if the shallow features The feature extraction branch consists of four layers (n=4), corresponding to feature maps of the input image with downsampling factors of 2, 4, 8, and 16, respectively. The first n / 3=1 layer (rounded down, downsampling factor 2) is a micro-scale detail layer, and the last n / 3=1 layer (rounded down, downsampling factor 16) is a large-scale coarse contour layer, both of which are discarded. The middle two layers (downsampling factors of 4 and 8, corresponding to i=2 and i=3 levels) are retained as the starting input for the deep branches. This selection rule can be adjusted according to the actual network architecture and computing resources, and the optimal configuration can be obtained through experiments. S3.2: Deep semantic feature extraction branch: It consists of a series of inverse bottleneck modules and heterogeneous convolutional modules. First, the inverse bottleneck module extracts shallow features F through the shallow branch's cross-branch bypass. de As input, channel compression processing is performed to obtain the compressed feature F. com Based on compression feature F com Calculate the channel attention weight matrix, and use the attention weight matrix to apply F. com Weighted optimization yields feature F att and F att Input heterogeneous convolutional module; for shallow features F de Perform a channel compression operation with an inverted bottleneck, and the intermediate feature F is obtained after compression. com =Compress(F de Compress(·) is a channel compression mapping operation (usually implemented by 1×1 convolution to reduce the number of channels), which reduces the dimensionality of the feature channels according to a set compression ratio (the channel compression ratio ranges from 1:3 to 1:5) to obtain the compressed feature F. com Based on compression feature F com Obtain the channel attention weight matrix A c And then with Fcom The final output feature F of the inverse bottleneck module is obtained by multiplying each channel. att A c =σ(FC2(δ(FC1(GAP(F com ))))), GAP() is the global average pooling element, FC1 and FC2 are two fully connected mapping layers, δ is the activation function (such as ReLU), and σ is the sigmoid activation function. For feature-channel multiplication, F att The compressed feature map is optimized by attention weighting; then, a heterogeneous convolutional module is configured in parallel with at least one large-size convolutional kernel and at least one small-size convolutional kernel to obtain the deep multi-scale semantic feature map F. deep In a preferred embodiment, the attention-weighted optimized compressed feature map F att The input is fed into a heterogeneous convolution module, where multi-scale convolution kernels are configured in parallel, including large-size dilated convolutions with dilation coefficients of 2, 3, and 4, and small-size dense convolutions of 1×1, 2×2, and 3×3. This enables multi-receptive field feature extraction, adapting to the differentiated feature requirements of large-size peeling, medium-sized cracks, and micropores in the wind turbine's inner cavity, resulting in a deep multi-scale semantic feature map F. deep F deep =Concat(Conv d=2 (F att ),Conv d=3 (F att ),Conv d=4 (F att ),Conv 1×1 (F att ),Conv 2×2 (F att ),Conv 3×3 (F att Concat is a concatenation operation performed on the feature channel dimensions; S3.3: Hierarchical Feature Fusion Module: Divided into surface fusion unit and high-level fusion unit, which perform differentiated fusion processing on shallow detail features and deep semantic features respectively. First, the surface fusion unit, based on the shallow detail feature map F... sh Feature aggregation is performed using pixel-weighted summation to obtain the detailed feature map F after surface fusion. surf , a i The pixel fusion weights for the corresponding feature layer (the weights are learnable parameters of the network, adaptively and iteratively optimized during backpropagation of training after model initialization) satisfy ∑a i =1; then the high-level fusion unit, based on the deep multi-scale semantic feature map F deep The spatial attention weight matrix A is generated through the spatial attention module.s Then A s With F deep Multiplying by spatial location yields the enhanced high-level semantic feature map F. high , (The spatial attention module adopts a convolution + sigmoid structure, with F...) deep As input, the input is sequentially processed through global average pooling, a 1×1 convolution, and a sigmoid activation function to generate channel attention weights; then through a 3×3 convolutional layer and a sigmoid activation function to generate spatial attention weights; finally, the detail feature map F is processed. surf and high-level semantic feature map F high After uniformly scaling to the same height and width dimensions (which can be obtained using the well-known bilinear interpolation resize operation), we obtain F respectively. surf q and F high q The total feature F is obtained by pixel-by-pixel addition and fusion. fusi F fusi =F surf +F high ; S3.4: Adaptive Noise Suppression Module: A noise suppression module is inserted into the feature fusion path, in the fused feature F fusi Slide an upward local calculation window of size M×N, and calculate the characteristic local variance σ within the window. lo 2 =[∑(F(x,y)-F avg ) 2 ] / (M×N), F(x,y) is the feature value of a single position within the window, F avg The variance is the mean of all features within the current window. A larger variance indicates more drastic pixel fluctuations and stronger dust noise in that region. Based on the local variance σ... lo 2 Adaptive solution for bilateral filter window size W fil =W min +η×(σ lo 2 / σ lo,max 2 ), W min The preset minimum reference window for filtering is η, which is the fixed window adjustment coefficient (per unit pixel, the optimal fixed pixel value is determined in advance on the validation set through grid search, and in this embodiment, η = 2 pixels), σ lo,max 2 The maximum local variance, expressed as W fil For the real-time window, the total feature F fusi Performing a bilateral filtering operation (Bilateral(·)) yields the denoising and fusion features F. fusiz F fusi z =Bilateral(F fusi W fil ,σ sp ,σ ra ), σ sp σ ra These are the standard deviations of the Gaussian kernel in the spatial domain and the standard deviations of the Gaussian kernel in the pixel value domain, respectively. S3.5: Detection head: Combines noise-suppressed and fused features F fusi z The input detection head has three parallel convolutional prediction branches: a bounding box regression branch, a category classification branch, and a defect geometry regression branch. Through a preset number of convolutional layers (as a preferred implementation, each branch uses three consecutive stacked convolutional layers; adjustments can be made based on the actual dataset size and hardware deployment conditions), they respectively complete boundary coordinate prediction, defect classification, and size regression. The three branches operate synchronously and output the detection results for the three types of defects. The bounding box regression branch outputs the defect pixel coordinates b=Regressor(F... fusi z ), Regressor(·) is the bounding box convolution branch, b=(x min ,y min ,x max ,y max ) represents the top-left and bottom-right pixel coordinates of the predicted outer boundary. The category classification branch outputs the defect category label c = Classifier(F). fusi z Classifier(·) is the defect classification convolution branch, c is the defect category label, and the value corresponds to the three types of defects: crack, porosity, and material spalling. The defect geometry regression branch simultaneously regresses the defect geometry parameters based on the confidence prediction, s=Sigmoid(conf Θ (F fusi z )), P def y =SizeReg(F fusi z ), conf Θ (·) represents the confidence prediction convolution branch, s∈[0,1] represents the defect prediction confidence, SizeReg(·) represents the new geometric size regression convolution branch, and P def y To predict defect geometric parameters, and compare them with the preset defect morphology parameter P def One-to-one correspondence; finally, the defect detection results are obtained, including defect category, defect pixel boundary, defect geometry and defect prediction confidence. S4: A three-stage progressive differential training strategy is used to train the defect detection network. Each stage employs differentiated sample ratios, loss functions, and parameter training strategies. The final output is the trained wind turbine cavity defect detection network, including: S4.1: Select all simple samples from the hierarchically labeled training dataset, and set the learning rate to an interval of η1 (e.g., 1×10). -3 ~5×10 -3 ), using cross-entropy L ce Normalized bounding box regression mean square error loss L MSE,b The mean square error of the regression of normalized geometric dimensions L MSE,s The defect detection network backbone (including shallow feature extraction branches and deep semantic feature extraction branches) is trained using the joint loss L1 method to learn the basic texture features of defects, where L1 = L ce +γ1×L MSE,b +γ3×L MSE,s L ce Used for defect category classification training, γ1 and γ3 are MSE loss weight coefficients (both ranging from 0.5 to 1.0); after the first stage of iteration convergence, the network has basic defect recognition capabilities, laying the parameter groundwork for introducing difficult and small defect samples in the next stage (during the first stage of training, after each iteration, the loss value L1 and recognition accuracy are calculated on the validation set. When the decrease in the validation set loss value for a set number of consecutive iterations (e.g., 5 iterations) is less than the preset value (e.g., 0.5%), and the fluctuation of the accuracy for a set number of consecutive iterations is less than 1%, or when the maximum number of iterations in this stage is reached, the model is considered to have converged, and the first stage of training ends. The above threshold parameters (5 iterations, 0.5%, 1%) can be adjusted appropriately according to the actual dataset size and training situation). S4.2: After the first stage convergence, proceed to the second stage of optimization training. Freeze all convolution parameters of the shallow feature extraction branches, and supplement the original set of simple samples with all difficult samples to construct a mixed dataset of easy and difficult samples. Set the learning rate to an interval of η² (e.g., 1×10⁻⁶). -4 ~5×10 -4 ), using focus loss L Focal +Scale crossover loss L SIoU +L MSE,s Joint loss L2, training gradient backpropagation to the inverse bottleneck module and heterogeneous convolution module of the deep semantic feature extraction branch, L Focal The loss weights used to suppress simple samples (focusing on minor, difficult defects) correspond to the output of the detection head classification branch, L. SIoUUsed to constrain the bounding box coordinate regression results of defect pixels, addressing the technical shortcomings of weak features and easy missed detection of micropores and fine cracks; after the second stage of iteration, the parameters of the deep multi-scale semantic extraction module were refined and optimized, significantly improving the ability to extract both deep and shallow features, laying the foundation for the third stage of introducing CGAN to generate artificial samples and fine-tuning the fusion / denoising / detection head structure; L2=L Focal +γ2×L SIoU +γ4×L MSE,s γ2 and γ4 are weighting coefficients (γ2 ranges from 1.0 to 1.5, and γ4 ranges from 0.5 to 1.0), L Focal =-a t ×(1-p t ) β log(p t ), a t This is the category balance coefficient (which can be set based on the reciprocal of the number of defect samples of each category in the training set; for the three types of defects—cracks, porosity, and material spalling—the value is typically in the range of 0.25 to 0.75; in this embodiment, we take a). t =0.5), p t β is the model's predicted probability for the target category, and β is the focusing coefficient (with a value of 2, which strengthens the loss weight for low-probability difficult defect samples). IoU is the predicted bounding box B pre With real frame B re The intersection-union ratio is given by w and h, which are the width and height of the bounding box, respectively, and ε is a very small positive number. The convergence determination in the second stage adopts the same strategy as in the first stage. S4.3: After the second stage convergence, a hierarchically labeled training dataset (i.e., a mixed dataset of easy and difficult samples is constructed by supplementing the original set of easy samples with all difficult samples) is used for mixed training. The learning rate is set to an interval of η3 (e.g., 1×10). -5 ~5×10 -5 The convolutional parameters of the shallow feature extraction branches are frozen (the shallow layers have already had their features solidified in the first two stages and will not be updated again). The second stage uses the joint loss L2 inverse to adjust the three types of structural parameters of the hierarchical feature fusion module, the adaptive noise suppression module, and the detection head. These structural parameters include the pixel fusion weight α. i Spatial attention weight matrix A sThe hyperparameters in the adaptive filtering window (spatial domain Gaussian kernel standard deviation and pixel value domain Gaussian kernel standard deviation) and the three-branch convolution weights (including learnable weight parameters corresponding to all convolutional layers inside the bounding box regression branch, category classification branch, and geometric size regression branch); after the third stage of iteration convergence, the trained wind turbine cavity defect detection network is obtained; η1 > η2 > η3; the third stage convergence determination: after each iteration, the detection accuracy mAP is calculated on the validation set. When the improvement of the validation set mAP for a set number of consecutive iterations (e.g., 5 iterations) is less than a preset value (e.g., 0.5%), the model convergence is determined when the maximum number of iterations in this stage is reached, and the model weight with the highest validation set mAP is saved as the final wind turbine cavity defect detection network; In this embodiment, it should be specifically noted that the AdamW optimizer is used for training, and the weight decay coefficient is set to 1×10. -4 The number of iterations in the three stages is different: the number of iterations in the first stage (e.g., 1-70) > the number of iterations in the second stage (e.g., 71-120) > the number of iterations in the third stage (e.g., 121-150); the coordinates of the outer boundary of the defect and the geometric parameters of the defect (crack aspect ratio, equivalent diameter of pores, and material spalling area) are all first subjected to global normalization and mapped to the interval [0,1]; all loss terms are dimensionless scalars to ensure that the dimensions are consistent and the value range is matched when multiple loss terms are added.

[0018] S5: The image of the inner cavity of the wind turbine blade to be detected is preprocessed and then input into the trained defect detection network to obtain the defect detection result; the image of the inner cavity of the wind turbine blade to be detected is then subjected to adaptive median filtering for noise reduction, CLAHE contrast enhancement, and gradient edge enhancement in sequence to obtain the preprocessed standard image of the inner cavity; the standard image of the inner cavity is input into the trained defect detection network to obtain the defect detection result, including: defect category, defect pixel boundary, defect geometric size, and defect prediction confidence. In a preferred embodiment of this invention, an industrial endoscopic robot is used to acquire images of the internal cavity of wind turbine blades. The endoscope is equipped with a 5-megapixel CMOS sensor and an LED ring light. Step 1: During acquisition, the robot moves longitudinally along the internal cavity of the blade, acquiring one image every 0.5 meters. The acquisition locations for each blade include four main areas: the leading edge cavity, the web bonding section, the trailing edge cavity, and the irregularly shaped area at the corner of the internal cavity. A total of 2000 original images of the internal cavities of three wind turbines (each with three blades) are acquired. Among these, 800 images are marked with defects (248 cracks, 310 pores, and 242 material spalling), and 1200 images are defect-free. The image resolution is uniformly adjusted to 640×48. 0 pixels, generate 3000 controllable defect images; automatically complete all image annotation, divide the sample set according to the 5‰ pixel ratio threshold, finally obtain 2800 simple samples and 1600 difficult samples, construct a hierarchical annotation training dataset with a total capacity of 4400 images; Step 2, build a dual-branch feature enhancement detection network, the shallow feature extraction branch stacks 4 layers of 3×3 convolutions, set 2 layers of cross-branch short connections; the deep reverse bottleneck module has a channel compression ratio of 1:4 and embeds a channel attention mechanism; the heterogeneous convolution module is configured in parallel with dilated convolutions with dilation coefficients of 2, 3, and 4 and 1×1, 2×2, and 3×3 convolutions; the hierarchical feature fusion module completes the fusion of deep and shallow features and embeds the prior parameters of defect morphology; the adaptive noise suppression module has a minimum window W min =5, window adjustment coefficient η=0.1; the detection head outputs three-dimensional detection results of defect category, coordinates, and size; Step 3, three-stage differential model training, first stage: using only 2800 simple samples, learning rate 2×10 -3 The loss weight γ1=0.8, and the training is iterated for 70 rounds to complete the learning of the basic features of the network backbone; the second stage: 1600 difficult samples are added, with a learning rate of 2×10. -4 The loss weights γ1=0.8 and γ2=1.2, the focusing coefficient β=2, and the training was iterated for 120 rounds to optimize the parameters of the multi-scale feature extraction module; the third stage: mixed training with all 4400 samples, with a learning rate of 2×10. -5 The parameters of the four shallow convolutional layers were frozen, and only the fusion module, noise suppression module, and detection head were fine-tuned. The model was iterated for 60 rounds to achieve convergence, resulting in the final detection model. Step 4: Actual defect detection. 100 images of the inner cavity of wind turbine blades that were not used in the training were selected, preprocessed uniformly, and input into the trained model. The evaluation metric was the mean accuracy (mAP@0.5:0.95). The detection results are as follows:

[0019] Step 5: Comparative Experiment:

[0020] Note: Experiment A serves as the baseline, using 800 original real defect images + 1200 defect-free images, employing a conventional single-stage training strategy; Experiment B adds synthetic defect images to the foundation of A; Experiment C adds a progressive training strategy to the foundation of B; Experiment D adds a noise suppression module to the foundation of C. Modules are added one by one, and performance is continuously improved, verifying the effectiveness of each module.

[0021] Secondly: The accompanying drawings of the embodiments disclosed in this invention only involve the structures involved in the embodiments disclosed in this invention. Other structures can refer to the general design. In the absence of conflict, the same embodiment and different embodiments of this invention can be combined with each other. In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting internal defects in wind turbine blades based on image recognition and deep learning, characterized in that: include: S1: Collect the original image set of the inner cavity of the wind turbine blade and perform preprocessing to obtain the original standard image set of the inner cavity composed of preprocessed defect images and preprocessed defect-free images; S2: Based on conditional generative adversarial networks, a set of internal cavity defect samples is generated from the original standard internal cavity image set, and all image data are labeled, managed and divided to construct a hierarchical labeled training dataset; The implementation of S2 includes: S2.1: from the preprocessed defect-free image set S good Selecting single intracavitary images I in sequence bg,i As a background base for generation, for a fixed background image I bg,i Three types of defects were selected: cracks, pores, and material spalling. Multiple sets of two-dimensional spatial coordinates Pos were then configured for each defect, each with a different center pixel position. k and the differential defect morphology parameter P def,k Each parameter combination corresponds one-to-one with a single sample generation task; the conditional generative adversarial network consists of a generator G and a discriminator D, which combine the ternary combinations {I bg,i Pos k ,P def,k Input generator G, generator mapping relationship satisfies: I bg,k h =G(I bg,i Pos k ,P def,k Output a synthetic defect image I bg,k h For the preprocessed defect image subset S def Real defect image I re,i Its corresponding condition vector is c i For the synthetic defect image I bg,k h Its corresponding condition vector is c k， The input to discriminator D is an "image-condition" pair, i.e., {I re,i ,c i } and {I bg,k h ,c k The output is the true / false probability D(I,c) of the "image-condition" pair; a defect morphology regularization constraint term is added to the basic adversarial loss of the discriminator to obtain the total discriminator loss L. D ; S2.2: According to the total loss L of the discriminator D Total generator loss L G The generator and discriminator are trained alternately and iteratively, and the total loss L of the discriminator is recorded in each iteration. D t And generator loss L G t Where t is the iteration round, convergence is determined by the relative rate of change of loss for t0 consecutive rounds; convergence occurs when the relative rates of change of the discriminator loss and the generator loss are simultaneously less than the convergence threshold η(L) for t0 consecutive rounds. th Once the network converges, the iteration stops. After convergence, without changing the background image, the defect generation coordinates and morphological parameters such as crack aspect ratio, crack overlap coefficient, pore equivalent diameter, and material spalling area threshold are continuously adjusted to obtain batch simulation generation I of defects of different locations and specifications on the same internal cavity background image. bg,k h =G(I bg,i Pos k ,P def,k ), k=1,2,...,K, where K is the total number of parameter configuration groups corresponding to a single background image, I bg,k h The synthesized defect image is generated for the k-th set of parameters; after traversing all parameter sets corresponding to a single background, a new defect-free background image is generated cyclically, and finally, all background images are collected into a set S. good The defect image sample set S is obtained. gen ; S2.3: For real defect images I re,i ∈S def Labels are generated manually, including defect category, defect pixel boundary, and defect geometry, forming a set of labeled real defect samples S. def y ; Targeting the generation of defective image sample set S gen The images in the dataset are generated based on the input spatial coordinates Pos corresponding to each group of generated images. k With the preset defect morphology parameter P def,k The defect bounding box pixel coordinates are converted, and preset parameters are directly used as geometric dimension annotations. The defect type is determined by the currently used defect morphology parameters. Defect-free cavity images are uniformly annotated as defect-free samples, and all annotated real defect samples S are... def b Generate defect sample S gen b Defect-free sample S good b The dataset is formed by merging the original, fully labeled dataset S. all ; S2.4: Let the total number of pixels in a single image be N. all The total number of defective pixels N in a single image def According to the percentage of defective pixels R pix =N def / N all The full original labeled dataset S all Divide into simple sample subsets S easy and difficult sample subset S hard Finally, a hierarchical labeled training dataset S1 was constructed. all =S easy ∪S hard ; S3: Construct a defect detection network, which consists of a shallow feature extraction branch, a deep semantic feature extraction branch, a hierarchical feature fusion module, an adaptive noise suppression module, and a detection head cascaded in sequence to obtain the defect detection results; The hierarchical feature fusion module is divided into a surface fusion unit and a high-level fusion unit. Firstly, the surface fusion unit is based on the shallow detail feature map F. sh Feature aggregation is performed using pixel-weighted summation to obtain the detailed feature map F after surface fusion. surf Then, the high-level fusion unit, based on the deep multi-scale semantic feature map F... deep The spatial attention weight matrix A is generated through the spatial attention module. s Then A s With F deep Multiplying by spatial location yields the enhanced high-level semantic feature map F. high Finally, the detailed feature map F surf and high-level semantic feature map F high The data is uniformly scaled to the same height and width, and the total feature F is obtained by pixel-by-pixel addition and fusion. fusi ; The adaptive noise suppression module: inserts a noise suppression module into the feature fusion path, and in the fused feature F fusi Slide an upward local calculation window of size M×N, and calculate the characteristic local variance σ within the window. lo 2 Based on local variance σ lo 2 Adaptive solution for bilateral filter window size W fil , with W fil For the real-time window, the total feature F fusi Perform bilateral filtering to obtain the noise reduction and fusion features F. fusi z ; S4: A three-stage progressive differential training strategy is adopted to train the defect detection network. Differentiated sample ratios, loss functions and parameter training strategies are used in each stage. Finally, the trained wind turbine cavity defect detection network is output. The implementation of S4 includes: S4.1: Selecting all simple samples in the hierarchically labeled training dataset, setting the learning rate interval to η1, and using cross-entropy L ce Normalized bounding box regression mean square error loss L MSE,b The mean square error of the regression of normalized geometric dimensions L MSE,s The backbone of the defect detection network is trained using a joint loss L1 method. S4.2: After the first stage convergence, the second stage of optimization training begins. All convolutional parameters of the shallow feature extraction branches are frozen. A mixed-difficulty dataset is constructed by supplementing the original set of simple samples with all difficult samples. The learning rate is set to an interval of η², and the focus loss L0 is used. Focal +Scale crossover loss L SIoU +L MSE,s Joint loss L2, training gradient backpropagation to the inverse bottleneck module and heterogeneous convolution module of the deep semantic feature extraction branch, L Focal The loss weights used to suppress simple samples correspond to the output of the classification branch of the detection head, L. SIoU Used to constrain the bounding box coordinates of defective pixels; S4.3: After the second stage convergence, a hierarchical labeled training dataset is used for hybrid training, with the learning rate set to η3. All convolutional parameters of the shallow feature extraction branch are frozen, and the hierarchical feature fusion module, adaptive noise suppression module, and the three types of structural parameters of the detection head are adjusted in reverse. The structural parameters include pixel fusion weight a. i Spatial attention weight matrix A s The hyperparameters in the adaptive filtering window and the convolution weights of the three branches of the detection head; after the third stage of iteration convergence, the trained wind turbine cavity defect detection network is obtained; η1>η2>η3; S5: The image of the inner cavity of the wind turbine blade to be detected is preprocessed and then input into the trained defect detection network to obtain the defect detection result.

2. The method for detecting internal defects in wind turbine blades based on image recognition and deep learning according to claim 1, characterized in that: The implementation of S3 includes: S3.1: Shallow feature extraction branch: extracting the preprocessed intracavitary standard image I pre Each feature is fed into a shallow feature extraction branch, and then a series of 3×3 convolutional layers are stacked sequentially. 3×3 Extracting shallow detail feature maps F sh And outputs the preset level shallow features F through the cross-branch short connection bypass. de Deep semantic feature extraction branch; S3.2: Deep semantic feature extraction branch: It consists of a series of inverse bottleneck modules and heterogeneous convolutional modules. First, the inverse bottleneck module extracts shallow features F through the shallow branch's cross-branch bypass. de As input, channel compression processing is performed to obtain the compressed feature F. com Based on compression feature F com Calculate the channel attention weight matrix, and use the attention weight matrix to apply F. com Weighted optimization yields feature F att and F att Input a heterogeneous convolutional module; then, the heterogeneous convolutional module is configured in parallel with at least one large-sized dilated convolutional kernel and at least one small-sized convolutional kernel to obtain a deep multi-scale semantic feature map F. deep .

3. The method for detecting internal defects in wind turbine blades based on image recognition and deep learning according to claim 1, characterized in that: The S3 implementation also includes a detection head: fusing the noise-suppressed, denoised feature F... fusi z The input detection head has three parallel convolutional prediction branches: a bounding box regression branch, a category classification branch, and a defect geometry regression branch. Through convolution operations with a preset number of layers, the boundary coordinate prediction, defect classification, and size regression are completed respectively. The three branches operate synchronously and output the three types of defect detection results together. Finally, the defect detection results are obtained, including the defect category, the outer boundary of the defect pixels, the defect geometry, and the defect prediction confidence.

Images