SASED-YOLO algorithm-based wind power blade surface multi-scale defect detection method

Abstract

The invention provides a wind power blade surface multi-scale defect detection method based on an SASED-YOLO algorithm. The wind power blade surface multi-scale defect detection method comprises the steps of performing preprocessing and image enhancement processing on a wind power blade defect image; cascading the feature extraction network with M first modules, performing down-sampling on the processed wind power blade defect image, and performing feature extraction based on a down-sampling result to obtain a feature map of the wind power blade defect image; calibrating the channel weight of the feature map based on a second module; contextual information from different receptive fields is fused based on a third module; the attention weights of the multi-scale targets in different feature maps are distributed based on the dependency relationship between attention mechanism attention space and channels and the dependency relationship; and training the network model according to the training set and the verification set based on a weight distribution result to obtain a defect detection model, and performing defect detection on the wind power blade surface detection image based on the defect detection model.

Description

Technical Field

[0001] The present invention relates to the field of target detection technology, and in particular to a method for detecting multi-scale defects on the surface of wind turbine blades based on a SASED-YOLO algorithm. Background Art

[0002] On the future development path, it has become a global consensus to improve the green and low-carbon development mechanism and accelerate the planning and construction of a new energy system. Therefore, the development prospects of the wind power industry are promising. With the continuous innovation of technology and the improvement of the industrial chain, floating offshore wind power is steadily moving from the experimental stage to commercial application. As one of the key components of wind turbines, the safe operation of wind turbine blades is an important factor in ensuring the stable operation of wind turbines. Most wind farms are built in areas with harsh environments and strong winds. Due to long-term exposure to harsh natural environments, wind turbine blades will suffer from lightning burns, cracking, acid rain erosion, breakage, oil corrosion and other damages, affecting power generation efficiency and service life. Therefore, the safety monitoring of wind turbine blades is very important.

[0003] However, common detection methods for wind turbine blade defects include ultrasonic testing, infrared thermal imaging testing, and acoustic emission testing. These methods significantly reduce detection accuracy when the surface of the wind turbine blade is not smooth or coated.

[0004] Therefore, in order to overcome the above defects, the present invention provides a multi-scale defect detection method for wind turbine blade surface based on SASED-YOLO algorithm. Summary of the Invention

[0005] The present invention provides a multi-scale defect detection method for wind turbine blade surfaces based on the SASED-YOLO algorithm, which is used to obtain wind turbine blade defect images, process and extract features from the wind turbine blade defect images, and further analyze the extracted features, thereby accurately and effectively constructing a SASED-YOLO model. Secondly, the wind turbine blade surface detection image is input into the obtained SASED-YOLO model for defect detection, and finally target detection is achieved on the input image, the category of each defect in the wind turbine blade image is accurately determined, and the specific position of the defect is located, thereby improving the accuracy and reliability of multi-scale defect detection on the wind turbine blade surface.

[0006] The present invention provides a method for detecting multi-scale defects on the surface of wind turbine blades based on the SASED-YOLO algorithm, comprising:

[0007] Step 1: Obtain a wind turbine blade defect image, and perform preprocessing and image enhancement on the wind turbine blade defect image;

[0008] Step 2: cascade the M first modules of the feature extraction network, and sequentially downsample the processed wind turbine blade defect image according to the spatial resolution reduction value of the feature map, and perform feature extraction based on the downsampling result to obtain a feature map of the wind turbine blade defect image;

[0009] Step 3: Calibrate the feature map channel weights based on the second module;

[0010] Step 4: Fusion of contextual information from different receptive fields based on the third module;

[0011] Step 5: Focus on the dependencies between spaces and channels based on the attention mechanism, and assign attention weights to multi-scale targets in different feature maps based on the dependencies;

[0012] Step 6: Based on the weight distribution results, the network model is trained according to the training set and the validation set to obtain the SASED-YOLO model, and defect detection is performed on the wind turbine blade surface inspection image based on the SASED-YOLO model.

[0013] Preferably, a multi-scale defect detection method for a wind turbine blade surface based on the SASED-YOLO algorithm, in step 1, obtaining a wind turbine blade defect image, and performing preprocessing and image enhancement processing on the wind turbine blade defect image, including:

[0014] Accessing a preset image database and extracting a set of wind turbine blade defect images based on the access results;

[0015] Performing object recognition on each wind turbine blade defect image in the extracted wind turbine blade defect image set, and screening the wind turbine blade defect images based on the object recognition results to obtain a valid wind turbine blade defect image set;

[0016] Randomly select four wind turbine blade defect images from a valid wind turbine blade defect image set, and flip the randomly selected wind turbine blade defect images in relative directions respectively;

[0017] At the same time, the defective image of the wind turbine blade after flipping is resized based on a preset size index, and based on the size scaling result, a neural network is called to perform color gamut transformation on the defective image of the wind turbine blade according to a multi-dimensional color gamut quantization index;

[0018] The enhanced wind turbine blade defect image is obtained based on the color gamut transformation, and the enhanced wind turbine blade defect image is split into a training set, a verification set and a test set based on a preset ratio.

[0019] Preferably, a multi-scale defect detection method for a wind turbine blade surface based on the SASED-YOLO algorithm, in step 2, cascades M first modules of the feature extraction network, and sequentially downsamples the processed wind turbine blade defect image according to the spatial resolution reduction value of the feature map, and performs feature extraction based on the downsampling result to obtain a feature map of the wind turbine blade defect image, including:

[0020] Acquire structural parameters of the first module based on the management terminal, and determine a connection interface of the first module during cascading based on the structural parameters;

[0021] Extracting service execution parameters of the feature extraction network and the first module, determining a logical order of feature extraction based on the service execution parameters, and determining a cascading order of the feature extraction network and the first module based on the logical order;

[0022] Cascading the feature extraction network and the M first modules based on the cascade order and the connection interface, configuring the input and output format compatibility of the feature extraction network and the first modules based on the cascade result, and adapting the hyperparameters of the feature extraction network and the first modules based on the input and output format compatibility configuration result and task requirements;

[0023] At the same time, a spatial resolution reduction value of the feature map is obtained, and a downsampling ratio is determined based on the spatial resolution reduction value of the feature map, and the processed wind turbine blade defect image is downsampled based on the downsampling ratio;

[0024] Based on the hyperparameter adaptation results, the feature extraction network and the first module are divided into a dual-branch feature extraction structure, and feature extraction is performed on the downsampling results based on the differential feature extraction strategy of the dual-branch feature extraction structure;

[0025] The extracted features are correlated and summarized to obtain the feature map of the wind turbine blade defect image.

[0026] Preferably, a multi-scale defect detection method for wind turbine blade surfaces based on the SASED-YOLO algorithm performs feature extraction on the downsampling results based on a difference feature extraction strategy of a dual-branch feature extraction structure, including:

[0027] Obtain the obtained dual-branch feature extraction structure, and map and match the dual-branch feature extraction structure with the difference feature extraction strategy to obtain the specific feature extraction strategy corresponding to each branch feature extraction structure, where one branch feature extraction structure is slice feature extraction and the other branch feature extraction structure is convolutional network extraction;

[0028] When extracting features for slices:

[0029] Determine the slice size based on task requirements and wind turbine blade defect images, and determine the step size based on feature extraction requirements;

[0030] Performing a set sequence of slicing operations on the wind turbine blade defect image based on the slice size and step size, and obtaining a plurality of slice regions corresponding to the wind turbine blade defect image based on the slicing operations;

[0031] Sequentially numbering the plurality of slice regions based on a set sequence, and sequentially adjusting the contrast of each slice region based on the sequence numbering result;

[0032] Based on the contrast adjustment result, the corresponding slice area is input into the preset convolutional neural network for feature extraction to obtain the feature representation corresponding to each slice area;

[0033] Performing associative splicing on feature representations of adjacent slice regions based on sequential numbers, and adaptively adjusting the splicing gap of the associated splicing regions based on the step size to obtain a first feature map;

[0034] When extracted for a convolutional network:

[0035] The size of the wind turbine blade defect image is cropped based on 3×3 maximum pooling and 1×1 convolution, and feature extraction is performed based on the cropping result to obtain the second feature map;

[0036] The first feature map and the second feature map are associated and summarized to obtain a feature map of the wind turbine blade defect image.

[0037] Preferably, a multi-scale defect detection method for a wind turbine blade surface based on the SASED-YOLO algorithm, in step 3, calibrating the feature map channel weights based on the second module includes:

[0038] Extract the channel adaptation mechanism of the second module and use 1×1 global average pooling to adjust the feature map size based on the channel adaptation mechanism;

[0039] The pooled results are processed based on the fully connected layer, and the weight distribution of the feature map channels is obtained based on the processing results.

[0040] Preferably, a multi-scale defect detection method for a wind turbine blade surface based on the SASED-YOLO algorithm, in step 4, context information from different receptive fields is fused based on the third module, including:

[0041] Extracting configuration parameters of the third module, where the configuration parameters include a global receptive field branch, a cascaded maximum pooling branch, and a spatial and channel reconstruction convolution branch;

[0042] Based on the global receptive field branch, 1×1 convolution and average pooling are used to compress the number of channels and spatial dimensions of the feature map respectively, and the spatial size of the feature map is adjusted according to the bilinear interpolation method based on the compression result to obtain the first context information;

[0043] Based on the cascaded maximum pooling branch, three 5×5 convolutions are used to extract features of different receptive fields from top to bottom to obtain the second context information;

[0044] Based on the spatial and channel reconstruction convolution branch, the spatial reconstruction unit and the channel reconstruction unit are used to remove redundant information in the feature map to obtain the third context information;

[0045] The first context information, the second context information, and the third context information are fused.

[0046] Preferably, a multi-scale defect detection method for a wind turbine blade surface based on the SASED-YOLO algorithm, in step 5, comprises:

[0047] Based on the attention mechanism, the multi-channel selective attention module and the position-aware self-attention module are combined, and the target features in space and channels are extracted. The weights of the feature map are adjusted according to the target features to obtain the feature information of the wind turbine blade defect image.

[0048] Preferably, a multi-scale defect detection method for wind turbine blade surfaces based on the SASED-YOLO algorithm extracts target features in space and channels according to a multi-channel selective attention module, including:

[0049] The multi-channel selective attention module includes a first processing unit and a second processing unit;

[0050] Wherein, the first processing unit is used to:

[0051] Divide the feature map X to obtain N sub-feature maps;

[0052] Perform mean pooling operation on the width and height of N sub-feature maps;

[0053] Perform a depthwise convolution operation on the N sub-feature maps after the mean pooling operation according to a preset convolution kernel, and extract spatial information of the feature maps at different scales according to the results of the depthwise convolution operation to obtain a first target feature map and a second target feature map;

[0054] The first target feature map and the second target feature map are spliced based on the channel dimension to obtain a third target feature map. At the same time, the third target feature map is group normalized and the spatial region is activated according to a preset function.

[0055] The second processing unit is configured to:

[0056] Perform maximum pooling and averaging operations on the input feature map X to obtain matrix a and matrix b;

[0057] Extract the local features of matrix a and the global features of matrix b based on the fully connected layer, and add and fuse the local features of matrix a and the global features of matrix b to obtain the target fusion result;

[0058] Map the target fusion result to a first preset range based on a preset function, determine the attention weight according to the mapping result, and input the attention weight into the feature map X to obtain a feature tensor;

[0059] Based on the dimension direction, the feature tensor and the spatial region are spliced to obtain a new feature map P;

[0060] In the position-aware self-attention module, the new feature map P is downsampled according to the average pooling with a preset step size to obtain the feature y;

[0061] Perform group normalization on feature y to obtain feature map n i ;

[0062] For feature map n i Perform deep point-by-point convolution processing to extract feature map n i The feature information of the feature vector is obtained by calculating the weights according to the multi-head attention mechanism and performing weighted aggregation, and the feature map Attn is obtained according to the feature vector.

[0063] The feature map Attn is reshaped and the height dimension and the number of attention heads are merged to obtain the channel dimension. The mean operation is performed on the height and width of the processed feature map Attn respectively and converted into feature weights within a second preset range based on a preset function. The feature weights are applied to the feature map P obtained after multi-channel selective attention feature extraction.

[0064] Preferably, a multi-scale defect detection method for a wind turbine blade surface based on the SASED-YOLO algorithm, in step 6, a network model is trained according to a training set and a validation set based on the weight distribution result to obtain a SASED-YOLO model, and defect detection is performed on a wind turbine blade surface inspection image based on the SASED-YOLO model, including:

[0065] S61: Input the paths of the training set and validation set and the initial training weight file path into the parameters required for network model training to obtain the SASED-YOLO model;

[0066] S62: Perform defect detection on the wind turbine blade surface inspection image based on the SASD-YOLO model and output the target recognition result.

[0067] Preferably, a multi-scale defect detection method for wind turbine blade surface based on SASED-YOLO algorithm, the target recognition results include: surface sand holes, damage, cracks, surface oil stains, fractures and damage, lightning strikes, surface attachments, and surface rust.

[0068] Compared with the prior art, the present invention has the following beneficial effects:

[0069] By acquiring wind turbine blade defect images, processing and feature extraction of the wind turbine blade defect images, and further analyzing the extracted features, the SASED-YOLO model can be accurately and effectively constructed. Secondly, the wind turbine blade surface detection image is input into the obtained SASED-YOLO model for defect detection, and finally target detection is achieved on the input image, and the category of each defect in the wind turbine blade image is accurately determined and the specific location of the defect is located, thereby improving the accuracy and reliability of multi-scale defect detection on the wind turbine blade surface.

[0070] Other features and advantages of the present invention will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present invention. The purpose and other advantages of the present invention can be achieved and obtained through the structures specifically pointed out in this application document.

[0071] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0073] Figure 1 This is a flow chart of a method for detecting multi-scale defects on a wind turbine blade surface based on the SASED-YOLO algorithm in an embodiment of the present invention;

[0074] Figure 2 Schematic diagram of a multi-scale defect detection method for wind turbine blade surfaces based on the SASED-YOLO algorithm in an embodiment of the present invention;

[0075] Figure 3 This is a schematic diagram of step 3 in a method for multi-scale defect detection on a wind turbine blade surface based on the SASED-YOLO algorithm in an embodiment of the present invention;

[0076] Figure 4 This is a schematic diagram of step 4 in a multi-scale defect detection method for wind turbine blade surfaces based on the SASED-YOLO algorithm in an embodiment of the present invention. DETAILED DESCRIPTION

[0077] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0078] Example 1:

[0079] This embodiment provides a method for detecting multi-scale defects on the surface of wind turbine blades based on the SASED-YOLO algorithm. Figure 1 Shown, including:

[0080] Step 1: Obtain a wind turbine blade defect image, and perform preprocessing and image enhancement on the wind turbine blade defect image;

[0081] Step 2: cascade the M first modules of the feature extraction network, and sequentially downsample the processed wind turbine blade defect image according to the spatial resolution reduction value of the feature map, and perform feature extraction based on the downsampling result to obtain a feature map of the wind turbine blade defect image;

[0082] Step 3: Calibrate the feature map channel weights based on the second module;

[0083] Step 4: Fusion of contextual information from different receptive fields based on the third module;

[0084] Step 5: Focus on the dependencies between spaces and channels based on the attention mechanism, and assign attention weights to multi-scale targets in different feature maps based on the dependencies;

[0085] Step 6: Based on the weight distribution results, the network model is trained according to the training set and the validation set to obtain the SASED-YOLO model, and defect detection is performed on the wind turbine blade surface inspection image based on the SASED-YOLO model.

[0086] In this embodiment, the principle diagram of the wind turbine blade surface multi-scale defect detection method based on the SASED-YOLO algorithm is as follows: Figure 2 shown.

[0087] In this embodiment, the first module is a FADonw module.

[0088] In this embodiment, the second module is the C2f_SENetV2 module.

[0089] In this embodiment, the third module is the SPPSCCAP module.

[0090] In this embodiment, the image enhancement processing refers to adjusting the color range and contrast, etc. in the wind turbine blade defect image.

[0091] In this embodiment, the context information refers to the relationship between the wind turbine blade defect image and the surrounding environment.

[0092] In this embodiment, the preprocessing refers to performing denoising processing on the defective image of the wind turbine blade.

[0093] In this embodiment, the feature extraction network is constructed in advance and is used to extract features from defect images of wind turbine blades.

[0094] In this embodiment, the spatial resolution reduction value is set in advance, the purpose of which is to reduce the spatial resolution of the feature map and enhance the deep network's ability to capture global features based on the subtle global spatial information of the defect features.

[0095] In this embodiment, downsampling is a process of reducing the image resolution by reducing the number of pixels in the image.

[0096] In this embodiment, feature map channel weight calibration refers to the operation of assigning a weight value to each channel of the feature map. This weight value is used to measure the importance of the feature information contained in each channel.

[0097] The working principle and beneficial effects of the above technical solution are: by acquiring the defect image of the wind turbine blade, processing and feature extraction of the wind turbine blade defect image, and further analyzing the extracted features, the SASED-YOLO model is accurately and effectively constructed. Secondly, the surface detection image of the wind turbine blade is input into the obtained SASED-YOLO model for defect detection, and finally target detection is achieved on the input image, the category of each defect in the wind turbine blade image is accurately determined, and the specific location of the defect is located, thereby improving the accuracy and reliability of multi-scale defect detection on the surface of the wind turbine blade.

[0098] Example 2:

[0099] Based on Example 1, this embodiment provides a multi-scale defect detection method for wind turbine blade surfaces based on the SASED-YOLO algorithm. In step 1, a wind turbine blade defect image is acquired, and the wind turbine blade defect image is preprocessed and image enhanced, including:

[0100] Accessing a preset image database and extracting a set of wind turbine blade defect images based on the access results;

[0101] Performing object recognition on each wind turbine blade defect image in the extracted wind turbine blade defect image set, and screening the wind turbine blade defect images based on the object recognition results to obtain a valid wind turbine blade defect image set;

[0102] Randomly select four wind turbine blade defect images from a valid wind turbine blade defect image set, and flip the randomly selected wind turbine blade defect images in relative directions respectively;

[0103] At the same time, the defective image of the wind turbine blade after flipping is resized based on a preset size index, and based on the size scaling result, a neural network is called to perform color gamut transformation on the defective image of the wind turbine blade according to a multi-dimensional color gamut quantization index;

[0104] The enhanced wind turbine blade defect image is obtained based on the color gamut transformation, and the enhanced wind turbine blade defect image is split into a training set, a verification set and a test set based on a preset ratio.

[0105] In this embodiment, the preset image database is set in advance and is used to store defect images of wind turbine blades.

[0106] In this embodiment, object recognition refers to identifying the wind turbine blades in the wind turbine blade defect image.

[0107] In this embodiment, the effective wind turbine blade defect image set refers to images that can effectively record wind turbine blade defects and are obtained by screening each image in the extracted wind turbine blade defect image set.

[0108] In this embodiment, the preset size index is set in advance.

[0109] In this embodiment, the multi-dimensional color gamut quantization index is set in advance and is used to represent the value for color gamut adjustment of the wind turbine blade defect image.

[0110] In this embodiment, the preset ratio is 7:2:1.

[0111] The working principle and beneficial effects of the above technical solution are: by accessing the preset image database, the set of wind turbine blade defect images can be accurately and effectively determined. At the same time, the obtained wind turbine blade defect images are screened and processed to achieve reliable preprocessing and image enhancement of the wind turbine blade images, ensuring the reliability of the final wind turbine blade defect images and providing convenience and guarantee for the construction of the SASED-YOLO model.

[0112] Example 3:

[0113] Based on Example 1, this embodiment provides a multi-scale defect detection method for a wind turbine blade surface based on the SASED-YOLO algorithm. In step 2, the feature extraction network is cascaded to form M first modules, and the processed wind turbine blade defect image is sequentially downsampled according to the spatial resolution reduction value of the feature map, and feature extraction is performed based on the downsampling result to obtain a feature map of the wind turbine blade defect image, including:

[0114] Acquire structural parameters of the first module based on the management terminal, and determine a connection interface of the first module during cascading based on the structural parameters;

[0115] Extracting service execution parameters of the feature extraction network and the first module, determining a logical order of feature extraction based on the service execution parameters, and determining a cascading order of the feature extraction network and the first module based on the logical order;

[0116] Cascading the feature extraction network and the M first modules based on the cascade order and the connection interface, configuring the input and output format compatibility of the feature extraction network and the first modules based on the cascade result, and adapting the hyperparameters of the feature extraction network and the first modules based on the input and output format compatibility configuration result and task requirements;

[0117] At the same time, a spatial resolution reduction value of the feature map is obtained, and a downsampling ratio is determined based on the spatial resolution reduction value of the feature map, and the processed wind turbine blade defect image is downsampled based on the downsampling ratio;

[0118] Based on the hyperparameter adaptation results, the feature extraction network and the first module are divided into a dual-branch feature extraction structure, and feature extraction is performed on the downsampling results based on the differential feature extraction strategy of the dual-branch feature extraction structure;

[0119] The extracted features are correlated and summarized to obtain the feature map of the wind turbine blade defect image.

[0120] In this embodiment, the FADonw module includes a slice convolution part and a pooling part;

[0121] The slice convolution part divides the image into four different sub-regions, extracts the features in each region, and finally compresses the spatial information to the channel dimension to enhance the detailed features of the fuzzy edges in the feature map.

[0122] The pooling part uses 3×3 maximum pooling to scan the local area line by line, remove unnecessary redundant features in the defect image, promote the network's ability to focus on global features, and reduce the spatial dimension of the feature map. Then, a 1×1 convolution is used to intercept the subtle features X'2, greatly reducing the computational complexity of the network: X'2 = Conv(MaxPooling(X2)), where Conv() is standard convolution and MaxPooling() is maximum pooling.

[0123] In this embodiment, the structural parameters refer to the configuration parameters and functional parameters of the first module.

[0124] In this embodiment, the service execution parameters are the service execution flow and service processing method of the feature extraction network and the first module when they are working.

[0125] In this embodiment, the input-output format compatibility configuration refers to configuring data interaction between the feature extraction network and the first module to ensure that data can effectively interact between the feature extraction network and the first module.

[0126] In this embodiment, hyperparameter adaptation refers to the process of finding a set of optimal or near-optimal hyperparameter values for a specific task, data set, and model structure, that is, configuring the working performance of the feature extraction network and the first module.

[0127] In this embodiment, the dual-branch feature extraction structure refers to splitting the feature extraction network and the first module into two parallel feature extraction structures, and adopting different feature extraction methods to perform feature extraction.

[0128] The working principle and beneficial effects of the above technical solution are: by analyzing the structural parameters of the first module, the connection interface of the first module is determined; secondly, by determining the business execution parameters of the feature extraction network and the first module, the cascade order of the feature extraction network and the first module is determined according to the business execution parameters, thereby cascading the feature extraction network and the first module according to the cascade order and the connection interface; finally, the cascade result is hyperparameter adapted, and the wind turbine blade defect image is downsampled according to the hyperparameter adaptation result, finally achieving accurate and effective determination of the feature map of the wind turbine blade defect image, ensuring the accuracy and reliability of the final feature map.

[0129] Example 4:

[0130] Based on Example 3, this embodiment provides a multi-scale defect detection method for wind turbine blade surfaces based on the SASED-YOLO algorithm. The method extracts features from the downsampling results based on a difference feature extraction strategy using a dual-branch feature extraction structure, including:

[0131] Obtain the obtained dual-branch feature extraction structure, and map and match the dual-branch feature extraction structure with the difference feature extraction strategy to obtain the specific feature extraction strategy corresponding to each branch feature extraction structure, where one branch feature extraction structure is slice feature extraction and the other branch feature extraction structure is convolutional network extraction;

[0132] When extracting features for slices:

[0133] Determine the slice size based on task requirements and wind turbine blade defect images, and determine the step size based on feature extraction requirements;

[0134] Performing a set sequence of slicing operations on the wind turbine blade defect image based on the slice size and step size, and obtaining a plurality of slice regions corresponding to the wind turbine blade defect image based on the slicing operations;

[0135] Sequentially numbering the plurality of slice regions based on a set sequence, and sequentially adjusting the contrast of each slice region based on the sequence numbering result;

[0136] Based on the contrast adjustment result, the corresponding slice area is input into the preset convolutional neural network for feature extraction to obtain the feature representation corresponding to each slice area;

[0137] Performing associative splicing on feature representations of adjacent slice regions based on sequential numbers, and adaptively adjusting the splicing gap of the associated splicing regions based on the step size to obtain a first feature map;

[0138] When extracted for a convolutional network:

[0139] The size of the wind turbine blade defect image is cropped based on 3×3 maximum pooling and 1×1 convolution, and feature extraction is performed based on the cropping result to obtain the second feature map;

[0140] The first feature map and the second feature map are associated and summarized to obtain a feature map of the wind turbine blade defect image.

[0141] In this embodiment, the difference feature extraction strategy refers to the feature extraction strategy corresponding to the dual-branch feature extraction structure.

[0142] In this embodiment, the task requirements are known in advance.

[0143] In this embodiment, the preset convolutional neural network is trained in advance and is used to extract image features from the image, that is, the final feature representation.

[0144] In this embodiment, the first feature map refers to a feature map of a wind turbine blade defect image extracted through slicing features.

[0145] In this embodiment, the second feature map refers to a feature map of a wind turbine blade defect image extracted by a convolutional network.

[0146] The working principle and beneficial effects of the above technical solution are: by adopting a dual-branch feature extraction structure to extract features of the wind turbine blade defect image respectively, and obtain corresponding feature maps respectively, and finally associate and summarize the obtained first feature map and second feature map, the feature map of the wind turbine blade defect image is effectively obtained, ensuring the accuracy and reliability of the final feature map.

[0147] Example 5:

[0148] Based on Example 1, this embodiment provides a multi-scale defect detection method for wind turbine blade surfaces based on the SASED-YOLO algorithm. Figure 3 As shown, in step 3, the feature map channel weights are calibrated based on the second module, including:

[0149] Extract the channel adaptation mechanism of the second module and use 1×1 global average pooling to adjust the feature map size based on the channel adaptation mechanism;

[0150] The pooled results are processed based on the fully connected layer, and the weight distribution of the feature map channels is obtained based on the processing results.

[0151] In this embodiment, the C2f_SENetV2 module adopts a channel adaptive mechanism, which uses a compression-first-then-excitation method. The compression method uses 1×1 global average pooling to adjust the feature map size, and the excitation method uses a fully connected layer to process the pooled result to optimize the weight distribution between each channel.

[0152] In this embodiment, the C2f_SENetV2 module compresses the spatial dimension of the input feature map by using global average pooling in the compression process, converting the spatial information of each channel into global statistics, and transforming the feature image into R B×C×1×1 Shape: During the excitation process, four parallel fully connected branches are used to process the results of global average pooling. Each branch processes global information to varying degrees through parallel computation, aiming to enhance the network's ability to express features from multiple perspectives.

[0153] In this embodiment, the channel adaptation mechanism is a configuration mechanism in the second module, which is used to adjust the size of the feature map according to the specific situation of the feature map.

[0154] The working principle and beneficial effects of the above technical solution are: by processing the feature map through the channel adaptation mechanism in the second module, the weights of the feature map channels are effectively allocated, which provides convenience and guarantee for building the SASED-YOLO model.

[0155] Example 6:

[0156] Based on Example 1, this embodiment provides a multi-scale defect detection method for wind turbine blade surfaces based on the SASED-YOLO algorithm. Figure 4 As shown, in step 4, context information from different receptive fields is fused based on the third module, including:

[0157] Extracting configuration parameters of the third module, where the configuration parameters include a global receptive field branch, a cascaded maximum pooling branch, and a spatial and channel reconstruction convolution branch;

[0158] Based on the global receptive field branch, 1×1 convolution and average pooling are used to compress the number of channels and spatial dimensions of the feature map respectively, and the spatial size of the feature map is adjusted according to the bilinear interpolation method based on the compression result to obtain the first context information;

[0159] Based on the cascaded maximum pooling branch, three 5×5 convolutions are used to extract features of different receptive fields from top to bottom to obtain the second context information;

[0160] Based on the spatial and channel reconstruction convolution branch, the spatial reconstruction unit and the channel reconstruction unit are used to remove redundant information in the feature map to obtain the third context information;

[0161] The first context information, the second context information, and the third context information are fused.

[0162] In this embodiment, in the global receptive field branch, the number of channels of the feature map is first compressed through a 1×1 convolution operation to help the model focus on key features and avoid excessive loss of local detail information in the subsequent average pooling process. Then, an adaptive average pooling operation is used to extract the global context information of the feature map and reduce the spatial dimension to a size of 1×1, thereby effectively reducing the computational burden. Finally, bilinear interpolation is used for upsampling to adjust the spatial size of the feature map to ensure that the feature map output by this branch is consistent with the feature maps of other branches in terms of spatial dimension.

[0163] In the maximum pooling branch, three layers of 5×5 maximum pooling are used to process the feature map step by step from top to bottom to extract local details of different scales while maintaining the ability to fuse multi-scale features.

[0164] In the spatial and channel reconstruction branches, the segmentation-transformation-fusion method is used to deeply analyze and select the input features, remove redundant noise information in the feature image, optimize feature expression, and improve the performance of subsequent tasks.

[0165] In this embodiment, the configuration parameters refer to the functional attributes and service processing parameters of the third module.

[0166] In this embodiment, the first context information refers to the result obtained by processing the number of channels and spatial dimensions of the feature map through the global receptive field branch.

[0167] In this embodiment, the second context information refers to the result obtained by processing the feature map using a cascaded maximum pooling branch.

[0168] In this embodiment, the third context information refers to the result obtained by processing the feature map through the spatial and channel reconstruction convolution branch.

[0169] The working principle and beneficial effects of the above technical solution are: by adopting the global receptive field branch, cascaded maximum pooling branch, and spatial and channel reconstruction convolution branch in the third module to process the feature map respectively, the first context information, second context information and third context information under different branches can be effectively determined. Finally, the obtained first context information, second context information and third context information are fused to achieve effective determination of the global spatial features of the feature map.

[0170] Example 7:

[0171] Based on Example 1, this embodiment provides a method for detecting multi-scale defects on the surface of a wind turbine blade based on the SASED-YOLO algorithm. Step 5 includes:

[0172] Based on the attention mechanism, the multi-channel selective attention module and the position-aware self-attention module are combined, and the target features in space and channels are extracted. The weights of the feature map are adjusted according to the target features to obtain the feature information of the wind turbine blade defect image.

[0173] In this embodiment, the target features refer to the spatial and channel features obtained by segmenting the feature map through the attention mechanism.

[0174] The working principle and beneficial effects of the above technical solution are: extracting target features in space and channels through the attention mechanism, and adjusting the weights of the feature maps according to the target features, so as to effectively determine the feature information of the defective images of wind turbine blades.

[0175] Example 8:

[0176] Based on Example 7, this embodiment provides a method for detecting multi-scale defects on the surface of wind turbine blades based on the SASED-YOLO algorithm. The method extracts spatial and channel target features according to the multi-channel selective attention module, including:

[0177] The multi-channel selective attention module includes a first processing unit and a second processing unit;

[0178] Wherein, the first processing unit is used to:

[0179] Divide the feature map X to obtain N sub-feature maps;

[0180] Perform mean pooling operation on the width and height of N sub-feature maps;

[0181] Perform a depthwise convolution operation on the N sub-feature maps after the mean pooling operation according to a preset convolution kernel, and extract spatial information of the feature maps at different scales according to the results of the depthwise convolution operation to obtain a first target feature map and a second target feature map;

[0182] The first target feature map and the second target feature map are spliced based on the channel dimension to obtain a third target feature map. At the same time, the third target feature map is group normalized and the spatial region is activated according to a preset function.

[0183] The second processing unit is configured to:

[0184] Perform maximum pooling and averaging operations on the input feature map X to obtain matrix a and matrix b;

[0185] Extract the local features of matrix a and the global features of matrix b based on the fully connected layer, and add and fuse the local features of matrix a and the global features of matrix b to obtain the target fusion result;

[0186] Map the target fusion result to a first preset range based on a preset function, determine the attention weight according to the mapping result, and input the attention weight into the feature map X to obtain a feature tensor;

[0187] Based on the dimension direction, the feature tensor and the spatial region are spliced to obtain a new feature map P;

[0188] In the position-aware self-attention module, the new feature map P is downsampled according to the average pooling with a preset step size to obtain the feature y;

[0189] Perform group normalization on feature y to obtain feature map n i ;

[0190] For feature map n i Perform deep point-by-point convolution processing to extract feature map n i The feature information of the feature vector is obtained by calculating the weights according to the multi-head attention mechanism and performing weighted aggregation, and the feature map Attn is obtained according to the feature vector.

[0191] The feature map Attn is reshaped and the height dimension and the number of attention heads are merged to obtain the channel dimension. The mean operation is performed on the height and width of the processed feature map Attn respectively and converted into feature weights within a second preset range based on a preset function. The feature weights are applied to the feature map P obtained after multi-channel selective attention feature extraction.

[0192] In this embodiment, the feature map X is divided to obtain N sub-feature maps, where N is 4.

[0193] In this embodiment, the first preset range is [0, 1].

[0194] In this embodiment, the second preset range is [1, 1].

[0195] In this embodiment, in the multi-channel selective attention module, the feature map is processed by two different parts, one of which first processes the feature map X∈R B×C×H×W The width and height dimensions are averaged and pooled to obtain and Where C represents the number of channels of the feature map, i represents the i-th sub-feature map, N represents the number of sub-feature maps, N is set to 4, X h () Perform mean pooling along the height dimension, X w () Perform mean pooling along the width dimension; then apply the obtained four sub-feature maps to depthwise convolution with kernel sizes of 3, 5, 7, and 9 respectively to extract spatial structure information at different scales: Where DWConv1d() is a 1-dimensional depth convolution; and After the two new feature maps are formed, they are concatenated along the channel dimension, and then the concatenated feature maps are group normalized, and the Sigmoid function is used to activate specific spatial regions: Where φ() is the activation function and GN() is the group normalization function. Another part of the multi-channel selective attention module is used to perform spatial weighting on the feature map. First, the input feature map X is subjected to maximum pooling and average pooling operations to obtain matrices a and b. The captured local features of a and the global features of b are added and fused using a fully connected layer. Subsequently, the Sigmoid activation function is applied to map the fused results to the range of [0,1] to calculate the most refined attention weights. This weight vector is then applied to the input feature map X, allowing the model to adaptively weight specific spatial positions and channels to obtain the feature tensor S. Finally, the feature tensor S and the feature tensor Y are added in the dimensional direction. g Splicing is performed to obtain the feature map P: Among them, MLP() is the fully connected function, Maxpooling() is the maximum pooling function, Averagepooling() is the average pooling function, Sigmoid() is the activation function, Concat() is the channel dimension splicing function, and Conv() is a 1×1 standard convolution.

[0196] In the position-aware self-attention module, the fused feature P is first downsampled using an average pooling with a step size of 7 and a size of 7×7 to obtain the feature y. Then, the downsampled y is group normalized to obtain the feature map n. i : Where GN() is the group normalization function; then the information n i After three independent depth-wise convolutions, the feature is refined and cut to obtain feature N. mi: where σ s () is the SiLU activation function, BN() is the batch normalization function, Represents a depthwise convolution with a kernel size of 1 and a stride of 1. It is a point-by-point convolution. Secondly, the weights are calculated through the multi-head attention mechanism, and the three feature values of Query, Key, and Value are weighted and aggregated to obtain the feature vector Attn. Then, after adjusting the shape of the feature map Attn, its height dimension and the number of attention heads are merged into the channel dimension. After that, the height and width of the processed feature map are averaged and the Sigmoid activation function is applied to convert it into a [1,1] feature weight Attn. mean , and finally Attn mean Applied to the feature map P obtained after multi-channel selective attention feature extraction: Among them, transpose() is the dimension exchange function. represents multiplication, Softmax() is the normalization function, rearrange() is the dimension transformation function, and σ() is the Sigmoid activation function.

[0197] In this embodiment, the sub-feature map refers to an image region obtained by dividing the feature map.

[0198] In this embodiment, the preset convolution kernel is set in advance.

[0199] In this embodiment, the preset function is known in advance.

[0200] In this embodiment, the feature tensor refers to a tensor that contains data feature information. For image data, a color image is usually composed of height (height), width (width) and the number of channels (channel, such as RGB images have 3 channels). In the deep learning framework, such an image can be represented as a three-dimensional feature tensor with a shape of ((height, width, channel)). For example, a (224\times224) pixel RGB image has a feature tensor shape of ((224,224,3)). This feature tensor contains various feature information of the image, such as edges, textures, etc. These features are gradually extracted through operations such as convolutional layers and pooling layers in models such as convolutional neural networks (CNN).

[0201] In this embodiment, the preset step size is set in advance.

[0202] The working principle and beneficial effects of the above technical solution are: by processing the feature map through the multi-channel selective attention module, the target features in space and channels can be effectively extracted, ensuring the accuracy and reliability of the target features finally obtained.

[0203] Example 9:

[0204] Based on Example 1, this embodiment provides a multi-scale defect detection method for a wind turbine blade surface based on the SASED-YOLO algorithm. In step 6, a network model is trained based on a training set and a validation set based on the weight distribution result to obtain a SASED-YOLO model. Defect detection is then performed on a wind turbine blade surface inspection image based on the SASED-YOLO model, including:

[0205] S61: Input the paths of the training set and validation set and the initial training weight file path into the parameters required for network model training to obtain the SASED-YOLO model;

[0206] S62: Perform defect detection on the wind turbine blade surface inspection image based on the SASD-YOLO model and output the target recognition result.

[0207] The working principle and beneficial effects of the above technical solution are: by training the network model according to the weight distribution results and the training set and validation set, the SASED-YOLO model can be accurately and effectively acquired, and the surface inspection images of wind turbine blades can be used for defect detection through the SASED-YOLO model, thereby ensuring the efficiency and accuracy of defect detection.

[0208] Example 10:

[0209] Based on Example 9, this embodiment provides a multi-scale defect detection method for wind turbine blade surfaces based on the SASED-YOLO algorithm. The target recognition results include: surface sand holes, damage, cracks, surface oil stains, fractures and damage, lightning strikes, surface attachments, and surface rust.

[0210] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A multi-scale defect detection method for wind turbine blade surface based on SASED-YOLO algorithm, characterized in that: include: Step 1: Obtain a wind turbine blade defect image, and perform preprocessing and image enhancement on the wind turbine blade defect image; Step 2: cascade the M first modules of the feature extraction network, and sequentially downsample the processed wind turbine blade defect image according to the spatial resolution reduction value of the feature map, and perform feature extraction based on the downsampling result to obtain a feature map of the wind turbine blade defect image; Step 3: Calibrate the feature map channel weights based on the second module; Step 4: Fusion of contextual information from different receptive fields based on the third module; Step 5: Focus on the dependencies between spaces and channels based on the attention mechanism, and assign attention weights to multi-scale targets in different feature maps based on the dependencies; Step 6: Based on the weight distribution results, the network model is trained according to the training set and the validation set to obtain the SASED-YOLO model, and defect detection is performed on the wind turbine blade surface inspection image based on the SASED-YOLO model.

2. The method for detecting multi-scale defects on the surface of wind turbine blades based on the SASED-YOLO algorithm according to claim 1, characterized in that: In step 1, a wind turbine blade defect image is obtained, and the wind turbine blade defect image is preprocessed and image enhanced, including: Accessing a preset image database and extracting a set of wind turbine blade defect images based on the access results; Performing object recognition on each wind turbine blade defect image in the extracted wind turbine blade defect image set, and screening the wind turbine blade defect images based on the object recognition results to obtain a valid wind turbine blade defect image set; Randomly select four wind turbine blade defect images from a valid wind turbine blade defect image set, and flip the randomly selected wind turbine blade defect images in relative directions respectively; At the same time, the defective image of the wind turbine blade after flipping is resized based on a preset size index, and based on the size scaling result, a neural network is called to perform color gamut transformation on the defective image of the wind turbine blade according to a multi-dimensional color gamut quantization index; The enhanced wind turbine blade defect image is obtained based on the color gamut transformation, and the enhanced wind turbine blade defect image is split into a training set, a verification set and a test set based on a preset ratio.

3. The method for detecting multi-scale defects on the surface of wind turbine blades based on the SASED-YOLO algorithm according to claim 1, characterized in that: In step 2, the feature extraction network is cascaded to M first modules, and the processed wind turbine blade defect image is sequentially downsampled according to the spatial resolution reduction value of the feature map, and feature extraction is performed based on the downsampling result to obtain a feature map of the wind turbine blade defect image, including: Acquire structural parameters of the first module based on the management terminal, and determine a connection interface of the first module during cascading based on the structural parameters; Extracting service execution parameters of the feature extraction network and the first module, determining a logical order of feature extraction based on the service execution parameters, and determining a cascading order of the feature extraction network and the first module based on the logical order; Cascading the feature extraction network and the M first modules based on the cascade order and the connection interface, configuring the input and output format compatibility of the feature extraction network and the first modules based on the cascade result, and adapting the hyperparameters of the feature extraction network and the first modules based on the input and output format compatibility configuration result and task requirements; At the same time, a spatial resolution reduction value of the feature map is obtained, and a downsampling ratio is determined based on the spatial resolution reduction value of the feature map, and the processed wind turbine blade defect image is downsampled based on the downsampling ratio; Based on the hyperparameter adaptation results, the feature extraction network and the first module are divided into a dual-branch feature extraction structure, and feature extraction is performed on the downsampling results based on the differential feature extraction strategy of the dual-branch feature extraction structure; The extracted features are correlated and summarized to obtain the feature map of the wind turbine blade defect image.

4. The method for detecting multi-scale defects on the surface of wind turbine blades based on the SASED-YOLO algorithm according to claim 3, characterized in that: The downsampling results are extracted using a differential feature extraction strategy based on a dual-branch feature extraction structure, including: Obtain the obtained dual-branch feature extraction structure, and map and match the dual-branch feature extraction structure with the difference feature extraction strategy to obtain the specific feature extraction strategy corresponding to each branch feature extraction structure, where one branch feature extraction structure is slice feature extraction and the other branch feature extraction structure is convolutional network extraction; When extracting features for slices: Determine the slice size based on task requirements and wind turbine blade defect images, and determine the step size based on feature extraction requirements; Performing a set sequence of slicing operations on the wind turbine blade defect image based on the slice size and step size, and obtaining a plurality of slice regions corresponding to the wind turbine blade defect image based on the slicing operations; Sequentially numbering the plurality of slice regions based on a set sequence, and sequentially adjusting the contrast of each slice region based on the sequence numbering result; Based on the contrast adjustment result, the corresponding slice area is input into the preset convolutional neural network for feature extraction to obtain the feature representation corresponding to each slice area; Performing associative splicing on feature representations of adjacent slice regions based on sequential numbers, and adaptively adjusting the splicing gap of the associated splicing regions based on the step size to obtain a first feature map; When extracted for a convolutional network: The size of the wind turbine blade defect image is cropped based on 3×3 maximum pooling and 1×1 convolution, and feature extraction is performed based on the cropping result to obtain the second feature map; The first feature map and the second feature map are associated and summarized to obtain a feature map of the wind turbine blade defect image.

5. The method for detecting multi-scale defects on the surface of wind turbine blades based on the SASED-YOLO algorithm according to claim 1, characterized in that: In step 3, the feature map channel weights are calibrated based on the second module, including: Extract the channel adaptation mechanism of the second module and use 1×1 global average pooling to adjust the feature map size based on the channel adaptation mechanism; The pooled results are processed based on the fully connected layer, and the weight distribution of the feature map channels is obtained based on the processing results.

6. The method for detecting multi-scale defects on the surface of wind turbine blades based on the SASED-YOLO algorithm according to claim 1, characterized in that: In step 4, contextual information from different receptive fields is fused based on the third module, including: Extracting configuration parameters of the third module, where the configuration parameters include a global receptive field branch, a cascaded maximum pooling branch, and a spatial and channel reconstruction convolution branch; Based on the global receptive field branch, 1×1 convolution and average pooling are used to compress the number of channels and spatial dimensions of the feature map respectively, and the spatial size of the feature map is adjusted according to the bilinear interpolation method based on the compression result to obtain the first context information; Based on the cascaded maximum pooling branch, three 5×5 convolutions are used to extract features of different receptive fields from top to bottom to obtain the second context information; Based on the spatial and channel reconstruction convolution branch, the spatial reconstruction unit and the channel reconstruction unit are used to remove redundant information in the feature map to obtain the third context information; The first context information, the second context information, and the third context information are fused.

7. The method for detecting multi-scale defects on the surface of wind turbine blades based on the SASED-YOLO algorithm according to claim 1, characterized in that: Step 5 includes: Based on the attention mechanism, the multi-channel selective attention module and the position-aware self-attention module are combined, and the target features in space and channels are extracted. The weights of the feature map are adjusted according to the target features to obtain the feature information of the wind turbine blade defect image.

8. The method for detecting multi-scale defects on the surface of wind turbine blades based on the SASED-YOLO algorithm according to claim 7, characterized in that: The target features extracted on space and channels according to the multi-channel selective attention module include: The multi-channel selective attention module includes a first processing unit and a second processing unit; Wherein, the first processing unit is used to: Divide the feature map X to obtain N sub-feature maps; Perform mean pooling operation on the width and height of N sub-feature maps; Perform a depthwise convolution operation on the N sub-feature maps after the mean pooling operation according to a preset convolution kernel, and extract spatial information of the feature maps at different scales according to the results of the depthwise convolution operation to obtain a first target feature map and a second target feature map; The first target feature map and the second target feature map are spliced based on the channel dimension to obtain a third target feature map. At the same time, the third target feature map is group normalized and the spatial region is activated according to a preset function. The second processing unit is configured to: Perform maximum pooling and averaging operations on the input feature map X to obtain matrix a and matrix b; Extract the local features of matrix a and the global features of matrix b based on the fully connected layer, and add and fuse the local features of matrix a and the global features of matrix b to obtain the target fusion result; Map the target fusion result to a first preset range based on a preset function, determine the attention weight according to the mapping result, and input the attention weight into the feature map X to obtain a feature tensor; Based on the dimension direction, the feature tensor is spliced with the spatial region to obtain a new feature map P; In the position-aware self-attention module, the new feature map P is downsampled according to the average pooling with a preset step size to obtain the feature y; Perform group normalization on feature y to obtain feature map n i ; For feature map n i Perform deep point-by-point convolution processing to extract feature map n i The feature information of the feature vector is obtained by calculating the weights according to the multi-head attention mechanism and performing weighted aggregation, and the feature map Attn is obtained according to the feature vector. The feature map Attn is reshaped and the height dimension and the number of attention heads are merged to obtain the channel dimension. The mean operation is performed on the height and width of the processed feature map Attn respectively and converted into feature weights within a second preset range based on a preset function. The feature weights are applied to the feature map P obtained after multi-channel selective attention feature extraction.

9. The method for detecting multi-scale defects on the surface of wind turbine blades based on the SASED-YOLO algorithm according to claim 1, characterized in that: In step 6, the network model is trained based on the training set and the validation set based on the weight distribution result to obtain the SASED-YOLO model. The surface inspection image of the wind turbine blade is then subjected to defect detection based on the SASED-YOLO model, including: S61: Input the paths of the training set and validation set and the initial training weight file path into the parameters required for network model training to obtain the SASED-YOLO model; S62: Perform defect detection on the wind turbine blade surface inspection image based on the SASD-YOLO model and output the target recognition result.

10. The method for detecting multi-scale defects on the surface of wind turbine blades based on the SASED-YOLO algorithm according to claim 9, characterized in that: Target identification results include: surface sand holes, damage, cracks, surface oil stains, fractures and damage, lightning strikes, surface attachments, and surface rust.

Images