A wind power blade surface multi-scale defect detection method based on a SASED-YOLO algorithm

Abstract

The application provides a wind power blade surface multi-scale defect detection method based on a SASED-YOLO algorithm, comprising: pre-processing and image enhancement processing of a wind power blade defect image; cascading M first modules to a feature extraction network, and performing down-sampling on the processed wind power blade defect image, and performing feature extraction based on the down-sampling result to obtain a feature map of the wind power blade defect image; calibrating a feature map channel weight based on a second module; fusing context information from different receptive fields based on a third module; paying attention to the dependency relationship between space and channels based on an attention mechanism, and distributing attention weights of multi-scale targets in different feature maps based on the dependency relationship; training a network model according to a training set and a verification set based on the weight distribution result to obtain a defect detection model, and detecting defects of a wind power blade surface detection image based on the defect detection model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of target detection, and particularly relates to a wind turbine blade surface multi-scale defect detection method based on a SASED-YOLO algorithm. BACKGROUND

[0002] On the future development road, perfecting the green and low-carbon development mechanism and accelerating the planning and construction of a new energy system have become a global consensus, so the development prospect of the wind power industry is promising. With continuous technological innovation and the gradual perfection of the industrial chain, floating offshore wind power is steadily moving from the experimental stage to commercial application. As one of the key components of a wind turbine generator system, the safe operation of a wind turbine blade is an important factor in ensuring the stable operation of the wind turbine generator system. Most wind farms are built in harsh environments with strong winds. Due to long-term exposure to harsh natural environments, wind turbine blades may be damaged by lightning burns, cracking, acid rain erosion, breakage, oil corrosion, and the like, affecting power generation efficiency and service life. Therefore, the safe monitoring of wind turbine blades is very important work.

[0003] However, the commonly used detection methods for wind turbine blade defects include ultrasonic detection, infrared thermal imaging detection, and acoustic emission detection, and the like. In these detection methods, when the surface of the wind turbine blade is not smooth or has a coating, the detection accuracy will be greatly reduced.

[0004] Therefore, in order to overcome the above-mentioned defects, the present application provides a wind turbine blade surface multi-scale defect detection method based on a SASED-YOLO algorithm. SUMMARY

[0005] The present application provides a wind turbine blade surface multi-scale defect detection method based on a SASED-YOLO algorithm, which acquires a wind turbine blade defect image, processes and extracts features from the wind turbine blade defect image, and further analyzes the extracted features, thereby accurately and effectively constructing a SASED-YOLO model. Secondly, the wind turbine blade surface detection image is input into the obtained SASED-YOLO model for defect detection, and finally the input image is subjected to target detection, accurately judges the type of each defect in the wind turbine blade image, and locates the specific position of the defect, thereby improving the accuracy and reliability of the wind turbine blade surface multi-scale defect detection.

[0006] The present application provides a wind turbine blade surface multi-scale defect detection method based on a SASED-YOLO algorithm, which acquires a wind turbine blade defect image, processes and extracts features from the wind turbine blade defect image, and further analyzes the extracted features, thereby accurately and effectively constructing a SASED-YOLO model. Secondly, the wind turbine blade surface detection image is input into the obtained SASED-YOLO model for defect detection, and finally the input image is subjected to target detection, accurately judges the type of each defect in the wind turbine blade image, and locates the specific position of the defect, thereby improving the accuracy and reliability of the wind turbine blade surface multi-scale defect detection.

[0007] Step 1: Acquire a wind turbine blade defect image, and perform preprocessing and image enhancement processing on the wind turbine blade defect image.

[0008] Step 2: Cascade the feature extraction network with M first modules, and sequentially downsample the processed wind turbine blade defect image according to the spatial resolution reduction value of the feature map, and perform feature extraction based on the downsampled result to obtain the feature map of the wind turbine blade defect image;

[0009] Step 3: Calibrate the feature map channel weight based on the second module;

[0010] Step 4: Fuse context information from different receptive fields based on the third module;

[0011] Step 5: Based on the attention mechanism, focus on the dependency relationship between space and channel, and based on the dependency relationship, allocate attention weights to multi-scale targets in different feature maps;

[0012] Step 6: Based on the weight allocation result, train the network model according to the training set and the validation set to obtain the SASED-YOLO model, and perform defect detection on the wind turbine blade surface detection image based on the SASED-YOLO model.

[0013] Preferably, in step 1, the wind turbine blade defect image is obtained, and the wind turbine blade defect image is preprocessed and image enhanced, including:

[0014] Access the preset image database and extract the wind turbine blade defect image set based on the access result;

[0015] Perform object recognition on each wind turbine blade defect image in the extracted wind turbine blade defect image set, and perform filtering on the wind turbine blade defect image based on the object recognition result to obtain an effective wind turbine blade defect image set;

[0016] Randomly select four wind turbine blade defect images from the effective wind turbine blade defect image set, and flip the randomly selected wind turbine blade defect images in the relative direction respectively;

[0017] At the same time, based on the preset size index, the wind turbine blade defect image after flipping is scaled in size, and based on the size scaling result, the neural network is called to perform color gamut transformation on the wind turbine blade defect image according to the multi-dimensional color gamut quantization index;

[0018] Based on the color gamut transformation, the enhanced wind turbine blade defect image is obtained, and the enhanced wind turbine blade defect image is split into a training set, a validation set and a test set based on a preset ratio.

[0019] Preferably, a wind turbine blade surface multi-scale defect detection method based on the SASED-YOLO algorithm, in step 2, the feature extraction network is cascaded with M first modules, and the processed wind turbine blade defect image is sequentially down-sampled according to the spatial resolution reduction value of the feature map, and feature extraction is performed based on the down-sampling result to obtain the feature map of the wind turbine blade defect image, including:

[0020] Based on the management terminal, the structural parameters of the first module are obtained, and the connection interface of the first module when cascaded is determined based on the structural parameters;

[0021] The business execution parameters of the feature extraction network and the first module are extracted, and the logical order of feature extraction is determined based on the business execution parameters, and the cascading order of the feature extraction network and the first module is determined based on the logical order;

[0022] Based on the cascading order and the connection interface, the feature extraction network and the M first modules are cascaded, and the input and output format compatible configuration of the feature extraction network and the first module is performed based on the cascading result, and the hyperparameter adaptation of the feature extraction network and the first module is performed based on the input and output format compatible configuration result and the task demand;

[0023] At the same time, the spatial resolution reduction value of the feature map is obtained, and the down-sampling ratio is determined based on the spatial resolution reduction value of the feature map, and the processed wind turbine blade defect image is down-sampled based on the down-sampling ratio;

[0024] Based on the hyperparameter adaptation result, the feature extraction network and the first module are cut into a double-branch feature extraction structure, and the feature extraction of the down-sampling result is performed based on the difference feature extraction strategy of the double-branch feature extraction structure;

[0025] The extracted features are associated and summarized to obtain the feature map of the wind turbine blade defect image.

[0026] Preferably, a wind turbine blade surface multi-scale defect detection method based on the SASED-YOLO algorithm, the feature extraction of the down-sampling result is performed based on the difference feature extraction strategy of the double-branch feature extraction structure, including:

[0027] The obtained double-branch feature extraction structure is obtained, and the double-branch feature extraction structure and the difference feature extraction strategy are mapped and matched to obtain the specific feature extraction strategy corresponding to each branch feature extraction structure, wherein one branch feature extraction structure is slice feature extraction, and the other branch feature extraction structure is convolution network extraction;

[0028] When it is slice feature extraction:

[0029] The size of the slice is determined based on the task demand and the wind turbine blade defect image, and the step size is determined based on the feature extraction requirement;

[0030] The wind turbine blade defect image is subjected to a slice operation in a set order based on the size and step length of the slice, and a plurality of slice regions corresponding to the wind turbine blade defect image are obtained based on the slice operation;

[0031] The plurality of slice regions are sequentially numbered based on the set order, and each slice region is subjected to contrast adjustment in sequence based on the sequential numbering result;

[0032] The corresponding slice region is input to a pre-set convolutional neural network based on the contrast adjustment result for feature extraction, to obtain a feature representation corresponding to each slice region;

[0033] The feature representations of adjacent slice regions are associated and spliced based on the sequential numbering, and the associated spliced region is subjected to adaptive seam gap adjustment based on the step length, to obtain a first feature map;

[0034] When extracting for a convolutional network:

[0035] The size of the wind turbine blade defect image is cropped based on 3x3 max pooling and 1x1 convolution, and feature extraction is performed based on the cropping result to obtain a second feature map;

[0036] The first feature map and the second feature map are associated and summarized to obtain a feature map of the wind turbine blade defect image.

[0037] Preferably, a wind turbine blade surface multi-scale defect detection method based on the SASED-YOLO algorithm, in step 3, the feature map channel weight is calibrated based on the second module, including:

[0038] The channel adaptive mechanism of the second module is extracted, and the feature map size is adjusted based on the channel adaptive mechanism using 1x1 global average pooling;

[0039] The result after pooling is processed based on the fully connected layer, and the weight distribution of the feature map channel is obtained based on the processing result.

[0040] Preferably, a wind turbine blade surface multi-scale defect detection method based on the SASED-YOLO algorithm, in step 4, the context information from different receptive fields is fused based on the third module, including:

[0041] The configuration parameters of the third module are extracted, wherein the configuration parameters include a global receptive field branch, a cascaded maximum pooling branch, and a spatial and channel reconstruction convolution branch;

[0042] The channel number and spatial dimension of the feature map are compressed based on the global receptive field branch using 1x1 convolution and average pooling, and the spatial size of the feature map is adjusted based on the compression result according to the bilinear interpolation method, to obtain first context information;

[0043] The three 5*5 convolutions are used to extract features of different receptive fields of the features from top to bottom based on the cascaded maximum pooling branch, to obtain second context information;

[0044] The spatial reconstruction unit and the channel reconstruction unit are used to remove redundant information in the feature map based on the spatial and channel reconstruction convolution branch, to obtain third context information;

[0045] The first context information, the second context information and the third context information are fused.

[0046] Preferably, the wind turbine blade surface multi-scale defect detection method based on the SASED-YOLO algorithm comprises the following steps:

[0047] The multi-channel selective attention module and the position perception self-attention module are combined based on the attention mechanism, and target features in space and channels are extracted, the weights of the feature map are adjusted according to the target features, and the feature information of the wind turbine blade defect image is obtained.

[0048] Preferably, the wind turbine blade surface multi-scale defect detection method based on the SASED-YOLO algorithm comprises the following steps:

[0049] The multi-channel selective attention module comprises a first processing unit and a second processing unit.

[0050] The first processing unit is configured to:

[0051] The feature map X is divided to obtain N sub-feature maps;

[0052] The width and height of the N sub-feature maps are subjected to mean pooling operation;

[0053] The N sub-feature maps subjected to the mean pooling operation are subjected to depth convolution operation according to a preset convolution kernel, and spatial information of the feature map at different scales is extracted according to a depth convolution operation result, to obtain a first target feature map and a second target feature map;

[0054] The first target feature map and the second target feature map are spliced based on the channel dimension to obtain a third target feature map, and the third target feature map is subjected to group normalization processing, and a spatial region is activated according to a preset function;

[0055] The second processing unit is configured to:

[0056] The input feature map X is subjected to maximum pooling and averaging operation to obtain a matrix a and a matrix b;

[0057] The local features of the matrix a and the global features of the matrix b are extracted based on the full connection layer, and the local features of the matrix a and the global features of the matrix b are added and fused to obtain a target fusion result;

[0058] The target fusion result is mapped into a first preset range based on a preset function, and an attention weight is determined according to a mapping result, and the attention weight is input into the feature map X to obtain a feature tensor;

[0059] The feature tensor and the spatial region are spliced based on a dimension direction to obtain a new feature map P;

[0060] In the position-aware self-attention module, the new feature map P is down-sampled by average pooling based on a preset step to obtain a feature y;

[0061] The feature y is group-normalized to obtain a feature map n i ;

[0062] The feature map n i is processed by deep point-wise convolution to extract feature information of the feature map n i , and a feature vector is obtained by calculating a weight based on a multi-head attention mechanism and weighted aggregation, and a feature map Attn is obtained based on the feature vector;

[0063] The feature map Attn is adjusted in shape and merged with the height dimension and the number of attention heads to obtain a channel dimension, and the height and width of the processed feature map Attn are respectively subjected to mean value operation and converted into a feature weight in a second preset range based on a preset function, and the feature weight is applied to the feature map P obtained after multi-channel selective attention feature extraction.

[0064] Preferably, a wind turbine blade surface multi-scale defect detection method based on the SASED-YOLO algorithm, in step 6, based on the weight distribution result, the network model is trained based on the training set and the validation set to obtain the SASED-YOLO model, and the wind turbine blade surface detection image is detected based on the SASED-YOLO model, including:

[0065] S61: The paths of the training set and the validation set and the path of the initial training weight file are input into the parameters required for network model training to obtain the SASED-YOLO model;

[0066] S62: Based on the SASED-YOLO model, the wind turbine blade surface detection image is detected for defect detection, and a target recognition result is output.

[0067] Preferably, a wind turbine blade surface multi-scale defect detection method based on the SASED-YOLO algorithm, the target recognition result includes: surface sand eye, damage, crack, surface oil stain, fracture damage, lightning stroke, surface attachment, surface rust.

[0068] Compared with the prior art, the present application has the following advantages:

[0069] By acquiring the wind turbine blade defect image, processing and feature extraction are performed on the wind turbine blade defect image, and the extracted features are further analyzed, so that the SASED-YOLO model is accurately and effectively constructed, and then the wind turbine blade surface detection image is input into the obtained SASED-YOLO model for defect detection, and finally the input image is subjected to target detection, the category of each defect in the wind turbine blade image is accurately judged, and the specific position of the defect is located, and the accuracy and reliability of the wind turbine blade surface multi-scale defect detection are improved.

[0070] Other features and advantages of the present application will be described in the following description, and some will become apparent from the description, or will be understood from the practice of the present application. The objects and other advantages of the present application can be achieved and obtained by the structures specifically pointed out in the present application file.

[0071] The technical solutions of the present application will be further described in detail below with the help of the drawings and examples. BRIEF DESCRIPTION OF DRAWINGS

[0072] The accompanying drawings are used to provide a further understanding of the present application, and constitute a part of the specification, together with the embodiments of the present application, used to explain the present application, and do not constitute a limitation on the present application. In the drawings:

[0073] Figure 1 A flow chart of a wind turbine blade surface multi-scale defect detection method based on the SASED-YOLO algorithm in an embodiment of the present application;

[0074] Figure 2 A principle diagram of a wind turbine blade surface multi-scale defect detection method based on the SASED-YOLO algorithm in an embodiment of the present application;

[0075] Figure 3 A principle diagram of step 3 in a wind turbine blade surface multi-scale defect detection method based on the SASED-YOLO algorithm in an embodiment of the present application;

[0076] Figure 4 A principle diagram of step 4 in a wind turbine blade surface multi-scale defect detection method based on the SASED-YOLO algorithm in an embodiment of the present application. DETAILED DESCRIPTION

[0077] The preferred embodiments of the present application are described below in conjunction with the accompanying drawings, it should be understood that the preferred embodiments described here are only used to explain and illustrate the present application, and are not used to limit the present application.

[0078] Embodiment 1:

[0079] This embodiment provides a wind turbine blade surface multi-scale defect detection method based on SASED-YOLO algorithm, as shown in Figure 1 , which comprises:

[0080] Step 1: Obtain the wind turbine blade defect image, and pre-process and image enhance the wind turbine blade defect image;

[0081] Step 2: Cascade M first modules to the feature extraction network, and sequentially down-sample the processed wind turbine blade defect image according to the spatial resolution reduction value of the feature map, and extract features based on the down-sampling result to obtain the feature map of the wind turbine blade defect image;

[0082] Step 3: Calibrate the feature map channel weight based on the second module;

[0083] Step 4: Fuse the context information from different receptive fields based on the third module;

[0084] Step 5: Based on the attention mechanism, focus on the dependency relationship between space and channel, and distribute the attention weight of the multi-scale target in different feature maps based on the dependency relationship;

[0085] Step 6: Based on the weight distribution result, train the network model according to the training set and the validation set, obtain the SASED-YOLO model, and detect defects on the wind turbine blade surface detection image based on the SASED-YOLO model.

[0086] In this embodiment, the principle diagram of the wind turbine blade surface multi-scale defect detection method based on SASED-YOLO algorithm is as shown in Figure 2 .

[0087] In this embodiment, the first module is FADonw module.

[0088] In this embodiment, the second module is C2f_SENetV2 module.

[0089] In this embodiment, the third module is SPPSCCAP module.

[0090] In this embodiment, the image enhancement processing refers to adjusting the color gamut and contrast of the wind turbine blade defect image.

[0091] In this embodiment, the context information refers to the relationship between the wind turbine blade defect image and the surrounding environment.

[0092] In this embodiment, preprocessing refers to denoising and the like of the wind turbine blade defect image.

[0093] In this embodiment, the feature extraction network is constructed in advance and is used to extract features of the wind turbine blade defect image.

[0094] In this embodiment, the spatial resolution reduction value is set in advance, and the purpose is to reduce the spatial resolution of the feature map and enhance the ability of the deep network to capture global features for the global spatial information of the defect features.

[0095] In this embodiment, downsampling is a process of reducing the image resolution by reducing the number of pixels in the image.

[0096] In this embodiment, the feature map channel weight calibration refers to the operation of assigning a weight value to each channel of the feature map, and the weight value is used to measure the importance of the feature information contained in each channel.

[0097] The working principle and beneficial effects of the above technical solution are: by acquiring a wind turbine blade defect image, processing and feature extraction are performed on the wind turbine blade defect image, and the extracted features are further analyzed, thereby realizing accurate and effective construction of the SASED-YOLO model. Secondly, the wind turbine blade surface detection image is input into the obtained SASED-YOLO model for defect detection, and finally the input image is subjected to target detection, the category of each defect in the wind turbine blade image is accurately judged, and the specific position of the defect is located, thereby improving the accuracy and reliability of the wind turbine blade surface multi-scale defect detection.

[0098] Embodiment 2:

[0099] Based on embodiment 1, the embodiment provides a wind turbine blade surface multi-scale defect detection method based on the SASED-YOLO algorithm, in step 1, a wind turbine blade defect image is acquired, and the wind turbine blade defect image is preprocessed and image enhanced, including:

[0100] Accessing a pre-set image database and extracting a wind turbine blade defect image set based on the access result;

[0101] Performing object recognition on each wind turbine blade defect image in the extracted wind turbine blade defect image set, and performing filtering on the wind turbine blade defect image based on the object recognition result to obtain an effective wind turbine blade defect image set;

[0102] Randomly selecting four wind turbine blade defect images from the effective wind turbine blade defect image set, and respectively performing relative direction flipping on the randomly selected wind turbine blade defect images;

[0103] Meanwhile, the wind turbine blade defect image after the flipping processing is scaled in size based on a preset size index, and a neural network is called based on the size scaling result to perform color gamut transformation on the wind turbine blade defect image according to a multi-dimensional color gamut quantization index;

[0104] The wind turbine blade defect image after the enhancement processing is obtained based on the color gamut transformation, and the wind turbine blade defect image after the enhancement processing is split into a training set, a verification set and a test set based on a preset ratio.

[0105] In this embodiment, the preset image database is set in advance and is used to store wind turbine blade defect images.

[0106] In this embodiment, the object recognition refers to the recognition of the wind turbine blade in the wind turbine blade defect image.

[0107] In this embodiment, the effective wind turbine blade defect image set refers to the images that can effectively record the wind turbine blade defects after screening each image in the extracted wind turbine blade defect image set.

[0108] In this embodiment, the preset size index is set in advance.

[0109] In this embodiment, the multi-dimensional color gamut quantization index is set in advance and is used to represent the numerical value of the color gamut adjustment of the wind turbine blade defect image.

[0110] In this embodiment, the preset ratio is 7:2:1.

[0111] The working principle and beneficial effects of the above technical solution are as follows: by accessing the preset image database, the wind turbine blade defect image set is accurately and effectively determined, and the obtained wind turbine blade defect image is screened and processed, the wind turbine blade image is reliably preprocessed and image enhanced, and the reliability of the finally obtained wind turbine blade defect image is ensured, which provides convenience and protection for constructing the SASED-YOLO model.

[0112] Embodiment 3:

[0113] Based on embodiment 1, the embodiment provides a wind turbine blade surface multi-scale defect detection method based on the SASED-YOLO algorithm, in step 2, the feature extraction network is cascaded with M first modules, and the processed wind turbine blade defect image is down-sampled in turn according to the spatial resolution reduction value of the feature map, and the feature extraction is performed based on the down-sampling result to obtain the feature map of the wind turbine blade defect image, including:

[0114] The structure parameters of the first module are obtained based on the management terminal, and the connection interface of the first module when cascaded is determined based on the structure parameters;

[0115] extracting a service execution parameter of the feature extraction network and the first module, determining a logical order of feature extraction based on the service execution parameter, and determining a cascading order of the feature extraction network and the first module based on the logical order;

[0116] cascading the feature extraction network and the M first modules based on the cascading order and the connection interface, performing input-output format compatible configuration of the feature extraction network and the first module based on the cascading result, and performing hyperparameter adaptation of the feature extraction network and the first module based on the input-output format compatible configuration result and the task requirement;

[0117] Meanwhile, a spatial resolution reduction value of the feature map is obtained, and a down-sampling ratio is determined based on the spatial resolution reduction value of the feature map, and the processed wind power blade defect image is down-sampled based on the down-sampling ratio;

[0118] The feature extraction network and the first module are divided into a double-branch feature extraction structure based on the hyperparameter adaptation result, and the down-sampling result is feature-extracted based on a difference feature extraction strategy of the double-branch feature extraction structure.

[0119] The extracted features are associated and summarized to obtain a feature map of the wind power blade defect image.

[0120] In this embodiment, the FADonw module includes a slice convolution part and a pooling part.

[0121] The slice convolution part divides the image into four different sub-regions, extracts features in each region, and finally compresses the spatial information to the channel dimension to enhance the detailed features of the fuzzy edges in the feature map.

[0122] The pooling part performs line-by-line scanning on the local region through 3x3 maximum pooling, removes unnecessary redundant features in the defect image, promotes the focusing ability of the network on global features, and reduces the spatial dimension of the feature map, and then uses 1x1 convolution to intercept the fine features X'2, greatly reducing the computational complexity of the network: X'2 = Conv(MaxPooling(X2)), where Conv() is a standard convolution and MaxPooling() is a maximum pooling.

[0123] In this embodiment, the structural parameters refer to the configuration parameters and functional parameters of the first module.

[0124] In this embodiment, the service execution parameter is the service execution process and service processing method of the feature extraction network and the first module when working.

[0125] In this embodiment, the input / output format compatible configuration refers to configuring data interaction between the feature extraction network and the first module to ensure that data can be effectively interacted between the feature extraction network and the first module.

[0126] In this embodiment, the hyperparameter adaptation refers to a process of finding a set of optimal or near-optimal hyperparameter values for a specific task, dataset, and model structure, i.e., configuring the working performance of the feature extraction network and the first module.

[0127] In this embodiment, the dual-branch feature extraction structure refers to splitting the feature extraction network and the first module into two parallel feature extraction structures and using different feature extraction methods for feature extraction.

[0128] The working principle and beneficial effects of the above technical solution are as follows: by analyzing the structural parameters of the first module, the connection interface of the first module is determined, then by determining the business execution parameters of the feature extraction network and the first module, the cascade order of the feature extraction network and the first module is determined according to the business execution parameters, so that the feature extraction network and the first module are cascaded according to the cascade order and the connection interface, finally, the hyperparameter adaptation is performed on the cascade result, and the wind turbine blade defect image is down-sampled through the hyperparameter adaptation result, so that the feature map of the wind turbine blade defect image is accurately and effectively determined, and the accuracy and reliability of the final feature map are ensured.

[0129] Embodiment 4:

[0130] Based on embodiment 3, the embodiment provides a wind turbine blade surface multi-scale defect detection method based on the SASED-YOLO algorithm, which performs feature extraction on the down-sampling result based on the difference feature extraction strategy of the dual-branch feature extraction structure, including:

[0131] The dual-branch feature extraction structure is obtained, and the dual-branch feature extraction structure and the difference feature extraction strategy are mapped and matched to obtain the specific feature extraction strategy corresponding to each branch feature extraction structure, wherein one branch feature extraction structure is slice feature extraction, and the other branch feature extraction structure is convolution network extraction.

[0132] When it is slice feature extraction:

[0133] The size of the slice is determined based on the task requirement and the wind turbine blade defect image, and the step size is determined based on the feature extraction requirement;

[0134] The wind turbine blade defect image is subjected to slice operation in a set order based on the size and step size of the slice, and a plurality of slice regions corresponding to the wind turbine blade defect image are obtained based on the slice operation.

[0135] The plurality of slice regions are sequentially numbered based on the set order, and each slice region is sequentially adjusted in contrast based on the sequential numbering result;

[0136] The corresponding slice region is input to the preset convolutional neural network based on the contrast adjustment result to obtain a feature representation corresponding to each slice region;

[0137] The feature representations of adjacent slice regions are associated and spliced based on the sequential numbering, and the associated spliced region is adaptively adjusted based on the step to obtain a first feature map;

[0138] When extracting for the convolutional network:

[0139] The size of the wind turbine blade defect image is cropped based on 3x3 max pooling and 1x1 convolution, and feature extraction is performed based on the cropping result to obtain a second feature map;

[0140] The first feature map and the second feature map are associated and summarized to obtain a feature map of the wind turbine blade defect image.

[0141] In this embodiment, the difference feature extraction strategy refers to the feature extraction strategy corresponding to the double-branch feature extraction structure respectively.

[0142] In this embodiment, the task requirement is known in advance.

[0143] In this embodiment, the preset convolutional neural network is trained in advance and is used to extract image features in the image, i.e., the final feature representation.

[0144] In this embodiment, the first feature map refers to the feature map of the wind turbine blade defect image extracted by slicing.

[0145] In this embodiment, the second feature map refers to the feature map of the wind turbine blade defect image extracted by the convolutional network.

[0146] The working principle and beneficial effects of the above technical solution are: by adopting a double-branch feature extraction structure to extract features of the wind turbine blade defect image respectively, and obtaining corresponding feature maps, finally associating and summarizing the obtained first feature map and second feature map, effectively obtaining the feature map of the wind turbine blade defect image, ensuring the accuracy and reliability of the final feature map.

[0147] Embodiment 5:

[0148] Based on the embodiment 1, the embodiment provides a wind turbine blade surface multi-scale defect detection method based on SASED-YOLO algorithm, as shown in Figure 3 As shown in FIG. 3, in step 3, the feature map channel weight is calibrated based on the second module, including:

[0149] The channel adaptation mechanism of the second module is extracted, and the feature map size is adjusted by 1×1 global average pooling based on the channel adaptation mechanism;

[0150] The pooling results are processed using a fully connected layer, and the weight allocation of the feature map channels is obtained based on the processing results.

[0151] In this embodiment, the C2f_SENetV2 module adopts a channel adaptive mechanism, which uses a compression-then-excitation method. The compression method uses 1×1 global average pooling to adjust the feature map size, and the excitation method uses a fully connected layer to process the pooled result and optimize the weight distribution between each channel.

[0152] In this embodiment, the C2f_SENetV2 module uses a compression-then-excitation approach. During compression, global average pooling is used to compress the spatial dimension of the input feature map, transforming the spatial information of each channel into global statistics, resulting in a feature image transformation R. B×C×1×1 Shape: During the activation process, four parallel fully connected branches are used to process the results after global average pooling. Each branch processes global information to varying degrees through parallel computation, aiming to enhance the network's ability to represent features from multiple perspectives.

[0153] In this embodiment, the channel adaptive mechanism is a configuration mechanism in the second module, used to adjust the size of the feature map according to the specific situation of the feature map.

[0154] The working principle and beneficial effects of the above technical solution are as follows: the feature map is processed through the channel adaptive mechanism in the second module, and the weights of the feature map channels are effectively allocated, which provides convenience and guarantee for building the SASED-YOLO model.

[0155] Example 6:

[0156] Based on Example 1, this example provides a method for detecting multi-scale defects on the surface of wind turbine blades based on the SASED-YOLO algorithm, such as... Figure 4 As shown, in step 4, contextual information from different receptive fields is fused based on the third module, including:

[0157] Extract the configuration parameters of the third module, which include the global receptive field branch, the cascaded max pooling branch, and the spatial and channel reconstruction convolution branch;

[0158] Based on the global receptive field branch, 1×1 convolution and average pooling are used to compress the number of channels and spatial dimension of the feature map, respectively. Based on the compression result, the spatial size of the feature map is adjusted according to bilinear interpolation to obtain the first context information.

[0159] The second context information is obtained by extracting features of different receptive fields of the features from top to bottom based on a cascaded maximum pooling branch using three 5*5 convolutions.

[0160] The third context information is obtained by removing redundant information in the feature map using a spatial reconstruction unit and a channel reconstruction unit based on a spatial and channel reconstruction convolution branch.

[0161] The first context information, the second context information and the third context information are fused.

[0162] In this embodiment, in the global receptive field branch, first, the channel number of the feature map is compressed through a 1*1 convolution operation, which helps the model focus on key features and avoids excessive loss of local detail information in the subsequent average pooling process. Then, an adaptive average pooling operation is used to extract the global context information of the feature map, and the spatial dimension is reduced to 1*1 size, thereby effectively reducing the computational burden. Finally, the bilinear interpolation method is used for upsampling to adjust the spatial size of the feature map, ensuring that the feature map output by this branch is consistent with the feature map of other branches in the spatial dimension.

[0163] In the maximum pooling branch, three layers of 5*5 maximum pooling are used to process the feature map from top to bottom, extracting local details of different scales while maintaining the fusion ability of multi-scale features.

[0164] In the spatial and channel reconstruction branch, the input features are analyzed and selected in depth using the segmentation-transformation-fusion method, and redundant noise information in the feature image is removed to optimize feature expression and improve the performance of subsequent tasks.

[0165] In this embodiment, the configuration parameter refers to the functional attribute and business processing parameter of the third module.

[0166] In this embodiment, the first context information refers to the result obtained after processing the channel number and spatial dimension of the feature map through the global receptive field branch.

[0167] In this embodiment, the second context information refers to the result obtained after processing the feature map using the cascaded maximum pooling branch.

[0168] In this embodiment, the third context information refers to the result obtained by processing the feature map through the spatial and channel reconstruction convolution branch.

[0169] The working principle and beneficial effects of the above technical solution are: by adopting the global receptive field branch, the cascaded maximum pooling branch, and the spatial and channel reconstruction convolution branch in the third module to process the feature map, the first context information, the second context information, and the third context information under different branches are effectively determined, and finally, the first context information, the second context information, and the third context information are fused to effectively determine the global spatial features of the feature map.

[0170] Embodiment 7:

[0171] Based on the embodiment 1, the embodiment provides a wind turbine blade surface multi-scale defect detection method based on the SASED-YOLO algorithm, and in step 5, the method comprises:

[0172] The multi-channel selective attention module and the position perception self-attention module are combined based on the attention mechanism, and target features on the space and the channel are extracted, the weight of the feature map is adjusted according to the target features, and the feature information of the wind turbine blade defect image is obtained.

[0173] In this embodiment, the target features refer to the features on the space and the channel obtained after the feature map is processed by the attention mechanism.

[0174] The working principle and beneficial effects of the above technical solution are: by extracting the target features on the space and the channel based on the attention mechanism, and adjusting the weight of the feature map according to the target features, the feature information of the wind turbine blade defect image is effectively determined.

[0175] Embodiment 8:

[0176] Based on the embodiment 7, the embodiment provides a wind turbine blade surface multi-scale defect detection method based on the SASED-YOLO algorithm, and the target features on the space and the channel extracted by the multi-channel selective attention module comprise:

[0177] The multi-channel selective attention module comprises a first processing unit and a second processing unit;

[0178] The first processing unit is configured to:

[0179] The feature map X is divided to obtain N sub-feature maps;

[0180] The width and the height of the N sub-feature maps are subjected to a mean pooling operation;

[0181] The N sub-feature maps subjected to the mean pooling operation are subjected to a depth convolution operation according to a preset convolution kernel, and spatial information of the feature map at different scales is extracted according to a depth convolution operation result to obtain a first target feature map and a second target feature map;

[0182] The first target feature map and the second target feature map are spliced based on a channel dimension to obtain a third target feature map, and the third target feature map is subjected to group normalization processing, and a spatial region is activated according to a preset function;

[0183] The second processing unit is configured to:

[0184] The input feature map X is subjected to a max-pooling and averaging operation to obtain a matrix a and a matrix b;

[0185] Local features of the matrix a and global features of the matrix b are extracted based on a full connection layer, and the local features of the matrix a and the global features of the matrix b are added and fused to obtain a target fusion result;

[0186] The target fusion result is mapped into a first preset range based on a preset function, and an attention weight is determined according to a mapping result, and the attention weight is input into the feature map X to obtain a feature tensor;

[0187] The feature tensor and the spatial region are spliced based on a dimension direction to obtain a new feature map P;

[0188] In the position-aware self-attention module, the new feature map P is subjected to a down-sampling processing according to a preset step average pooling to obtain a feature y;

[0189] The feature y is subjected to group normalization to obtain a feature map n i ;

[0190] The feature map n i is subjected to a deep point-wise convolution processing to extract feature information of the feature map n i , and a weight is calculated according to a multi-head attention mechanism and is weighted and aggregated to obtain a feature vector, and a feature map Attn is obtained according to the feature vector;

[0191] The feature map Attn is subjected to shape adjustment and merging of a height dimension and a number of attention heads to obtain a channel dimension, and a mean value operation is respectively performed on a height and a width of the processed feature map Attn, and a feature weight in a second preset range is converted based on a preset function, and the feature weight is applied to the feature map P obtained after multi-channel selective attention feature extraction.

[0192] In this embodiment, the feature map X is divided to obtain N sub-feature maps, where N is 4.

[0193] In this embodiment, the first preset range is [0, 1].

[0194] In this embodiment, the second preset range is [1, 1].

[0195] In this embodiment, in the multi-channel selective attention module, the feature map is processed by two different parts respectively. One part first performs mean pooling operation on the width and height dimensions of the feature map X e R B×C×H×W to obtain and where C represents the number of channels of the feature map, i represents the i-th sub-feature map, N represents the number of sub-feature maps, N is set to 4, X h () performs mean pooling operation along the height dimension, and X w () performs mean pooling operation along the width dimension; then the obtained four sub-feature maps are respectively applied to depth convolution with kernel size of 3, 5, 7 and 9 to extract spatial structure information of different scales: where DWConv1d() is 1-dimensional depth convolution; two new feature maps are obtained and After that, the channel dimension is spliced, then the feature map obtained by splicing is processed by group normalization, and the Sigmoid function is used to activate the specific spatial region: where φ() is the activation function, and GN() is the group normalization function. The other part in the multi-channel selective attention module is used for spatial weighting of the feature map. First, the input feature map X is subjected to max pooling and average pooling operations to obtain matrix a and matrix b. The local features of a and the global features of b are added and fused by using a fully connected layer. Then, the Sigmoid activation function is applied to map the fused result to the range of [0, 1], so as to calculate the most excellent attention weight. Then, this weight vector is applied to the input feature map X, so that the model can adaptively weight the specific spatial position and channel, thereby obtaining a feature tensor S. Finally, the feature tensor S and the feature tensor Y g are spliced in the dimension direction to obtain a feature map P: where MLP() is a fully connected function, Maxpooling() is a max pooling function, Averagepooling() is an average pooling function, Sigmoid() is an activation function, Concat() is a channel dimension splicing function, and Conv() is a standard convolution of 1x1.

[0196] In the position-aware self-attention module, first, the fused feature P is down-sampled to obtain a feature y by using an average pooling with a step of 7 and a size of 7x7, then the down-sampled y is processed by group normalization to obtain a feature map n i : where GN() is a group normalization function; then the obtained information n i is subjected to three independent depth pointwise convolutions to obtain a feature N mi where σ s () is a SiLU activation function, BN() is a batch normalization function, represents a deep convolution with a kernel size of 1 and a step of 1, is a point-wise convolution; secondly, the weight is calculated by a multi-head attention mechanism, and the feature vector Attn is obtained by weighted aggregation of Query, Key and Value three feature values; then the shape of the feature map Attn is adjusted, and the height dimension and the number of attention heads are merged into the channel dimension; then the height and width of the processed feature map are respectively subjected to mean operation and applied to the Sigmoid activation function, and converted into a feature weight Attn of [1, 1] mean , finally Attn mean is applied to the feature map P obtained after multi-channel selective attention feature extraction: where transpose() is a dimension exchange function, represents multiplication, Softmax() is a normalization function, rearrange() is a dimension transformation function, and σ() is a Sigmoid activation function.

[0197] In this embodiment, the sub-feature map refers to an image region obtained after the feature map is divided.

[0198] In this embodiment, the preset convolution kernel is set in advance.

[0199] In this embodiment, the preset function is known in advance.

[0200] In this embodiment, the feature tensor refers to a tensor containing data feature information. For image data, a color image is usually composed of height, width and channel number (such as a RGB image with 3 channels). In the deep learning framework, such an image can be represented as a three-dimensional feature tensor with a shape of ((height, width, channel)). For example, a (224\times224) pixel RGB image has a feature tensor shape of ((224, 224, 3)). This feature tensor contains various feature information of the image, such as edges and textures, which are gradually extracted by convolution layers, pooling layers and other operations in models such as convolutional neural networks (CNN).

[0201] In this embodiment, the preset step is set in advance.

[0202] ​The working principle and beneficial effects of the above technical solution are: the multi-channel selective attention module processes the feature map, effectively extracts the target features in space and channels, and guarantees the accuracy and reliability of the final target features.

[0203] Embodiment 9:

[0204] Based on the embodiment 1, the embodiment provides a wind power blade surface multi-scale defect detection method based on the SASED-YOLO algorithm, in step 6, the network model is trained based on the weight distribution result according to the training set and the validation set, and the SASED-YOLO model is obtained, and the wind power blade surface detection image is detected based on the SASED-YOLO model, including:

[0205] S61: input the path of the training set and the validation set and the initial training weight file path into the parameters required for network model training for training, and obtain the SASED-YOLO model;

[0206] S62: based on the SASED-YOLO model, the wind power blade surface detection image is detected for defect detection, and the target recognition result is output.

[0207] The working principle and beneficial effects of the above technical solution are: the network model is trained according to the weight distribution result and the training set and the validation set, the SASED-YOLO model is accurately and effectively obtained, and the wind power blade surface detection image is detected based on the SASED-YOLO model, and the efficiency and accuracy of the defect detection are ensured.

[0208] Embodiment 10:

[0209] Based on the embodiment 9, the embodiment provides a wind power blade surface multi-scale defect detection method based on the SASED-YOLO algorithm, and the target recognition result includes: surface sand eye, damage, crack, surface oil stain, fracture damage, lightning stroke, surface attachment, and surface rust.

[0210] Obviously, those skilled in the art can make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application also intends to include these modifications and variations.

Claims

1. A wind turbine blade surface multi-scale defect detection method based on a SASED-YOLO algorithm, characterized in that, The method comprises the following steps: Step 1: Obtain the wind turbine blade defect image, and perform preprocessing and image enhancement processing on the wind turbine blade defect image; Step 2: Cascade M first modules to the feature extraction network, and sequentially downsample the processed wind turbine blade defect image according to the spatial resolution reduction value of the feature map, and perform feature extraction based on the downsampled result to obtain the feature map of the wind turbine blade defect image, wherein the first module is an FADonw module; Step 3: Calibrate the channel weight of the feature map based on the second module, wherein the second module is a C2f_SENetV2 module; Step 4: Fuse context information from different receptive fields based on the third module, wherein the third module is an SPPSCCAP module; Step 5: Focus on the dependency between space and channel based on the attention mechanism, and allocate attention weights to multi-scale targets in different feature maps based on the dependency; Step 6: Train the network model based on the weight allocation result according to the training set and the validation set to obtain the SASED-YOLO model, and perform defect detection on the wind turbine blade surface detection image based on the SASED-YOLO model; In step 2, the feature extraction network is cascaded with M first modules, and the processed wind turbine blade defect image is sequentially downsampled according to the spatial resolution reduction value of the feature map, and feature extraction is performed based on the downsampled result to obtain the feature map of the wind turbine blade defect image, comprising: Obtain the structure parameters of the first module based on the management terminal, and determine the connection interface of the first module when cascading based on the structure parameters; Extract the business execution parameters of the feature extraction network and the first module, and determine the logical order of feature extraction based on the business execution parameters, and determine the cascading order of the feature extraction network and the first module based on the logical order; Cascade the feature extraction network and the M first modules based on the cascading order and the connection interface, and perform input-output format compatible configuration on the feature extraction network and the first module based on the cascading result, and perform hyperparameter adaptation on the feature extraction network and the first module based on the input-output format compatible configuration result and the task demand; At the same time, obtain the spatial resolution reduction value of the feature map, and determine the downsample ratio based on the spatial resolution reduction value of the feature map, and downsample the processed wind turbine blade defect image based on the downsample ratio; Based on the hyperparameter adaptation result, the feature extraction network and the first module are divided into a double-branch feature extraction structure, and the feature extraction network and the first module are divided into a double-branch feature extraction structure based on the difference feature extraction strategy of the double-branch feature extraction structure; Correlate and summarize the extracted features to obtain the feature map of the wind turbine blade defect image; In step 3, the channel weight of the feature map is calibrated based on the second module, comprising: Extract the channel adaptive mechanism of the second module, and adjust the feature map size using 1x1 global average pooling based on the channel adaptive mechanism; Process the pooled result based on the fully connected layer, and obtain the weight distribution of the feature map channel based on the processing result; In step 4, the context information from different receptive fields is fused based on the third module, comprising: extracting configuration parameters of the third module, wherein the configuration parameters comprise a global receptive field branch, a cascaded max-pooling branch, and a spatial and channel reconstruction convolution branch; based on the global receptive field branch, adopting 1x1 convolution and average pooling to compress the channel number and spatial dimension of the feature map respectively, and based on the compression result, adjusting the spatial size of the feature map according to the bilinear interpolation method to obtain first context information; based on the cascaded max-pooling branch, adopting three 5x5 convolutions to extract features of different receptive fields of the feature from top to bottom to obtain second context information; based on the spatial and channel reconstruction convolution branch, adopting a spatial reconstruction unit and a channel reconstruction unit to remove redundant information in the feature map to obtain third context information; fusing the first context information, the second context information and the third context information.

2. The method of claim 1, wherein the method is based on a SASED-YOLO algorithm. In step 1, the wind turbine blade defect image is obtained, and the wind turbine blade defect image is preprocessed and image enhanced, including: accessing the preset image database, and extracting a set of wind turbine blade defect images based on the access result; performing object recognition on each wind turbine blade defect image in the extracted set of wind turbine blade defect images, and performing filtering on the wind turbine blade defect image based on the object recognition result to obtain a set of effective wind turbine blade defect images; four wind turbine blade defect images are randomly selected from the set of effective wind turbine blade defect images, and the randomly selected wind turbine blade defect images are flipped in the relative direction respectively; At the same time, based on the preset size index, the wind turbine blade defect image after the flipping processing is scaled in size, and based on the size scaling result, the neural network is called to perform color gamut transformation on the wind turbine blade defect image according to the multi-dimensional color gamut quantization index; based on the color gamut transformation, the enhanced wind turbine blade defect image is obtained, and the enhanced wind turbine blade defect image is split into a training set, a validation set and a test set based on a preset ratio.

3. The method of claim 1, wherein the method is based on a SASED-YOLO algorithm. based on the difference feature extraction strategy of the double-branch feature extraction structure, the down-sampling result is extracted, including: obtaining the double-branch feature extraction structure, and mapping and matching the double-branch feature extraction structure and the difference feature extraction strategy to obtain the specific feature extraction strategy corresponding to each branch feature extraction structure, wherein one branch feature extraction structure is slice feature extraction, and the other branch feature extraction structure is convolution network extraction; when it is slice feature extraction: based on the task requirement and the wind turbine blade defect image, the size of the slice is determined, and the step is determined based on the feature extraction requirement; based on the size and step of the slice, the wind turbine blade defect image is sequentially sliced, and a plurality of slice regions corresponding to the wind turbine blade defect image are obtained based on the slicing operation; based on the sequential order, the plurality of slice regions are sequentially numbered, and each slice region is sequentially adjusted in contrast based on the sequential numbering result; based on the contrast adjustment result, the corresponding slice region is input into the preset convolutional neural network for feature extraction to obtain the feature representation corresponding to each slice region; based on the sequential numbering, the feature representations of adjacent slice regions are associated and spliced, and the associated splicing region is adaptively adjusted in the gap based on the step to obtain a first feature map; when it is convolution network extraction: The size of the wind turbine blade defect image is cropped based on 3*3 max pooling and 1*1 convolution, and feature extraction is performed based on the cropping result to obtain a second feature map; The first feature map and the second feature map are associated and summarized to obtain a feature map of the wind turbine blade defect image.

4. The method of claim 1, wherein the method is based on a SASED-YOLO algorithm. In step 5, it includes: Based on the attention mechanism, the multi-channel selective attention module and the position-aware self-attention module are combined, and the target features in the space and channel are extracted, the weight of the feature map is adjusted according to the target features, and the feature information of the wind turbine blade defect image is obtained.

5. The method of claim 4, wherein the method is based on a SASED-YOLO algorithm. According to the multi-channel selective attention module, the target features in the space and channel are extracted, including: The multi-channel selective attention module includes a first processing unit and a second processing unit; The first processing unit is configured to: Divide the feature map X to obtain N sub-feature maps; Perform mean pooling operation on the width and height of the N sub-feature maps; Perform depth convolution operation on the N sub-feature maps after the mean pooling operation according to the preset convolution kernel, and extract the spatial information of the feature map at different scales according to the depth convolution operation result to obtain a first target feature map and a second target feature map; The first target feature map and the second target feature map are spliced based on the channel dimension to obtain a third target feature map, and the third target feature map is normalized and activated in the spatial region according to the preset function; The second processing unit is configured to: Perform max pooling and averaging operation on the input feature map X to obtain matrix a and matrix b; Extract the local features of matrix a and the global features of matrix b based on the full connection layer, and add and fuse the local features of matrix a and the global features of matrix b to obtain a target fusion result; Map the target fusion result to a first preset range based on a preset function, determine the attention weight according to the mapping result, and input the attention weight into the feature map X to obtain a feature tensor; Splice the feature tensor and the spatial region based on the dimension direction to obtain a new feature map P; In the position-aware self-attention module, the new feature map P is down-sampled by average pooling with a preset step to obtain a feature y; grouping the features y into normalized features, obtaining a feature map n i ; The feature map n i is subjected to a deep pointwise convolution process to extract feature information of the feature map n i , and a weight is calculated and weighted aggregation is performed according to a multi-head attention mechanism to obtain a feature vector, and a feature map Attn is obtained according to the feature vector. Adjust the shape of the feature map Attn, merge the height dimension and the number of attention heads to obtain the channel dimension, and perform mean operation on the height and width of the processed feature map Attn respectively and convert them into feature weights in a second preset range based on a preset function, and apply the feature weights to the feature map P obtained after multi-channel selective attention feature extraction.

6. The method of claim 1, wherein the method is based on a SASED-YOLO algorithm. In step 6, based on the weight allocation result, the network model is trained according to the training set and the validation set to obtain a SASED-YOLO model, and the SASED-YOLO model is used for defect detection of the wind turbine blade surface detection image, including: S61: Input the path of the training set and the validation set and the path of the initial training weight file into the parameters required for network model training to obtain a SASED-YOLO model; S62: Perform defect detection on the wind turbine blade surface detection image based on the SASED-YOLO model, and output the target recognition result.

7. The method of claim 6, wherein the method is based on a SASED-YOLO algorithm. The target recognition result includes: surface sand, injury, crack, surface oil, broken damage, lightning, surface attachment, surface rust.

Images