A wind turbine blade tip detection method and system based on brightness self-adaption and multi-scale attention

Abstract

A wind turbine blade tip detection method and system based on brightness adaptation and multi-scale attention, comprising the following steps: S1, first collect multiple wind turbine blade images containing the blade tip, and after preprocessing, construct a data set; S2, replace the first convolutional layer in the backbone network of the original YOLOv8 network with a brightness perception evaluation and branch fusion module, a small target sensitive feature extraction backbone network and a multi-scale context attention module connected in turn, and after replacement, obtain an improved YOLOv8 network; S3, train the improved YOLOv8 network using the data set to obtain a trained YOLOv8 network; S4, deploy the trained YOLOv8 network to the device end and perform wind turbine blade tip detection to obtain a detection result. The present application combines blade and blade tip joint detection and dynamic tracking, and improves the stability of blade tip positioning and the accuracy of clearance estimation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of wind turbine blade tip detection, and in particular to a wind turbine blade tip detection method and system based on brightness self-adaptation and multi-scale attention. BACKGROUND

[0002] As an important part of clean and renewable energy, the installed capacity and operation time of wind power continue to grow, and the intelligent monitoring and maintenance of wind turbine units have become an important link to ensure safe operation and improve power generation efficiency. In a wind turbine unit, the blade tip is not only a key part reflecting the aerodynamic and structural state of the blade, but also an important object for wind turbine operation safety and blade clearance detection. The so-called blade clearance refers to the minimum vertical distance between the wind turbine blade and the tower surface during operation, which directly relates to whether the unit is in a safe operating state. If the clearance is insufficient, it may lead to reduced wind-swept area, reduced power generation efficiency, or even serious accidents such as tower sweeping and blade rupture, posing a threat to equipment and personnel safety.

[0003] Currently, the methods for wind turbine clearance detection and blade tip position monitoring mainly include three categories: physical sensing measurement, SCADA data-driven modeling, and visual image recognition. Physical measurement methods such as acceleration sensors, interferometric radar or microwave synchronization systems can achieve millimeter-level clearance measurement, but the equipment cost is high, the calibration is complex and the maintenance pressure is great, making it difficult to be widely applied in large-scale wind farms. The data-driven method based on SCADA can utilize operation data for fault warning and health assessment, but it relies on the original signals of the unit and has limited real-time recognition capability for local deformation of the blade and spatial position of the blade tip. In contrast, the visual detection scheme relies on high-resolution image acquisition devices and directly utilizes the shape, texture and spatial relationship information of the blade surface to achieve non-contact detection, which not only has the advantages of low cost and flexible deployment, but also can realize a unified detection framework in clearance measurement, icing detection and surface damage identification. With the application of unmanned aerial vehicle inspection and high-magnification monitoring systems, visual detection technology has gradually become an important direction for intelligent operation and maintenance of wind farms.

[0004] However, the actual operation environment of the wind farm is complex and changeable, often accompanied by low illumination at night, fog, strong reflection and local overexposure and other problems. These factors lead to extremely uneven image brightness distribution, which in turn affects feature extraction and detection accuracy. Especially in the case of small tip target volume, long boundary and easy background interference, traditional image detection algorithms often have problems such as false detection, missed detection or boundary drift. In addition, if an independent night detection module or a brightness threshold switching strategy is used, it not only increases the complexity of model maintenance and deployment, but also easily causes the cross-scene generalization performance to decline. To deal with low light, reflection and other extreme lighting conditions, researchers have proposed various visual robustness strategies. One of the strategies is to enhance the image at the input end, improve the brightness and contrast to improve the visibility of the dark part. For example, PE-YOLO proposed by Wu et al. introduces a brightness enhancement path and an image prior module in the detection chain, which effectively improves the night detection performance; the low-light enhancement method based on Retinex theory such as Retinexformer uses reflection and illumination decomposition to realize real-time enhancement, balancing between dark contrast and edge fidelity. However, such methods are prone to color distortion, artifacts or over-enhancement problems, and cannot adapt to multiple complex lighting environments at the same time.

[0005] Another type of research focuses on the positioning and boundary stability optimization of small-scale targets. Zhang et al. proposed PP-HGNet, which improves small target detection capability through a lightweight pyramid branch; Chen et al. fused CBAM and BiFPN modules in YOLOv8 to realize multi-scale feature fusion and channel attention enhancement, improving the sensitivity to local abnormal regions; Although these methods have achieved certain results in multi-scale modeling and small target detection, for the tip of the blade which is long, low pixel ratio and connected to the main body of the blade, it is still difficult to maintain stable boundary regression and key part positioning at a long distance perspective. In addition, the interference of the tower cylinder, background clouds and strong reflection area also easily leads to non-maximum suppression errors, thereby reducing the reliability of detection. SUMMARY

[0006] The present application provides a wind turbine blade tip detection method and system based on brightness adaptation and multi-scale attention to solve the technical problems mentioned in the background art.

[0007] To achieve the above purpose, the technical scheme of the present application is as follows:

[0008] The present application provides a wind turbine blade tip detection method based on brightness adaptation and multi-scale attention, comprising the following steps:

[0009] S1, first collect multiple wind turbine blade images containing the tip, and after pretreatment, use the pretreated wind turbine blade images to construct a data set;

[0010] S2, replace the first convolutional layer in the backbone network of the original YOLOv8 network with a brightness perception evaluation and branch fusion module, a small target sensitive feature extraction backbone network and a multi-scale context attention module connected in sequence, and after the replacement, an improved YOLOv8 network is obtained;

[0011] S3, training the improved YOLOv8 network using a data set to obtain a trained YOLOv8 network;

[0012] S4, deploying the trained YOLOv8 network to a device end and performing wind turbine blade tip detection to obtain a detection result.

[0013] Further, the brightness perception evaluation and branch fusion module in S2 includes an environment perception module, a dark branch, a bright branch, a fully connected layer and an activation function.

[0014] The output end of the environment perception module is connected to the input end of the dark branch and the bright branch, and the environment perception module is used to generate a continuous illumination weight The environment perception module includes a first branch and a second branch, wherein the first branch includes a deep convolutional layer for extracting deep convolutional features Feat The second branch is used to calculate multi-dimensional brightness features according to multi-dimensional illumination features of the wind turbine blade image Handcrafted .

[0015] The dark branch includes a plurality of standard convolutional layers connected in series, and the bright branch includes an enhanced convolutional layer. The output ends of the dark branch and the bright branch are connected to the fully connected layer and the activation function in sequence.

[0016] Further, the small target sensitive feature extraction backbone network in S2 includes a brightness enhancement unit LiteRetinex and a feature enhancement unit EEB connected in sequence.

[0017] The brightness enhancement unit LiteRetinex adopts a double convolution branch structure and includes a 5x5 convolutional layer and a 15x15 convolutional layer connected in parallel.

[0018] The feature enhancement unit EEB includes a depth separable convolutional layer and a lightweight channel attention ECA connected in sequence. The lightweight channel attention ECA includes a global average pooling layer, a one-dimensional convolutional layer and a Sigmoid activation function connected in sequence. The output of the feature enhancement unit EEB is connected to the brightness enhancement unit LiteRetinex through a residual.

[0019] Further, S3 specifically includes the following steps:

[0020] S31, input the data set into the brightness perception evaluation and branch fusion module in the improved YOLOv8 network, generate a continuous light weight through the environment perception module in the brightness perception evaluation and branch fusion module , input the continuous light weight into the dark branch and the bright branch respectively, fuse the outputs of the dark branch and the bright branch to obtain fused features F* ;

[0021] S32, input the fused features F* into the brightness enhancement unit LiteRetinex of the small target sensitive feature extraction backbone network, perform feature extraction and weighted fusion to obtain enhanced features ; input the enhanced features into the feature enhancement unit EEB to obtain channel weights , then solve the output features according to the enhanced features and the channel weights ;

[0022] S33, input the output features into the multi-scale context attention module to obtain context feature maps with direction perception ;

[0023] S34, input into the detection head Head of the improved YOLOv8 network to obtain a detection result

[0024] S35, construct a total loss function according to the detection result and the real data, and dynamically adjust the parameters of the improved YOLOv8 network through the total loss

[0025] S36, judge whether the iteration stopping condition is reached, if yes, output the trained YOLOv8 network, otherwise, cycle S31 to S36 until the iteration stopping condition is reached

[0026] Further, the S31 specifically includes the following steps:

[0027] S311, input the data set into the brightness perception evaluation and branch fusion module in the improved YOLOv8 network, generate deep convolutional features Feat through the first branch in the environment perception module

[0028] S312, in the second branch in the environment perception module, perform weighted summation on the RGB channels of the wind turbine blade image to obtain ;

[0029] S313, calculating the contrast of the wind turbine blade image according to the height and width of the wind turbine blade image C ;

[0030] S314, calculating the multi-dimensional brightness feature according to the sharpness, the standard deviation of the hue, the average of the saturation, the average of the whole image gray value and the contrast of the wind turbine blade image C Handcrafted ;

[0031] S315, inputting the deep convolution feature Feat and the multi-dimensional brightness feature Handcrafted into the full connection layer to obtain the continuous illumination weight through the activation function ;

[0032] S316, smoothing and regulating the dark branch and the bright branch through the continuous illumination weight and fusing to obtain the fusion feature F* .

[0033] Further, the calculation formula of in the S312 is as follows:

[0034]

[0035] wherein, represents the gray value of a single pixel, R, G, B represents the pixel value of three color channels of the image; , , represent the coefficients;

[0036] The calculation formula of the contrast C in the S313 is as follows:

[0037]

[0038] wherein, H , W respectively represent the height and the width of the wind turbine blade image; i, j respectively represent the current height and the current width of the wind turbine blade image; represents the gray matrix after the gray value of a single pixel is applied to each pixel in the image, is the average of the whole image gray value;

[0039] The calculation formula of the multi-dimensional brightness feature Handcrafted in the S314 is as follows:

[0040]

[0041] wherein, ​hue standard deviation of the wind turbine blade image, used to measure the dispersion degree of the overall color distribution of the image; saturation mean of the wind turbine blade image, used to reflect the vividness of the color of the image; laplacian response result variance, used to quantify the sharpness of the blade image; applying a laplacian operator to the grayscale image;

[0042] continuous illumination weight in S315 The calculation formula is as follows:

[0043]

[0044] wherein, and b are learnable parameters; Sigmoid activation function;

[0045] fusion feature in S316 F* The calculation formula is as follows:

[0046]

[0047] wherein, , respectively represent the output feature maps of the bright branch and the dark branch.

[0048] Further, the S32 specifically comprises the following steps:

[0049] S321, input the fusion feature F* to the brightness enhancement unit LiteRetinex of the small target sensitive feature extraction backbone network, and after the convolution operation of the two branches, respectively obtain the convolution feature and the convolution feature ;

[0050] S322, weight fuse the convolution feature and the convolution feature to obtain the enhanced feature ;

[0051] S323, input the enhanced feature to the depth separable convolution layer of the feature enhancement unit EEB, and extract to obtain the spatial feature ;

[0052] S324, input the spatial feature to the lightweight channel attention ECA, and after respectively passing through the global average pooling layer, the one-dimensional convolution layer and the Sigmoid activation function in the lightweight channel attention ECA, obtain the channel weight ;

[0053] S325, then according to the channel weight , enhanced features and spatial features The output features are calculated.

[0054] Further, the convolutional features and the convolutional features in S321 are calculated as follows:

[0055]

[0056]

[0057] wherein, , represent the convolutional operation of the 5x5 convolutional layer and the 15x15 convolutional layer, respectively;

[0058] The calculation formula of the enhanced features in S322 is as follows:

[0059]

[0060] wherein, , represent the learnable parameters, which are adaptively adjusted by the improved YOLOv8 network during the training process to cope with different lighting environments, and the sum of the learnable parameters , is 1;

[0061] The calculation formula of the spatial features in S323 is as follows:

[0062]

[0063] wherein, represent the convolutional operation of the depth separable convolutional layer;

[0064] The calculation formula of the channel weight in S324 is as follows:

[0065]

[0066] wherein, represent the average pooling operation of the global average pooling layer; represent the convolutional operation of the one-dimensional convolutional layer;

[0067] The calculation formula of the output features in S325 is as follows:

[0068]

[0069] wherein, represents residual connection.

[0070] Further, the S33 specifically comprises the following steps:

[0071] S331, output features are input into a multi-scale context attention module, and multi-scale information of context is aggregated in horizontal and vertical directions and channel enhancement is performed to obtain context features context information aggregated along the horizontal direction and context information aggregated along the vertical direction

[0072]

[0073]

[0074] wherein, represents horizontal aggregation operation; represents vertical aggregation operation;

[0075] S332, context information , is convoluted and mapped and spliced, and normalized and selectively enhanced in the channel dimension to output context feature maps with direction perception , and the calculation formula is as follows:

[0076]

[0077] wherein, represents that the context information in different directions is respectively convoluted and mapped; represents channel splicing; represents a composite mapping of normalization and selective enhancement on the spliced features.

[0078] Another aspect of the present application also provides a wind turbine blade tip detection system, comprising a device end, and the device end is configured or executes the wind turbine blade tip detection method described above.

[0079] The present application has the following beneficial effects:

[0080] The application discloses a wind power blade tip detection method based on brightness self-adaptation and multi-scale attention, and internally discloses an improved YOLOv8 network.

[0081] The brightness perception evaluation and branch fusion module internally comprises an environment perception module and a dark branch and a bright branch, and for complex illumination, fog, rain and night scenes, the application drives the dark branch and the bright branch to be soft fused through the environment perception module, realizes self-adaptive adjustment of the illumination state, combines blade and blade tip joint detection and dynamic tracking, and improves the blade tip positioning stability and the clearance estimation accuracy.

[0082] The small target sensitive feature extraction backbone network internally comprises a brightness enhancement unit LiteRetinex and a feature enhancement unit EEB, the application enhances dark contrast and edge texture through a double convolution branch structure in the brightness enhancement unit LiteRetinex, and strengthens detail features through a lightweight channel attention ECA in the feature enhancement unit EEB, so that the detection robustness and accuracy of the blade tip small target in a complex background are improved.

[0083] In addition, the multi-scale context attention module in the application aggregates multi-scale information of contexts in horizontal and vertical directions, guarantees the coherence and positioning stability of the blade tip edge in the spatial scale, and enhances the adaptability of the improved YOLOv8 network to non-uniform illumination and strong reflection scenes.

[0084] In addition, the application cooperatively designs the brightness perception evaluation and branch fusion module, the small target sensitive feature extraction backbone network and the multi-scale context attention module, so that the improved YOLOv8 network can realize balance between local sensitivity and global consistency, effectively alleviates the detection deviation and boundary drift problems under non-uniform illumination. BRIEF DESCRIPTION OF DRAWINGS

[0085] Fig. 1 The figure is a structural block diagram of the improved YOLOv8 network in the application.

[0086] Fig. 2 The figure is a running block diagram of the brightness perception evaluation and branch fusion module in the application.

[0087] Fig. 3 The figure is a running block diagram of the small target sensitive feature extraction backbone network in the application. DETAILED DESCRIPTION

[0088] For the purposes of promoting an understanding of the principles of the application, reference will now be made to the embodiments illustrated in the drawings. There is shown best mode embodiments of the application. It is expressly understood that the application can be practiced otherwise than as specifically described herein. Accordingly, this application is not intended to be limited in scope by the embodiments described herein. Rather, these embodiments are provided so that this disclosure will satisfy associated patent applying requirements.

[0089] Referring to Fig. 1 , the embodiment of the present application provides a wind turbine blade tip detection method based on brightness adaptation and multi-scale attention, comprising the following steps:

[0090] S1, first, a plurality of wind turbine blade images containing blade tips are collected, and after preprocessing, a dataset is constructed using the preprocessed wind turbine blade images;

[0091] S2, replace the first convolutional layer in the backbone network of the original YOLOv8 network with a brightness perception evaluation and branch fusion module (BAE-FM), a small target sensitive feature extraction backbone network, and a multi-scale context attention module connected in turn. After replacement, an improved YOLOv8 network is obtained;

[0092] The multi-scale context attention module and the aforementioned modules (i.e. brightness perception evaluation and branch fusion module, small target sensitive feature extraction backbone network) are complementary. The fusion features and their illumination weights output by the brightness perception evaluation and branch fusion module reduce the statistical drift caused by lighting. The small target sensitive feature extraction backbone network improves the sensitivity of fine-grained texture and edges related to the blade tip. The multi-scale context attention module imposes a global consistency constraint on this basis. The global consistency constraint refers to keeping the same channel stable in response to the blade tip structure within the entire image range, thereby reducing the boundary drift caused by local interference such as reflection and shadow;

[0093] S3, training the improved YOLOv8 network using the dataset to obtain a trained YOLOv8 network;

[0094] S4, deploying the trained YOLOv8 network to the device end and performing wind turbine blade tip detection to obtain a detection result.

[0095] In some embodiments, referring to Fig. 2 , the brightness perception evaluation and branch fusion module in S2 includes an environment perception module (Illumination Evaluator network), a dark branch, a bright branch, a fully connected layer, and an activation function;

[0096] The output ends of the environment perception module are connected with the input ends of the dark branch and the bright branch respectively, and the environment perception module is configured to generate a continuous light weight to reflect the light intensity and uniformity of the current scene, the environment perception module comprises a first branch and a second branch, wherein the first branch comprises a deep convolutional layer configured to extract deep convolutional features Feat , and the second branch is configured to calculate a multi-dimensional brightness feature according to multi-dimensional light features (including a gray value, a sharpness, a hue standard deviation, a saturation mean value, a full-image gray mean value, and a contrast, etc.) of the wind turbine blade image Handcrafted ;

[0097] The dark branch comprises a plurality of standard convolutional layers connected in series, and the bright branch comprises an enhanced convolutional layer, and the output ends of the dark branch and the bright branch are connected with a full connection layer and an activation function in sequence. The bright branch is inclined to capture the overall contour and the main features of the strong light area, and has stronger overall modeling capability in sufficient light; the dark branch focuses on the detail enhancement and edge texture recovery of the dark area. At the same time, the dark branch enhances the dark edge and local texture with a smaller receptive field and higher channel sensitivity by introducing an enhanced convolutional layer and a lightweight processing unit, and combines residual and normalization to suppress noise amplification. The dark branch and the bright branch respectively undertake global consistency modeling and local detail compensation in semantics, and provide multi-granularity features for subsequent soft fusion of the continuous light weight gate.

[0098] In addition, the present application controls the smooth fusion of the continuous light weight , and the brightness perception evaluation and branch fusion module can adaptively adjust the feature response in the strong reflection, overexposure or shadow area, thereby reducing the detection drift problem caused by brightness mutation. The model instability and cross-scene degradation phenomenon caused by the traditional hard switching method based on brightness threshold are avoided.

[0099] In some embodiments, referring to Fig. 3 , the small target sensitive feature extraction trunk (STSFET) in S2 comprises a brightness enhancement unit LiteRetinex and a feature enhancement unit EEB connected in sequence; the small target sensitive feature extraction trunk is configured to improve the distinguishability of the slender target such as the wind turbine blade tip under complex light and high dynamic background; in addition, the small target sensitive feature extraction trunk is introduced to solve the following problems: the wind turbine blade tip has the characteristics of small size, slender edge and weak texture, and is prone to feature dilution and boundary blur under non-uniform lighting caused by complex climate environment, thereby affecting the distinguishability and positioning accuracy.

[0100] The brightness enhancement unit LiteRetinex adopts a double convolution branch structure, including a 5*5 convolution layer and a 15*15 convolution layer connected in parallel; the 5*5 convolution kernel has a limited receptive field, can smooth the brightness distribution in a local range, suppress noise, and retain fine-grained edge and texture features; the 15*15 convolution kernel has a larger receptive field and pays more attention to overall light distribution and contour shape, thereby enhancing structural visibility on a global level. The brightness enhancement unit LiteRetinex in the application models local and global light changes at the same time, realizes adaptive fusion of light and dark areas through learnable parameters, and improves dark detail and contour clarity;

[0101] The feature enhancement unit EEB includes a depth separable convolution layer and a lightweight channel attention ECA connected in sequence; the lightweight channel attention ECA includes a global average pooling layer, a one-dimensional convolution layer and a Sigmoid activation function connected in sequence; and the output of the feature enhancement unit EEB is connected with the brightness enhancement unit LiteRetinex through a residual. The feature enhancement unit EEB strengthens the high-frequency texture and edge information related to the tip of the leaf by adaptively weighting the feature channels, and keeps the gradient stable transmission through the residual connection. The feature enhancement unit EEB enhances the feature representation capability of the low-contrast and weak-texture area under the premise of ensuring the lightweight of the network, thereby effectively improving the detection rate and positioning accuracy of the tip of the leaf target in the long-distance monitoring image.

[0102] In some embodiments, the S3 specifically includes the following steps:

[0103] S31, input the data set into the brightness perception evaluation and branch fusion module in the improved YOLOv8 network, and generate a continuous light weight through the environment perception module in the brightness perception evaluation and branch fusion module The continuous light weight is respectively input into the dark branch and the bright branch, the outputs of the dark branch and the bright branch are weighted and fused, soft switching and dynamic feature balance under the light condition are realized, and fused features are obtained ; F*

[0104] S32, input the fused features F* into the double convolution branch structure of the brightness enhancement unit LiteRetinex of the small target sensitive feature extraction backbone network, perform feature extraction and weighted fusion, and obtain enhanced features ; input the enhanced features into the feature enhancement unit EEB, obtain channel weights , then solve the output features according to the enhanced features and the channel weights ; ​

[0105] S33, output features are input into an efficient multi-scale context attention (EMA) module to obtain context feature maps with direction perception The EMA module can adaptively capture global dependencies between different scale features, suppress local false responses caused by changes in illumination or background interference, and strengthen geometric continuity and structural discrimination ability in small target regions. The EMA module captures long-range dependencies across space with a direction selection mechanism while maintaining discrimination of local structures, thereby effectively alleviating boundary drift and positioning errors on small targets.

[0106] S34, the input into the detection head Head of the improved YOLOv8 network to obtain the detection result;

[0107] S35, according to the detection result and the real data to build the total loss function, and through the total loss to dynamically adjust the parameters of the improved YOLOv8 network;

[0108] S36, judge whether to reach the iteration stop condition, if yes, output the trained YOLOv8 network, otherwise loop S31 to S36 until the iteration stop condition is reached.

[0109] In some embodiments, the S31 specifically includes the following steps:

[0110] S311, input the data set into the brightness perception evaluation and branch fusion module in the improved YOLOv8 network, generate deep convolutional features Feat through the first branch in the environment perception module;

[0111] S312, in order to obtain a gray scale image consistent with human visual perception, the RGB channels of the wind turbine blade image are weighted and summed in the second branch of the environment perception module to obtain ;

[0112] S313, calculate the contrast of the wind turbine blade image according to the height and width of the wind turbine blade image C ;

[0113] S314, calculate the multi-dimensional brightness feature C according to the sharpness, hue standard deviation, saturation mean, full image gray mean and contrast of the wind turbine blade image Handcrafted ;

[0114] S315, input the deep convolutional features Feat and the multi-dimensional brightness features Handcrafted ​The input is fed into a fully connected layer and passes through an activation function to obtain continuous illumination weights. ;

[0115] S316, Weighting continuous illumination The dark branch and the bright branch are smoothed and merged to obtain the fused feature. F* .

[0116] For complex lighting, fog, rain and nighttime scenarios, this invention achieves adaptive adjustment of lighting conditions through soft fusion of dark and bright branches driven by an environmental perception module. Combined with joint detection and dynamic tracking of blades and blade tips, it improves the stability of blade tip positioning and the accuracy of clearance estimation.

[0117] In some embodiments, in S312 The specific calculation formula is as follows:

[0118]

[0119] in, Represents the grayscale value of a single pixel. R, G, B Represents the pixel values ​​of the three color channels of an image; , , Represents the coefficients of each item;

[0120] Contrast in S313 C The calculation formula is as follows:

[0121]

[0122] in, H, W These represent the height and width of the wind turbine blade image, respectively. i, j These represent the current height and current width of the wind turbine blade image, respectively. This represents the grayscale value of a single pixel. The grayscale matrix applied to each pixel of the image. The average gray level of the entire image;

[0123] Multidimensional brightness features in S314 Handcrafted The calculation formula is as follows:

[0124]

[0125] in, This represents the standard deviation of the hue in a wind turbine blade image, used to measure the dispersion of the overall color distribution in the image. This represents the average saturation value of a wind turbine blade image, used to reflect the vibrancy of the image colors. This indicates that the variance of the Laplace response is used to quantify the sharpness of the blade image; represents applying Laplacian operator to the gray image;

[0126] Continuous illumination weight in S315 The calculation formula is as follows:

[0127]

[0128] wherein, and b are learnable parameters; represents Sigmoid activation function;

[0129] Fusion feature in S316 F* The calculation formula is as follows:

[0130]

[0131] wherein, , respectively represent output feature maps of bright branch and dark branch.

[0132] In some embodiments, the S32 specifically comprises the following steps:

[0133] S321, input the fusion feature F* to the brightness enhancement unit LiteRetinex of the small target sensitive feature extraction backbone network, and after the convolution operation of two branches, respectively obtain the convolution feature and the convolution feature ;

[0134] S322, weight fusion is performed on the convolution feature and the convolution feature , to obtain the enhanced feature ; through the complementary mechanism of local and global (i.e. weight fusion), the brightness enhancement unit LiteRetinex can improve the dark part contrast and global contour clarity at the same time, and provide more robust representation for subsequent small target detection;

[0135] S323, input the enhanced feature to the depth separable convolution layer of the feature enhancement unit EEB, and extract to obtain the spatial feature ;

[0136] S324, input the spatial feature to the lightweight channel attention ECA, and after respectively passing through the global average pooling layer, the one-dimensional convolution layer and the Sigmoid activation function in the lightweight channel attention ECA, obtain the channel weight ; the channel weight Adaptively depict the importance of different channels, thereby highlighting the fine-grained texture and edge information related to the tip of the leaf;

[0137] S325, then according to the channel weight , enhanced features and spatial features The output features are calculated .

[0138] The present application enhances dark contrast and edge texture through the double convolution branch structure in the brightness enhancement unit LiteRetinex, and enhances detailed features through the lightweight channel attention ECA in the feature enhancement unit EEB, thereby improving the detection robustness and accuracy of the tip of the small target in the complex background.

[0139] In some embodiments, the calculation formula of the convolution features and the convolution features in S321 are as follows, respectively:

[0140]

[0141]

[0142] wherein, , represent the convolution operation of the 5x5 convolution layer and the 15x15 convolution layer, respectively;

[0143] The calculation formula of the enhanced features in S322 is as follows:

[0144]

[0145] wherein, , represent the learnable parameters, which are adaptively adjusted by the improved YOLOv8 network during the training process to cope with different lighting environments, and the sum of the learnable parameters , is 1;

[0146] The calculation formula of the spatial features in S323 is as follows:

[0147]

[0148] wherein, represents the convolution operation of the depth separable convolution layer;

[0149] The calculation formula of the channel weight in S324 is as follows:

[0150]

[0151] wherein, denotes the average pooling operation of the global average pooling layer; denotes the convolution operation of the one-dimensional convolution layer;

[0152] the output feature in the S325 is calculated as follows:

[0153]

[0154] wherein, denotes the residual connection. The residual connection ensures stable transmission of features and avoids information loss caused by excessive weighting. Through this design, the small target sensitive feature extraction backbone first completes the tasks of improving visibility and refining discrimination, and the output feature is then aligned to the multi-scale feature layers P3, P4, and P5 of the detection head, and is used as the input of the subsequent multi-scale context attention module to complete cross-scale aggregation.

[0155] In some embodiments, the S33 specifically comprises the following steps:

[0156] S331, inputting the output feature into the multi-scale context attention module to aggregate the context in the horizontal and vertical directions and perform channel enhancement to obtain context information along the horizontal direction of the context feature and context information along the vertical direction of the context feature, and the calculation formulas are as follows:

[0157]

[0158]

[0159] wherein, denotes the horizontal direction aggregation operation; denotes the vertical direction aggregation operation;

[0160] S332, convoluting and mapping the context information , and splicing, and performing normalization and weighted selective enhancement in the channel dimension to output a context feature map with direction perception , and the calculation formula is as follows:

[0161]

[0162] wherein, denotes the convolution mapping of the context information in different directions respectively; denotes channel splicing; represents a composite mapping of normalization and selective enhancement on the stitching feature.

[0163] The multiscale context attention module in the application ensures the continuity and positioning stability of the tip edge in the spatial scale by aggregating the multiscale information of the context in the horizontal and vertical directions, and enhances the adaptability of the improved YOLOv8 network to non-uniform illumination and strong reflection light scenes.

[0164] The performance of the trained YOLOv8 network in the application is verified by the following experiments;

[0165] Experiment 1: Comparison experiment of the trained YOLOv8 network and other different models in various performances;

[0166] After the comparison experiment, the following data are obtained:

[0167] Table 1: Comparison data table of different models in various performance parameters;

[0168]

[0169] It can be seen that Faster R-CNN and SSD are limited in mAP@0.5, and have large parameter quantity and low FPS; the YOLO series is superior in speed due to lightweight design, but still has room for improvement in tip small target detection under complex lighting and extreme weather conditions. If the targeted improvement methods such as DCW-YOLO, YOLOv8-CM and MES-YOLOv8n are introduced, the precision can be enhanced to a certain extent. Compared with the above methods, the method of the present application has an accuracy of 94.1%, a recall rate of 94.7% and an average precision of 95.3%, while the parameter quantity is only 2.6M and the frame rate reaches 157 frames per second, achieving more balanced performance in efficiency and precision, and being suitable for real-time application in complex wind power scenes.

[0170] Experiment 2: Ablation experiment;

[0171] The ablation experiment of the application is carried out on the real wind power extreme weather dataset, and the evaluation indexes are accuracy, Recall and mAP@0.5. Except for the module configuration, the remaining training and evaluation settings remain the same. To verify the effectiveness and contribution of each component module, the following baseline and variants are constructed:

[0172] A1 (Baseline): YOLOv8n baseline model.

[0173] A2 + BAE-FM(Bright): Add a bright field adaptive branch (Bright) to the baseline.

[0174] A3 + BAE-FM(Dark): Add a dark field adaptive branch (Dark) to the baseline.

[0175] A4 + BAE-FM(All): Enable unified brightness domain adaptation (All).

[0176] A5 + EEB: Add efficient enhance block (EEB).

[0177] A6 + EMA: Add multi-scale context attention module (EMA).

[0178] A7 Improved YOLOv8 network: BAE-FM(All) + EEB + EMA three-module synergy.

[0179] The results of the ablation evaluation on the real wind farm extreme weather dataset are as follows. The accuracy of the baseline A1 is 83.6%, the recall rate is 87.1%, and the average precision is 61.4%, and there is still a large room for improvement in low-illumination and weak-texture scenarios. A2, that is, after adding brightness domain adaptation, the average precision is improved to 79.1%, but the accuracy and recall rate are decreased to 82.8% and 83.4% respectively, indicating that only adapting to the brightness domain has limited help for overall stability. A3 focuses on the dark domain, and the three indicators are improved to 83.1%, 83.76% and 81.1%, which is more suitable for low-illumination scenarios than brightness adaptation. A4 models the light and dark conditions uniformly, and the accuracy is improved to 85.2%, and the average precision is improved to 83.4%, and the robustness under different illumination conditions is better. A5 does not introduce brightness adaptation, and only uses detail enhancement, and the accuracy and recall rate are improved to 87.2%, and the average precision reaches 86.3%, indicating that strengthening the edge and weak texture can significantly improve the separability. When using attention only in A6, the accuracy reaches 86.3%, the recall rate is 83.1%, and the average precision is 66.7%, and the advantage of context aggregation is difficult to fully play when lacking the support of brightness adaptation and detail enhancement. A7 uses unified brightness domain adaptation, EEB and EMA in cooperation, and achieves the best results: the accuracy is 94.1%, the recall rate is 94.7%, and the average precision is 95.3%. This shows that there is an obvious complementary relationship between the three: brightness adaptation provides robust representation across illumination conditions, detail enhancement highlights edges and weak textures, and attention mechanism further integrates multi-scale context information, thereby achieving high accuracy and high real-time performance under complex weather conditions.

[0180] Table 2, ablation experiment data;

[0181]

[0182] Another aspect of the present application also provides a wind turbine blade tip detection system, comprising a device end configured or executing the wind turbine blade tip detection method described above.

[0183] The above merely provides a specific implementation of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered within the protection scope of the present application. Furthermore, the technical solutions of each embodiment of the present application can be combined with each other, but it must be based on the realization of the ordinary skilled in the art, when the combination of the technical solutions appears contradictory or unachievable, it should be considered that the combination of the technical solutions does not exist, nor within the protection scope required by the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A wind turbine blade tip detection method based on brightness self-adaption and multi-scale attention, characterized in that, The method comprises the following steps: S1, first, a plurality of wind turbine blade images containing a blade tip are collected, and after preprocessing, a dataset is constructed using the preprocessed wind turbine blade images; S2, the first convolutional layer in the backbone network of the original YOLOv8 network is replaced by a brightness perception evaluation and branch fusion module, a small target sensitive feature extraction backbone network, and a multi-scale context attention module connected in turn, and after the replacement, an improved YOLOv8 network is obtained; S3, the improved YOLOv8 network is trained using the dataset to obtain a trained YOLOv8 network; S4, the trained YOLOv8 network is deployed to the device end, and wind turbine blade tip detection is performed to obtain a detection result; The brightness perception evaluation and branch fusion module in S2 comprises an environment perception module, a dark branch, a bright branch, a fully connected layer, and an activation function; The output end of the environment perception module is connected with the input end of the dark branch and the bright branch respectively, and the environment perception module is used for generating a continuous light weight The environment perception module comprises a first branch and a second branch, wherein the first branch comprises a deep convolutional layer and is used for extracting deep convolutional features Feat The second branch is used for calculating a multi-dimensional brightness feature according to the multi-dimensional light feature of the wind power blade image Handcrafted ​ The dark branch comprises a plurality of standard convolutional layers connected in turn; the bright branch comprises an enhanced convolutional layer, and the output ends of the dark branch and the bright branch are connected with the fully connected layer and the activation function in turn; The small target sensitive feature extraction backbone network in S2 comprises a brightness enhancement unit LiteRetinex and a feature enhancement unit EEB connected in turn; The brightness enhancement unit LiteRetinex adopts a double convolution branch structure and comprises a 5×5 convolutional layer and a 15×15 convolutional layer connected in parallel; The feature enhancement unit EEB comprises a depth separable convolutional layer and a lightweight channel attention ECA connected in turn; the lightweight channel attention ECA comprises a global average pooling layer, a one-dimensional convolutional layer, and a Sigmoid activation function connected in turn; and the output of the feature enhancement unit EEB is connected with the brightness enhancement unit LiteRetinex through a residual; S3 specifically comprises the following steps: S31, input the data set into the brightness perception evaluation and branch fusion module in the improved YOLOv8 network, generate a continuous light weight through the environment perception module in the brightness perception evaluation and branch fusion module , input the continuous light weight into the dark branch and the bright branch respectively, fuse the outputs of the dark branch and the bright branch to obtain fused features F* ; S32, fusing features F* into the double convolution branch structure of the brightness enhancement unit LiteRetinex input to the small target sensitive feature extraction backbone network, performing feature extraction and weighted fusion to obtain enhanced features ; input the enhanced features into the feature enhancement unit EEB to obtain channel weights , and then according to the enhanced features and the channel weights solving to obtain output features ; S33, output the feature into the multi-scale context attention module to obtain a context feature map with direction perception ; S34, will The results are input into the detection header of the improved YOLOv8 network to obtain the detection results; S35, a total loss function is constructed according to the detection result and the true data, and the parameters of the improved YOLOv8 network are dynamically adjusted through the total loss; S36, it is judged whether the iteration stopping condition is reached, if yes, the trained YOLOv8 network is output, otherwise, steps S31 to S36 are cycled until the iteration stopping condition is reached; S31 specifically comprises the following steps: S311, input the data set into the brightness perception evaluation and branch fusion module in the improved YOLOv8 network, generate deep convolutional features through the first branch in the environment perception module Feat ; S312, in the second branch in the environment perception module, weighted sum is performed on the RGB channels of the wind turbine blade image to obtain ; S313、calculating the contrast of the wind turbine blade image according to the height and the width of the wind turbine blade image C ; S314、According to the sharpness, hue standard deviation, saturation mean value, full image gray mean value and contrast of the wind power blade image C The multi-dimensional brightness features are calculated Handcrafted ; S315、the deep convolutional features Feat and multi-dimensional intensity features Handcrafted are input into a fully connected layer, and a continuous illumination weight is obtained through an activation function ; S316、the continuous light exposure weight The dark branch and the bright branch are smoothly regulated and fused to obtain a fused feature F* ; The multi-dimensional luminance feature in S314 Handcrafted The calculation formula is as follows: wherein, represents the hue standard deviation of the wind turbine blade image; represents the mean saturation of the wind turbine blade image; represents taking the variance of the Laplacian response result; represents applying a Laplacian operator to the grayscale image; is the full image grayscale mean; represents the grayscale value of a single pixel; S315 the continuous illumination weight The calculation formula is as follows: wherein, and b are learnable parameters; denotes a Sigmoid activation function; Fusion features in S316 F* The calculation formula is as follows: wherein, , respectively represent the output feature maps of the bright branch and the dark branch.

2. The wind turbine blade tip detection method based on brightness self-adaption and multi-scale attention according to claim 1, characterized in that, In the S312 The calculation formula is as follows: wherein, R, G, B representing three color channel pixel values of an image; , , representing respective coefficients; The S313 contrast C The calculation formula of the S313 is as follows: wherein, H , W respectively represent the height and width of the wind turbine blade image; i, j respectively represent the current height and current width of the wind turbine blade image; represents the gray scale matrix applied to each pixel in the image after the gray scale value of a single pixel is converted to a value between 0 and 1. represents the gray scale matrix applied to each pixel in the image after the gray scale value of a single pixel is converted to a value between 0 and 1.

3. The wind turbine blade tip detection method based on brightness self-adaption and multi-scale attention according to claim 2, characterized in that, S32 specifically comprises the following steps: S321、fuse the features F* to the small target sensitive feature extraction backbone network, after the convolution operation of the two branches, respectively, get the convolution feature and the convolution feature ; S322、combine the convolution features and the convolution features to obtain an enhanced feature ; S323、enhancing the feature The spatial feature is extracted by inputting the depth separable convolution layer of the feature enhancement unit EEB ; S324、the spatial features are input into the lightweight channel attention ECA, and after passing through the global average pooling layer, the one-dimensional convolution layer and the Sigmoid activation function in the lightweight channel attention ECA respectively, the channel weights are obtained S325、Then, the channel weight is determined according to the channel weight , enhanced features , and spatial features to calculate output features .

4. The wind turbine blade tip detection method based on brightness self-adaption and multi-scale attention according to claim 3, characterized in that, The convolution feature in the S321 And the calculation formula of the convolution feature is as follows respectively: wherein, , denote a convolution operation of a 5x5 convolution layer and a 15x15 convolution layer, respectively; The enhanced feature in the S322 The calculation formula is as follows: wherein, , denote learnable parameters, which are self-adaptively adjusted by the improved YOLOv8 network during the training process to cope with different lighting environments, and the learnable parameters , the sum of the two is 1. The spatial feature in S323 The calculation formula is as follows: wherein, denotes a convolution operation of a depthwise separable convolution layer; The channel weight in the S324 The calculation formula is as follows: wherein, denotes an average pooling operation of a global average pooling layer; denotes a convolution operation of a one-dimensional convolution layer; The output feature in the S325 The calculation formula is as follows: wherein represents a residual connection.

5. The wind turbine blade tip detection method based on brightness self-adaption and multi-scale attention according to claim 4, characterized in that, S33 specifically comprises the following steps: S331、the output feature input into the multi-scale context attention module, the multi-scale information of the context is aggregated in horizontal and vertical directions and the channel enhancement is carried out, to obtain context information aggregated along the horizontal direction of the context feature and context information aggregated along the vertical direction of the context feature The calculation formulas are as follows, respectively: wherein, represents a horizontal direction aggregation operation; represents a vertical direction aggregation operation; S332、the context information is obtained , convolutional mapping and splicing, and normalization and weighted selection in the channel dimension are performed to selectively enhance, outputting a context feature map with direction perception , the calculation formula is as follows: wherein, represents respectively convoluting and mapping the context information in different directions; represents channel concatenation; represents a composite mapping of normalizing and selectively enhancing the concatenated features.

6. A wind turbine blade tip detection system, characterized in that The device end is configured or executes the wind turbine blade tip detection method of any one of claims 1 to 5.

Images