Wind power plant inspection method and system based on unmanned aerial vehicle

Abstract

The invention discloses a wind power plant inspection method and system based on an unmanned aerial vehicle, and relates to the technical field of wind turbine generator detection. Acquiring an image set of a target fan blade from multiple angles, and classifying images to obtain a first image set and a second image set; determining an adaptive weight according to the first image set, and updating a rule of a preset algorithm according to the adaptive weight to obtain a target algorithm; performing defect detection on the first image set according to a target algorithm to obtain defect data, and calculating according to the second image set to obtain an amplitude parameter of the target fan blade; wear judgment is conducted on the target fan blade according to the defect data and the amplitude parameters; through an adaptive weight updating algorithm rule, accurate detection and calculation of fan blade defect data and amplitude parameters are realized, the wear state of the fan blade is comprehensively evaluated, the contradiction between the detection precision and efficiency is effectively balanced, a reliable basis is provided for refined maintenance of a wind turbine generator, the reliability and safety of equipment operation are remarkably improved, and the method is suitable for popularization and application. And meanwhile, the maintenance cost is reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of wind turbine detection, and particularly relates to a method and system for wind farm inspection based on an unmanned aerial vehicle (UAV). Background Art

[0002] With the rapid development of wind energy and solar power generation, the UAV inspection technology for onshore wind / solar power stations has gradually attracted attention. In recent years, the research focus has been on how to balance the contradiction between detection accuracy and efficiency. On the one hand, the intelligent inspection technology based on UAVs has achieved efficient detection of defects in wind turbine blades and photovoltaic panels by carrying multi-spectral cameras and improved object detection algorithms. On the other hand, researchers have improved the detection accuracy while maintaining high detection efficiency by optimizing detection algorithms, such as the improved YOLOv5 algorithm. In addition, the infrared thermal imaging technology carried by UAVs has also been applied to detect internal defects of blades, further enriching the detection means. These research progresses provide new ideas and technical supports for realizing high-precision and high-efficiency detection of wind power and photovoltaic equipment.

[0003] Patent No.: CN118521544A discloses a panoramic defect intelligent recognition system and method for offshore wind turbine blades. The UAV is used to carry a variety of sensors to perform panoramic scanning on the blades to collect heterogeneous data such as vision, thermal infrared, and laser. Deep learning algorithms are used on the edge side to fuse and identify various defects in the heterogeneous data. The recognition results are three-dimensionally visualized and combined with a virtual maintenance training environment to provide intelligent maintenance decision support. The entire system realizes the full-process intelligence of detection, analysis, training, and decision-making, representing the forefront level in the field of wind power detection and operation and maintenance, which can greatly improve the operation and maintenance quality and efficiency and promote the green and sustainable development of the wind power industry.

[0004] Although some problems are solved in the above technologies, there are still some problems, such as: the diversity of traditional visual acquisition data is relatively low, and the dynamic vibration analysis data is fragmented, making it difficult to comprehensively evaluate the health status of the fan blades; the fixed algorithm rules have insufficient robustness under complex working conditions, and it is difficult to balance detection accuracy and efficiency at the same time. High-precision detection requires a large amount of computing resources, while simplified algorithms are prone to ignoring early subtle cracks or hidden structural damages, ultimately leading to delayed maintenance decisions, increased equipment operation risks, and increased costs of unplanned shutdowns. Summary of the Invention

[0005] The object of the present invention is to solve the problems that the diversity of traditional visual acquisition data is relatively low, the dynamic vibration analysis data is fragmented and it is difficult to comprehensively evaluate the health status of the fan blade, the fixed algorithm rules have insufficient robustness under complex working conditions, and it is difficult to balance the detection accuracy and efficiency. High-precision detection requires a large amount of computing resources, while simplified algorithms are prone to ignoring early subtle cracks or hidden structural damages, ultimately leading to delayed maintenance decisions, increased equipment operation risks and increased unplanned shutdown costs. Therefore, a method and system for wind farm inspection based on an unmanned aerial vehicle are proposed.

[0006] In the first aspect of the implementation of the present invention, a method for wind farm inspection based on an unmanned aerial vehicle is first proposed. The method includes:

[0007] Obtain an image set of the target fan blade from multiple angles, and classify the images to obtain a first image set and a second image set; the multiple angles include: the front, the back and the side;

[0008] Determine an adaptive weight according to the first image set, and update the rules of a preset algorithm according to the adaptive weight to obtain a target algorithm;

[0009] Perform defect detection on the first image set according to the target algorithm to obtain defect data, and calculate the amplitude parameter of the target fan blade according to the second image set;

[0010] Judge the wear of the target fan blade according to the defect data and the amplitude parameter.

[0011] Optionally, before classifying the images to obtain a first image set and a second image set, it includes:

[0012] Extract features of the images in the image set through a target YOLOv8 model; delete the C2f module of the eighth layer in the YOLOv8 model, and replace all Conv modules in the YOLOv8 model with a target Conv model to obtain a target YOLOv8 model;

[0013] The processing process of the target Conv model for images:

[0014] Obtain the original image, input the original image into the Conv module to obtain a global feature map, input the original image into the DSConv-x module to obtain a first tubular feature map, and input the original image into the DSConv-y module to obtain a second tubular feature map;

[0015] Stitch the global feature map, the first tubular feature map and the second tubular feature map to obtain a first feature map, perform a max pooling operation on the first feature map to obtain a second feature map, and use the second feature map as the output of the target Conv model.

[0016] Optionally, determining the adaptive weight according to the first image set includes:

[0017] Preprocessing the images in the first image set to obtain an initial image set, and segmenting the initial images to obtain a regional image set; the initial image is any one in the initial image set, and the target region is any one in the regional image set;

[0018] Obtain the average gray value of the target region, calculate the gray difference according to the average gray value and the maximum gray value, and if the gray difference > gray threshold, determine that the target region is a damaged region;

[0019] Calculate the dynamic threshold according to the average gray value and the gray span, obtain the number of pixels of the fan blade in the target region, and calculate the number of seeds according to the average gray value, the gray span and the number of pixels;

[0020] Use the dynamic threshold and the number of seeds as the adaptive weight of the preset algorithm; the preset algorithm is the region growing algorithm.

[0021] Optionally, calculating the amplitude parameter of the target fan blade according to the second image set includes:

[0022] Extract frames from the second image set according to a preset period to obtain a sequence image set, and perform feature recognition on the sequence image set to obtain a feature image set;

[0023] Calculate the node displacement set by performing displacement calculation on the feature image set through the optical flow algorithm, fit the amplitude curve according to the node displacement set, and calculate the amplitude parameter according to the slope of the amplitude curve.

[0024] Optionally, judging the wear of the target fan blade according to the defect data and the amplitude parameter includes:

[0025] If the defect data ≤ area threshold and the amplitude parameter ≤ amplitude threshold, determine that the target fan blade is in normal condition;

[0026] If the defect data > area threshold and the amplitude parameter ≤ amplitude threshold, determine that the target fan blade has external wear;

[0027] If the defect data ≤ area threshold and the amplitude parameter > amplitude threshold, determine that the target fan blade has internal wear;

[0028] If the defect data > area threshold and the amplitude parameter > amplitude threshold, determine that the target fan blade has comprehensive wear.

[0029] In the second aspect of the implementation of the present invention, a wind farm inspection system based on an unmanned aerial vehicle is proposed, including: a data acquisition module, a weight determination module, a defect detection module, and a wear judgment module:

[0030] The data acquisition module is used to obtain an image set of a target fan blade from multiple angles, and classify the images to obtain a first image set and a second image set; the multiple angles include: the front, the back, and the side;

[0031] The weight determination module is used to determine an adaptive weight according to the first image set, and update the rules of a preset algorithm according to the adaptive weight to obtain a target algorithm;

[0032] The defect detection module is used to perform defect detection on the first image set according to the target algorithm to obtain defect data, and calculate the amplitude parameter of the target fan blade according to the second image set;

[0033] The wear judgment module is used to judge the wear of the target fan blade according to the defect data and the amplitude parameter.

[0034] Optionally, the improvement of the target YOLOv8 model includes:

[0035] Delete the C2f module in the eighth layer of the YOLOv8 model, and replace all Conv modules in the YOLOv8 model with a target Conv model to obtain a target YOLOv8 model;

[0036] The improvement of the target Conv model includes:

[0037] Input the original image into the Conv module to obtain a global feature map, input the original image into the DSConv-x module to obtain a first tubular feature map, and input the original image into the DSConv-y module to obtain a second tubular feature map;

[0038] Stitch the global feature map, the first tubular feature map, and the second tubular feature map to obtain a first feature map, perform a max pooling operation on the first feature map to obtain a second feature map, and use the second feature map as the output of the target Conv model.

[0039] Optionally, the weight determination module includes: an image segmentation module, a damage judgment module, a parameter determination module, and an algorithm update module:

[0040] The image segmentation module is used to preprocess the images in the first image set to obtain an initial image set, and segment the initial images to obtain a regional image set; the initial image is any one in the initial image set, and the target region is any one in the regional image set;

[0041] The damage judgment module is used to obtain the average gray value of the target area, calculate the gray difference value according to the average gray value and the maximum gray value, and if the gray difference value > the gray threshold value, determine that the target area is a damaged area;

[0042] The parameter determination module is used to calculate a dynamic threshold according to the average gray value and the gray span, obtain the number of pixels of the fan blade in the target area, and calculate the number of seeds according to the average gray value, the gray span and the number of pixels;

[0043] The algorithm update module is used to use the dynamic threshold and the number of seeds as the adaptive weights of the preset algorithm; the preset algorithm is a region growing algorithm.

[0044] Optionally, the defect detection module further includes: an image frame extraction module and an amplitude calculation module:

[0045] The image frame extraction module is used to extract frames from the second image set at a preset period to obtain a sequence image set, and perform feature recognition on the sequence image set to obtain a feature image set;

[0046] The amplitude calculation module is used to calculate the node displacement set by performing displacement calculation on the feature image set through an optical flow algorithm, fit an amplitude curve according to the node displacement set, and calculate the amplitude parameter according to the slope of the amplitude curve.

[0047] Optionally, the wear judgment module includes:

[0048] The first judgment module is used to determine that the target fan blade is in a normal condition if the defect data ≤ the area threshold value and the amplitude parameter ≤ the amplitude threshold value;

[0049] The second judgment module is used to determine that the target fan blade has external wear if the defect data > the area threshold value and the amplitude parameter ≤ the amplitude threshold value;

[0050] The third judgment module is used to determine that the target fan blade has internal wear if the defect data ≤ the area threshold value and the amplitude parameter > the amplitude threshold value;

[0051] The fourth judgment module is used to determine that the target fan blade has comprehensive wear if the defect data > the area threshold value and the amplitude parameter > the amplitude threshold value.

[0052] The beneficial effects of the present invention:

[0053] The present invention provides a method. By acquiring an image set of a target fan blade from multiple angles, classifying the images to obtain a first image set and a second image set; determining an adaptive weight according to the first image set, and updating the rules of a preset algorithm according to the adaptive weight to obtain a target algorithm; performing defect detection on the first image set according to the target algorithm to obtain defect data, and calculating the amplitude parameter of the target fan blade according to the second image set; judging the wear of the target fan blade according to the defect data and the amplitude parameter; by updating the algorithm rules with the adaptive weight, accurate detection and calculation of the fan blade defect data and the amplitude parameter are realized, the wear state of the fan blade is comprehensively evaluated, the contradiction between detection accuracy and efficiency is effectively balanced, a reliable basis is provided for the refined maintenance of the wind turbine generator set, the reliability and safety of the equipment operation are significantly improved, and the maintenance cost is reduced at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The present invention will be further described below with reference to the accompanying drawings.

[0055] Figure 1 FIG. is a flowchart of a method for inspecting a wind farm based on an unmanned aerial vehicle provided by an embodiment of the present invention;

[0056] Figure 2 FIG. is a framework diagram of a system for inspecting a wind farm based on an unmanned aerial vehicle provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0057] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. The term "and / or" in this article is only a description of an association relationship, indicating that there may be three relationships. For example, A and B may represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the descriptions such as "first" and "second" in the present invention are only for the purpose of description, and cannot be understood as indicating or implying their relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments may be combined with each other, but it must be based on the fact that those skilled in the art can implement them. When the combination of technical solutions appears to be mutually contradictory or unable to be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the protection scope required by the present invention.

[0058] All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0059] An embodiment of the present invention provides a method. SeeFigure 1 , Figure 1 is a flowchart of a method provided by an embodiment of the present invention. The method includes the following steps:

[0060] An embodiment of the present invention provides a method for inspecting a wind farm based on a drone. Refer to Figure 1 , Figure 1 is a flowchart of a method for inspecting a wind farm based on a drone provided by an embodiment of the present invention. The method includes the following steps:

[0061] S101, obtaining an image set of a target fan blade from multiple angles, classifying the images to obtain a first image set and a second image set;

[0062] S102, determining an adaptive weight according to the first image set, and updating the rules of a preset algorithm according to the adaptive weight to obtain a target algorithm;

[0063] S103, performing defect detection on the first image set according to the target algorithm to obtain defect data, and calculating the amplitude parameter of the target fan blade according to the second image set;

[0064] S104, judging the wear of the target fan blade according to the defect data and the amplitude parameter.

[0065] Multiple angles include: the front, the back, and the side;

[0066] Based on the method for inspecting a wind farm based on a drone provided by an embodiment of the present invention, by obtaining images of a target fan blade from multiple angles and performing classification processing, and combining with the update algorithm rules of the adaptive weight, accurate detection and calculation of the defect data and amplitude parameter of the fan blade are realized. This comprehensive detection method can comprehensively evaluate the wear state of the fan blade, effectively balance the contradiction between detection accuracy and efficiency, provide a reliable basis for the refined maintenance of wind turbines, significantly improve the reliability and safety of equipment operation, and at the same time reduce the maintenance cost.

[0067] In one implementation, the drone inspects the wind turbines in the wind farm, takes pictures of the fan blades of the wind turbines through a pre-planned path, the drone hovers within a safe range, and obtains an image set of the fan blades at a preset period. The image set includes the front, the back, and the side. Among them, the front refers to the front view of the fan blade (the human eye observes it as a fan pattern), and the back is the opposite view of the front. Taking pictures from the front and back is to obtain a more comprehensive understanding of the wear condition on the surface of the fan blade. The first image set is the front and back of the fan blade, and the second image set is the side of the fan blade.

[0068] In one implementation, the wear of the fan blades is mainly affected by natural factors. Since wind turbines are often installed in open mountainous areas and offshore regions, the fan blades are vulnerable to wear such as lightning strikes, sand and dust erosion, and salt spray corrosion. If the inspection is not timely or accurate, it will lead to accelerated wear of the wind turbine. For example, abnormal wear such as internal structural damage of the fan blades caused by lightning strikes, expansion of surface cracks of the fan blades due to pits caused by sand and dust impact, and large-area peeling of the surface coating of the blades caused by salt spray corrosion.

[0069] In one implementation, the images are classified according to the type of angle. The first image set is used to judge the damage on the surface of the fan blade; the second image set is used to determine the amplitude of the fan blade, and the structural stress damage of the fan blade is judged according to the amplitude of the fan blade.

[0070] In one implementation, preset algorithms are as follows: 1. Object detection algorithm, such as using the Region Proposal Network (Faster R-CNN) to directly locate the defect position and classify (such as cracks, dents). Faster R-CNN has higher accuracy but slower speed, and is suitable for multi-class defect detection (such as simultaneously identifying lightning strike dents and salt spray corrosion). The detection of small targets (such as micro-cracks) depends on the optimization of the network structure and may be misdetected in complex backgrounds; 2. Semantic segmentation algorithms (U-Net, DeepLab), which achieve pixel-level defect segmentation through an encoder-decoder structure and output the precise contour of the defect (such as the coating peeling area). U-Net combines skip connections to retain details, and DeepLab uses dilated convolution to expand the receptive field. It has good segmentation effect on large-area irregular defects such as salt spray corrosion, and can quantify the damaged area, but requires high-resolution images and fine annotations, and the computational cost is relatively high; 3. Transformer series models (such as ViT, Swin Transformer), which capture global context relationships through self-attention mechanisms, process image patch sequences to identify long-range dependent defect patterns (such as through-cracks). They perform excellently on large-scale datasets and have strong feature extraction capabilities for complex defects (such as internal structural damage), but require a large amount of training data and GPU computing power support, and the implementation cost is high.

[0071] In one embodiment, before classifying the images to obtain the first image set and the second image set, it includes:

[0072] Extract features from the images in the image set through the target YOLOv8 model; delete the C2f module of the eighth layer in the YOLOv8 model, and replace all Conv modules in the YOLOv8 model with the target Conv model to obtain the target YOLOv8 model;

[0073] The processing process of the target Conv model for the images:

[0074] Obtain the original image, input the original image into the Conv module to obtain the global feature map, input the original image into the DSConv-x module to obtain the first tubular feature map, and input the original image into the DSConv-y module to obtain the second tubular feature map;

[0075] Concatenate the global feature map, the first tubular feature map, and the second tubular feature map to obtain the first feature map, perform a max pooling operation on the first feature map to obtain the second feature map, and use the second feature map as the output of the target Conv model.

[0076] In one implementation, the target YOLOv8 model is used to crop the images taken from multiple angles before image classification, only retaining most of the images of the fan blades, and then classifying the images.

[0077] In one implementation, by deleting the C2f module in the eighth layer of the YOLOv8 model, the model structure is simplified, and the computational complexity and the number of parameters are reduced. This structural optimization helps to improve the running efficiency of the model, reduce the inference time, and at the same time avoid the overfitting problem that may be caused by the complex structure, making the model more efficient and easier to deploy in practical applications.

[0078] In one implementation, the DSConv-x module focuses on the dynamic offset accumulation in the x-axis direction to achieve horizontal tubular feature extraction. It performs depthwise convolution on the input image along the x-axis to extract local features in the horizontal direction; generates dynamic offset amounts in the x-axis direction through learnable parameters, adjusts the convolutional kernel position along the horizontal serpentine path (such as alternating left and right sliding), and accumulates the offset amounts in adjacent regions to expand the receptive field; performs pointwise convolution on the offset features to fuse multi-channel information and generate the first tubular feature map focused on the x-axis direction; similarly, the DSConv-y module focuses on the vertical feature extraction in the y-axis direction, performs depthwise convolution along the y-axis to capture vertical local features; dynamically generates y-axis offset amounts, slides the convolutional kernel along the up-and-down alternating serpentine path, and covers a larger vertical range by accumulating the offset amounts; pointwise convolution integrates the channel dimension and outputs the second tubular feature map mainly structured by the y-axis to enhance the representation ability of vertical tubular targets.

[0079] In one implementation, direction specificity: DSConv-x / y respectively strengthens the capture of geometric features in the horizontal / vertical direction through axis-aligned depth convolution and dynamic offset; serpentine path and offset accumulation: referring to the alternating sliding mechanism of serpentine convolution in the figure, combined with the offset learning of deformable convolution (DConv), to achieve direction-sensitive feature enhancement; computational lightweight: depthwise separable convolution (depth convolution + point convolution) significantly reduces the number of parameters, and cooperates with the dynamic offset mechanism to balance performance and efficiency.

[0080] In one embodiment, determining the adaptive weights according to the first image set includes:

[0081] Preprocessing the images in the first image set to obtain an initial image set, and segmenting the initial images to obtain a regional image set; the initial image is any one in the initial image set, and the target region is any one in the regional image set;

[0082] Obtain the average gray value of the target region, calculate the gray difference according to the average gray value and the maximum gray value. If the gray difference > the gray threshold, determine that the target region is a damaged region;

[0083] Calculate the dynamic threshold according to the average gray value and the gray span, obtain the number of pixels of the fan blades in the target region, and calculate the number of seeds according to the average gray value, the gray span and the number of pixels;

[0084] Use the dynamic threshold and the number of seeds as the adaptive weights of the preset algorithm; the preset algorithm is the region growing algorithm.

[0085] In one implementation, preprocessing the color images captured by the drone includes: gray processing, bilateral filtering, histogram equalization, etc.; 1. First, convert the color images captured by the drone for monitoring into gray images for processing. The general methods of gray conversion include maximum gray, average gray and weighted average gray. Maximum gray is suitable for images with obvious brightness changes, while average gray is suitable for images that need to smooth the brightness changes, and weighted average gray is suitable for images that need to adjust the brightness changes. In order to highlight the defect area of the wind turbine blade, weighted average gray is used for processing to enhance the visual contrast effect (improve the contrast between the fan blade and the background and make the outline of the fan blade clearer); 2. Process the above images through bilateral filtering (the images captured by the drone often have problems such as noise and unclear blade edges). Bilateral filtering is used as a non-linear filter for image processing. Since bilateral filtering takes into account both spatial information and gray similarity, it can both reduce noise and ensure the smoothness of the edges of the wind turbine blades; 3. In order to distinguish the features of the fan blade and the background in the image, as well as the features of the defective area of the fan blade and the normal area of the fan blade, operate through histogram equalization. Histogram equalization refers to the operation of disturbing and redistributing the distribution of the original gray values that are too scattered or too concentrated, so as to increase the effective range of the gray values and improve the contrast of the gray image; 4. After the above features are strengthened, it is more conducive to foreground-background distinction, and this operation is carried out through the Grab-cut algorithm. The Grab-cut algorithm is an iterative energy minimization segmentation algorithm. Each iteration will optimize the parameters of the Gaussian mixture model in order to model the target and the background, and finally realize the segmentation of the background and the foreground.

[0086] In one implementation, the preset algorithm is the region growing algorithm. The gray difference is calculated based on the average gray value and the maximum gray value. The maximum gray value is the largest gray value in the target region. The gray difference is compared with the gray threshold to judge the gray boundary of the possible damaged region. The dynamic threshold is calculated based on the average gray value and the gray span. The dynamic threshold is used to distinguish the gray difference between the damaged region and the normal region. The dynamic threshold: E = x i *n 2 *α, where E is the dynamic threshold, α is a constant proportionality coefficient, x i is the minimum gray value, n is the gray span, and the value range of n is n = 1, 2, …, 255 - i. The number of seeds is calculated based on the average gray value, the gray span, and the number of pixels, and the number of initial points for region growth starting from the minimum gray point (x i ) is determined. The number of seeds: m = x i *n*β*K, where m is the number of seeds, β is a constant proportionality coefficient, and K is the number of pixels in the target region. The dynamic threshold defines the separation boundary between the damaged and normal regions, improving the detection specificity. The number of seeds ensures the complete coverage of the damaged region, enhancing the detection sensitivity. The combination of the two solves the overfitting or underfitting (such as missed detection) problems caused by fixed parameters in traditional methods.

[0087] In one implementation, for crack detection (medium area, small number): A higher threshold (E = 5 - 10) and fewer seed points (m is smaller) are used to accurately locate cracks scattered in the first half of the gray level by suppressing low-gray noise interference. For pitting detection (small area, large number): The threshold is reduced (E = 1 - 3) and the number of seed points is increased (m is larger) to enhance the sensitivity to dense and minute defects and avoid missed detection. For surface peeling and oil stain detection (large area, extremely small number): For features near gray level 0, a very low threshold (E < 1) and sparse seed points are used to quickly cover a large range of abnormal regions.

[0088] In one implementation, after defect extraction of the target fan blade, post-processing is required to obtain defect data (the defect data is the total area of the defect region). The defective points (the edge part is not clean, with burrs, affecting the accuracy of defect recognition) included in the binary image are optimized, and morphological algorithms are calculated for the defect recognition image: 1. Erosion operation (the erosion operation is used to shrink the target region, used to shrink the image or remove small regions), formula: where A here is the object of erosion, B is the structuring element. Only when each element of B is 1, the corresponding value of A at that position will be retained as 1, otherwise, it will be assigned 0, and then B continues to move until the algorithm ends. 2. Dilation operation (the dilation operation is used to expand the target region, used to fill the hollow part of the target region and eliminate its internal noise), formula: Among them, A here is the object to be dilated, and B represents the structuring element. When the structuring element B is at a certain position, if there is at least one corresponding position in A whose value is 1 at the position where the value of B is 1 (i.e., the marked position 1 of B), then the value of that position is assigned 1; otherwise, it is assigned 0. Then B continues to move until the algorithm ends. 3. Opening operation: The opening operation first performs an etching operation and then a hole-expanding operation, which can remove burrs and small areas and has little impact on the areas of other connected regions. 4. Closing operation: The closing operation is the opposite of the opening operation. It first performs a dilation operation and then an erosion operation, which can eliminate small holes or gaps inside the target region and has little impact on the areas of other connected regions.

[0089] In one embodiment, calculating the amplitude parameter of the target fan blade according to the second image set includes:

[0090] Extracting frames from the second image set according to a preset period to obtain a sequence image set, and performing feature recognition on the sequence image set to obtain a feature image set;

[0091] Performing displacement calculation on the feature image set through an optical flow algorithm to obtain a node displacement set, performing fitting according to the node displacement set to obtain an amplitude curve, and calculating the slope according to the amplitude curve to obtain the amplitude parameter.

[0092] In one implementation, extracting frames from the second image set according to a preset period can reduce the amount of data calculation. After the frame extraction operation, the time interval between two adjacent frames is Δt. If the frame rate is 30 Hz, then Δt = 1 / 30 ≈ 0.033 seconds; for example: calculating the feature point velocities u = 10 pixels / second and v = 5 pixels / second through an optical flow algorithm, Δq = u * Δt = 10 * (1 / 30) ≈ 0.333 pixels (horizontal), p = v * Δt = 5 * (1 / 30) ≈ 0.167 pixels (vertical); the node displacement represents the displacement amplitude between the current frame and the preset frame (the displacement deviation degree between the current moment and the preset moment). The node displacements in the node displacement set are fitted to obtain an amplitude curve, and the amplitude parameter is calculated through the slope of the amplitude curve, which can dynamically quantify the growth or decay rate of the swing amplitude.

[0093] In one implementation, frames are extracted from the original image set at preset intervals to reduce the number of redundant frames, avoid repeated calculations for similar scenes, and significantly reduce storage and computational overhead while retaining key motion information. This is particularly suitable for long-term monitoring scenarios to ensure real-time processing capabilities. The displacement set of nodes is calculated based on the optical flow algorithm to accurately quantify the displacement amplitude between adjacent frames (instantaneous motion within a time interval of Δt). Combining physical calibration parameters, the actual swing distance can be directly obtained, providing motion data with high spatio-temporal resolution for subsequent analysis. Curve fitting (such as polynomial or sine fitting) is performed on the node displacement set to smooth out random noise interference and extract the overall change trend and periodic pattern of the swing amplitude, facilitating the distinction between normal swings and abnormal jitters (such as sudden increases or unstable fluctuations), and enhancing the reliability of long-term fault identification.

[0094] In one embodiment, wear of the target fan blade is judged based on defect data and amplitude parameters, including:

[0095] If the defect data ≤ area threshold and the amplitude parameter ≤ amplitude threshold, the target fan blade is judged to be in normal condition;

[0096] If the defect data > area threshold and the amplitude parameter ≤ amplitude threshold, the target fan blade is judged to have external wear;

[0097] If the defect data ≤ area threshold and the amplitude parameter > amplitude threshold, the target fan blade is judged to have internal wear;

[0098] If the defect data > area threshold and the amplitude parameter > amplitude threshold, the target fan blade is judged to have comprehensive wear.

[0099] In one implementation, when the target fan blade is judged to be in normal condition, regular inspections are performed to continuously monitor the data; when the target fan blade is judged to have external wear, the monitoring period is shortened (such as once a week), the pneumatic balance and the tightness of the connecting parts are checked, maintenance spare parts are prepared, and a preventive maintenance plan is formulated; when the target fan blade is judged to have internal wear, the machine is immediately stopped for non-destructive testing (such as ultrasonic flaw detection), the worn area is repaired, the balance of the fan blade is recalibrated, and damaged components are replaced; when the target fan blade is judged to have comprehensive wear, the machine is emergently stopped, the faulty fan blade is isolated, the fan blade is completely replaced, and the damage to the drive chain and tower structure is checked.

[0100] In one implementation, the above judgment step realizes the accurate classification of the wear degree of the target fan blade through multi-level threshold judgment based on defect data and amplitude parameters, and formulates differentiated maintenance strategies for different wear levels. This hierarchical diagnosis and processing method can effectively optimize the allocation of maintenance resources, avoid over-maintenance or neglect of potential faults. For normal fan blades, regular inspections are sufficient to ensure the stable operation of the equipment; for externally worn fan blades, potential problems are prevented from escalating by shortening the monitoring cycle and preparing repair spare parts in advance; for internally worn fan blades, the equipment is stopped in time for repair and recalibration to prevent the expansion of faults; for comprehensively worn fan blades, the equipment is stopped urgently and components are replaced comprehensively to avoid major equipment accidents. This refined maintenance strategy can significantly improve the reliability and safety of equipment operation, extend the equipment life, and reduce the maintenance cost.

[0101] Based on the same inventive concept, an embodiment of the present invention also provides a wind farm inspection system based on an unmanned aerial vehicle. Refer to Figure 2 , Figure 2 which is a schematic structural diagram of a wind farm inspection system based on an unmanned aerial vehicle provided by an embodiment of the present invention, including: a data acquisition module, a weight determination module, a defect detection module, and a wear judgment module:

[0102] The data acquisition module is used to obtain an image set of the target fan blade from multiple angles, and classify the images to obtain a first image set and a second image set; the multiple angles include: the front, the back, and the side;

[0103] The weight determination module is used to determine an adaptive weight according to the first image set, and update the rules of a preset algorithm according to the adaptive weight to obtain a target algorithm;

[0104] The defect detection module is used to detect defects in the first image set according to the target algorithm to obtain defect data, and calculate the amplitude parameter of the target fan blade according to the second image set;

[0105] The wear judgment module is used to judge the wear of the target fan blade according to the defect data and the amplitude parameter.

[0106] Based on the wind farm inspection system based on an unmanned aerial vehicle provided by an embodiment of the present invention, by obtaining images of the target fan blade from multiple angles and performing classification processing, and combining the adaptive weight update algorithm rules, accurate detection and calculation of the fan blade defect data and amplitude parameter are realized. This comprehensive detection method can comprehensively evaluate the wear state of the fan blade, effectively balance the contradiction between detection accuracy and efficiency, provide a reliable basis for the refined maintenance of wind turbines, significantly improve the reliability and safety of equipment operation, and reduce the maintenance cost at the same time.

[0107] In one embodiment, the improvement of the target YOLOv8 model includes:

[0108] Delete the C2f module in the eighth layer of the YOLOv8 model, and replace all Conv modules in the YOLOv8 model with the target Conv model to obtain the target YOLOv8 model;

[0109] The improvement of the target Conv model includes:

[0110] Input the original image into the Conv module to obtain a global feature map, input the original image into the DSConv-x module to obtain a first tubular feature map, and input the original image into the DSConv-y module to obtain a second tubular feature map;

[0111] Concatenate the global feature map, the first tubular feature map, and the second tubular feature map to obtain a first feature map, perform a max pooling operation on the first feature map to obtain a second feature map, and use the second feature map as the output of the target Conv model.

[0112] In one embodiment, the weight determination module includes: an image segmentation module, a damage judgment module, a parameter determination module, and an algorithm update module:

[0113] The image segmentation module is used to preprocess the images in the first image set to obtain an initial image set, and segment the initial images to obtain a regional image set; the initial image is any one in the initial image set, and the target region is any one in the regional image set;

[0114] The damage judgment module is used to obtain the average gray value of the target region, calculate the gray difference according to the average gray value and the maximum gray value, and if the gray difference > the gray threshold, determine that the target region is a damaged region;

[0115] The parameter determination module is used to calculate a dynamic threshold according to the average gray value and the gray span, obtain the number of pixels of the fan blade in the target region, and calculate the number of seeds according to the average gray value, the gray span, and the number of pixels;

[0116] The algorithm update module is used to use the dynamic threshold and the number of seeds as the adaptive weights of a preset algorithm; the preset algorithm is a region growing algorithm.

[0117] In one embodiment, the defect detection module further includes: an image frame extraction module and an amplitude calculation module:

[0118] The image frame extraction module is used to extract frames from the second image set according to a preset period to obtain a sequence image set, and perform feature recognition on the sequence image set to obtain a feature image set;

[0119] An amplitude calculation module is configured to calculate the displacement of a set of feature images through an optical flow algorithm to obtain a set of node displacements, fit an amplitude curve based on the set of node displacements, and calculate an amplitude parameter based on the amplitude curve.

[0120] In one embodiment, the wear judgment module includes:

[0121] A first judgment module for determining that the target fan blade is in a normal condition if the defect data ≤ area threshold and the amplitude parameter ≤ amplitude threshold;

[0122] A second judgment module for determining that the target fan blade has external wear if the defect data > area threshold and the amplitude parameter ≤ amplitude threshold;

[0123] A third judgment module for determining that the target fan blade has internal wear if the defect data ≤ area threshold and the amplitude parameter > amplitude threshold;

[0124] A fourth judgment module for determining that the target fan blade has comprehensive wear if the defect data > area threshold and the amplitude parameter > amplitude threshold.

[0125] The above has described in detail an embodiment of the present invention, but the content is only a preferred embodiment of the present invention and cannot be considered as limiting the scope of implementation of the present invention. All equivalent changes and improvements made according to the scope of the present invention application shall still fall within the scope covered by the patent of the present invention.

Claims

1. A method for inspecting a wind farm based on an unmanned aerial vehicle, characterized in that, The method includes: Obtaining an image set of the target fan blade from multiple angles, and classifying the images to obtain a first image set and a second image set; the multiple angles include: the front, the back, and the side; Determining an adaptive weight according to the first image set, and updating the rules of a preset algorithm according to the adaptive weight to obtain a target algorithm; Performing defect detection on the first image set according to the target algorithm to obtain defect data, and calculating the amplitude parameter of the target fan blade according to the second image set; Judging the wear of the target fan blade according to the defect data and the amplitude parameter.

2. The method for inspecting a wind farm based on an unmanned aerial vehicle according to claim 1, wherein Before classifying the images to obtain a first image set and a second image set, it includes: Performing feature extraction on the images in the image set through a target YOLOv8 model; deleting the C2f module of the eighth layer in the YOLOv8 model, and replacing all Conv modules in the YOLOv8 model with a target Conv model to obtain a target YOLOv8 model; The processing process of the target Conv model for images: Obtaining an original image, inputting the original image into a Conv module to obtain a global feature map, inputting the original image into a DSConv-x module to obtain a first tubular feature map, and inputting the original image into a DSConv-y module to obtain a second tubular feature map; Splicing the global feature map, the first tubular feature map, and the second tubular feature map to obtain a first feature map, performing a max pooling operation on the first feature map to obtain a second feature map, and using the second feature map as the output of the target Conv model.

3. The method for inspecting a wind farm based on an unmanned aerial vehicle according to claim 1, wherein Determining an adaptive weight according to the first image set, including: Preprocessing the images in the first image set to obtain an initial image set, and segmenting the initial images to obtain a regional image set; the initial image is any one in the initial image set, and the target region is any one in the regional image set; Obtaining the average gray value of the target region, calculating the gray difference according to the average gray value and the maximum gray value, and if the gray difference > gray threshold, determining that the target region is a damaged region; Calculating a dynamic threshold according to the average gray value and the gray span, obtaining the number of pixels of the fan blade in the target region, and calculating the number of seeds according to the average gray value, the gray span, and the number of pixels; Using the dynamic threshold and the number of seeds as the adaptive weight of the preset algorithm; the preset algorithm is a region growing algorithm.

4. The method for inspecting a wind farm based on an unmanned aerial vehicle according to claim 1, wherein Calculating the amplitude parameter of the target fan blade according to the second image set, including: Extracting frames from the second image set at a preset period to obtain a sequence image set, and performing feature recognition on the sequence image set to obtain a feature image set; Calculating the node displacement set by using an optical flow algorithm for the feature image set, fitting an amplitude curve according to the node displacement set, and calculating the amplitude parameter according to the slope of the amplitude curve.

5. A method for inspecting a wind farm based on an unmanned aerial vehicle according to claim 1, characterized in that, Judging the wear of the target fan blade according to the defect data and the amplitude parameter, including: If the defect data ≤ area threshold and the amplitude parameter ≤ amplitude threshold, it is determined that the target fan blade is in normal condition; If the defect data > area threshold and the amplitude parameter ≤ amplitude threshold, it is determined that the target fan blade has external wear; If the defect data ≤ area threshold and the amplitude parameter > amplitude threshold, it is determined that the target fan blade has internal wear; If the defect data > area threshold and the amplitude parameter > amplitude threshold, it is determined that the target fan blade has comprehensive wear.

6. A drone-based wind farm inspection system, characterized in that, The system includes: a data acquisition module, a weight determination module, a defect detection module, and a wear judgment module: The data acquisition module is used to obtain an image set of the target fan blade from multiple angles, and classify the images to obtain a first image set and a second image set; the multiple angles include: the front, the back, and the side; The weight determination module is used to determine an adaptive weight according to the first image set, and update the rules of a preset algorithm according to the adaptive weight to obtain a target algorithm; The defect detection module is used to detect defects in the first image set according to the target algorithm to obtain defect data, and calculate the amplitude parameter of the target fan blade according to the second image set; The wear judgment module is used to judge the wear of the target fan blade according to the defect data and the amplitude parameter.

7. The wind farm inspection system based on an unmanned aerial vehicle according to claim 6, characterized in that, The improvement of the target YOLOv8 model includes: Delete the C2f module in the eighth layer of the YOLOv8 model, and replace all Conv modules in the YOLOv8 model with a target Conv model to obtain a target YOLOv8 model; The improvement of the target Conv model includes: Input the original image into the Conv module to obtain a global feature map, input the original image into the DSConv-x module to obtain a first tubular feature map, and input the original image into the DSConv-y module to obtain a second tubular feature map; Concatenate the global feature map, the first tubular feature map, and the second tubular feature map to obtain a first feature map, perform a max pooling operation on the first feature map to obtain a second feature map, and use the second feature map as the output of the target Conv model.

8. The wind farm inspection system based on an unmanned aerial vehicle according to claim 6, characterized in that, The weight determination module includes: an image segmentation module, a damage judgment module, a parameter determination module, and an algorithm update module: The image segmentation module is used to preprocess the images in the first image set to obtain an initial image set, and segment the initial images to obtain a regional image set; the initial image is any one in the initial image set, and the target region is any one in the regional image set; The damage judgment module is used to obtain the average gray value of the target region, calculate the gray difference according to the average gray value and the maximum gray value, and if the gray difference > gray threshold, it is determined that the target region is a damaged region; The parameter determination module is used to calculate a dynamic threshold according to the average gray value and the gray span, obtain the number of pixels of the fan blade in the target region, and calculate the number of seeds according to the average gray value, the gray span, and the number of pixels; The algorithm update module is used to use the dynamic threshold and the number of seeds as the adaptive weight of the preset algorithm; the preset algorithm is the region growing algorithm.

9. The wind farm inspection system based on an unmanned aerial vehicle according to claim 6, wherein, The defect detection module further includes: an image frame extraction module and an amplitude calculation module: The image frame extraction module is configured to extract frames from the second image set at a preset period to obtain a sequence image set, and perform feature recognition on the sequence image set to obtain a feature image set; The amplitude calculation module is configured to calculate the displacement of nodes of the feature image set through an optical flow algorithm to obtain a node displacement set, perform fitting according to the node displacement set to obtain an amplitude curve, and calculate the slope according to the amplitude curve to obtain an amplitude parameter.

10. The wind farm inspection system based on an unmanned aerial vehicle according to claim 6, wherein, The wear judgment module includes: The first judgment module is configured to determine that the target fan blade is in a normal condition if the defect data ≤ area threshold and the amplitude parameter ≤ amplitude threshold; The second judgment module is configured to determine that the target fan blade has external wear if the defect data > area threshold and the amplitude parameter ≤ amplitude threshold; The third judgment module is configured to determine that the target fan blade has internal wear if the defect data ≤ area threshold and the amplitude parameter > amplitude threshold; The fourth judgment module is configured to determine that the target fan blade has comprehensive wear if the defect data > area threshold and the amplitude parameter > amplitude threshold.

Images