Wind turbine blade detection method, system, device and storage medium

Abstract

The present invention discloses a method, system, device and storage medium for detecting wind turbine blades. First, the first point cloud data of the target blade is obtained by the lidar carried by the unmanned aerial vehicle (UAV); then, plane fitting and grid map conversion are performed on the first point cloud data to obtain the fitted plane and the grid map; next, the grid map is scanned and queried to obtain multiple blade occupancy points, and the blade occupancy points are clustered to obtain the blade position; based on the blade position, the hub position is determined, and the blade angle is obtained according to the hub position and the blade position; according to the fitted plane and the hub position, the orbital motion trajectory of the UAV is updated, and the step of obtaining the first point cloud data of the target blade by the lidar carried by the UAV is returned until the blade angle obtained currently and the blade angle obtained in the previous round meet the preset convergence condition. The present invention improves the accuracy of wind turbine blade angle detection and also improves the quality of wind turbine blade images, and can be applied to the field of machine vision technology.

Description

Technical Field

[0001] The present invention relates to the technical field of machine vision, and in particular to a method, system, device and storage medium for detecting wind turbine blades. Background Art

[0002] In recent years, driven by the continuous progress of renewable energy technologies, wind energy has become an important source of electricity. As of 2022, the global installed wind power capacity has soared to a remarkable 906 gigawatts. Wind farms are strategically distributed in hilly, mountainous, and coastal areas, leveraging the abundant wind resources to offer substantial energy potential for the installation of efficient wind turbines. Despite these advancements, severe weather conditions such as strong winds, lightning, hail, and rain pose threats to wind turbine blades, potentially leading to damage. Regular inspection of these blades is crucial for timely detection and correction of problems, thus avoiding interference with the normal operation of the generator.

[0003] Traditional methods for inspecting wind turbine blades have relied on manual techniques, involving the use of ground telescopes or personnel suspended on the generator using safety ropes. However, these methods are time-consuming, labor-intensive, inefficient, and pose safety risks. The introduction of drone technology has revolutionized these inspection methods, enabling skilled remote pilots to conduct comprehensive assessments efficiently. But this requires proficiency in both flying and photography. The emergence of drone-based automated inspection systems has fully utilized automation and artificial intelligence to autonomously guide the operation of drones during flight and image capture. These technological advancements can not only reduce labor costs but also improve inspection efficiency and enhance image quality.

[0004] Given the potential benefits of drone-based automated wind turbine blade inspection in maintenance, the field faces significant challenges. Although researchers have proposed various methods to address these issues, there are still some unresolved problems:

[0005] 1) Limited supply of advanced drone equipment: Commercial drones integrated with high-definition gimbal cameras, high-precision lidar, and powerful computing units are scarce. Although custom platforms exist, they often lack the stability and safety crucial for effective wind turbine inspection.

[0006] 2) Constraints in blade angle detection: Existing detection methods are usually limited to specific blade angles, restricting their versatility and adaptability. Some methods require preset blade angles, hindering automation. Vision-based blade angle detection methods are vulnerable to interference, affecting accuracy and robustness.

[0007] 3) Lack of automatic exposure adjustment: The exposure adjustment techniques adopted by commercial gimbal cameras usually rely on the overall image brightness, which limits their effectiveness in specific detection scenarios. Existing detection methods often overlook the importance of adjusting exposure during image capture to adapt to different lighting conditions. This oversight may result in under-exposure or over-exposure of the blade area in the image, especially under challenging lighting conditions, thereby compromising the visibility of key details and hindering subsequent damage assessment.

[0008] The above problems need to be solved urgently. Summary of the Invention

[0009] An object of the present invention is to solve at least to some extent one of the technical problems existing in the prior art.

[0010] To this end, an object of an embodiment of the present invention is to provide a method for detecting a wind turbine blade, which improves the accuracy of detecting the angle of the wind turbine blade and also improves the quality of the wind turbine blade image.

[0011] Another object of an embodiment of the present invention is to provide a wind turbine blade detection system.

[0012] In order to achieve the above technical objectives, the technical solutions adopted in the embodiments of the present invention include:

[0013] In the first aspect, an embodiment of the present invention provides a method for detecting a wind turbine blade, including the following steps:

[0014] Obtain the first point cloud data of the target blade through the lidar carried by the unmanned aerial vehicle;

[0015] Perform plane fitting and grid map conversion on the first point cloud data to obtain a fitted plane and a grid map;

[0016] Scan and query the grid map to obtain a plurality of blade occupancy points, and cluster the blade occupancy points to obtain the blade position;

[0017] Determine the hub position according to the blade position, and obtain the blade angle according to the hub position and the blade position;

[0018] Update the orbital motion trajectory of the unmanned aerial vehicle according to the fitted plane and the hub position, and return to the step of obtaining the first point cloud data of the target blade through the lidar carried by the unmanned aerial vehicle until the blade angle obtained currently and the blade angle obtained in the previous round meet the preset convergence condition, and output the currently obtained blade angle as the target blade angle.

[0019] Further, in an embodiment of the present invention, the performing plane fitting and grid map conversion on the first point cloud data to obtain a fitted plane and a grid map specifically includes:

[0020] Performing plane fitting on the first point cloud data by the random sample consensus algorithm to obtain the fitted plane and the corresponding unit normal vector;

[0021] Determining the map resolution according to the size of the wind turbine, and performing grid map conversion on the first point cloud data according to the map resolution to obtain the grid map.

[0022] Further, in an embodiment of the present invention, the performing a scan query on the grid map to obtain a plurality of blade occupancy points and clustering the blade occupancy points to obtain the blade positions specifically includes:

[0023] Determining a vertical scan direction according to the unit normal vector, and performing a vertical scan on the grid map according to the vertical scan direction to obtain a plurality of first sampling points;

[0024] Determining a circular scan trajectory on the fitted plane according to the first sampling points, and performing a circular scan on the grid map according to the circular scan trajectory to obtain a plurality of second sampling points;

[0025] Querying the grid status corresponding to the second sampling points in the grid map, and when the grid status is occupied, determining the second sampling points as the blade occupancy points;

[0026] Clustering the blade occupancy points to obtain three blade occupancy point sets;

[0027] Determining the corresponding blade positions according to the average positions of the blade occupancy points in each blade occupancy point set.

[0028] Further, in an embodiment of the present invention, the determining the hub position according to the blade positions and obtaining the blade angles according to the hub position and the blade positions specifically includes:

[0029] Constructing a target triangle according to the three blade positions, and taking the Fermat point of the target triangle as the hub position;

[0030] Determining the corresponding blade direction vector according to the directed connection line from the hub position to the blade position;

[0031] Determining the corresponding blade angle according to the blade direction vector.

[0032] Further, in an embodiment of the present invention, updating the orbital motion trajectory of the drone according to the fitting plane and the hub position specifically includes:

[0033] Determine an orbital plane according to the fitting plane, such that the orbital plane is parallel to the fitting plane;

[0034] Take the projection position of the hub position on the orbital plane as the orbital center position;

[0035] Determine the updated orbital motion trajectory on the orbital plane according to a preset orbital radius and the orbital center position.

[0036] Further, in an embodiment of the present invention, the drone is also equipped with a gimbal camera, and the wind turbine blade detection method further includes the following steps:

[0037] Obtain second point cloud data of the target blade through the lidar;

[0038] Perform linear fitting on the second point cloud data to obtain the straight line where the blade is located, and determine the detection target space coordinates according to the drone position and the straight line where the blade is located;

[0039] Adjust the shooting angle of the gimbal camera according to the detection target space coordinates, and determine the corresponding detection target image coordinates according to the detection target space coordinates;

[0040] Set the exposure value of the gimbal camera, and obtain image data of the target blade through the gimbal camera;

[0041] Convert the image data into a grayscale image, and determine the target circular area in the grayscale image according to the detection target image coordinates;

[0042] Determine the average grayscale value of the target circular area, adjust the exposure value of the gimbal camera according to the average grayscale value, and return to the step of obtaining the image data of the target blade through the gimbal camera until the average grayscale value meets a preset threshold range, and output the currently captured image data as the target blade image data.

[0043] Further, in an embodiment of the present invention, the threshold range includes a threshold upper limit and a threshold lower limit. Adjusting the exposure value of the gimbal camera according to the average grayscale value specifically includes:

[0044] When the average grayscale value is greater than the threshold upper limit, reduce the exposure value according to a preset step factor;

[0045] When the average gray value is less than the upper threshold, increase the exposure value according to the step factor.

[0046] In a second aspect, an embodiment of the present invention provides a wind turbine blade detection system, including:

[0047] A point cloud data acquisition module, configured to acquire first point cloud data of a target blade through a lidar carried by a drone;

[0048] A point cloud data processing module, configured to perform plane fitting and grid map conversion on the first point cloud data to obtain a fitted plane and a grid map;

[0049] A scanning and querying module, configured to scan and query the grid map to obtain a plurality of blade occupancy points, and cluster the blade occupancy points to obtain the blade position;

[0050] A blade angle determination module, configured to determine the hub position according to the blade position, and obtain the blade angle according to the hub position and the blade position;

[0051] A drone trajectory update module, configured to update the orbital motion trajectory of the drone according to the fitted plane and the hub position, and return to the step of acquiring the first point cloud data of the target blade through the lidar carried by the drone until the blade angle obtained currently and the blade angle obtained in the previous round meet a preset convergence condition, and output the currently obtained blade angle as the target blade angle.

[0052] In a third aspect, an embodiment of the present invention provides a wind turbine blade detection device, including:

[0053] At least one processor;

[0054] At least one memory, configured to store at least one program;

[0055] When the at least one program is executed by the at least one processor, the at least one processor is caused to implement the above-mentioned wind turbine blade detection method.

[0056] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, in which a program executable by a processor is stored, and the program executable by the processor is used to execute the above-mentioned wind turbine blade detection method when executed by the processor.

[0057] The advantages and beneficial effects of the present invention will be partially given in the following description, partially will become obvious from the following description, or be understood through the practice of the present invention:

[0058] In the embodiment of the present invention, the first point cloud data of the target blade is obtained by the lidar carried by the unmanned aerial vehicle (UAV). The first point cloud data is subjected to plane fitting and grid map conversion to obtain a fitting plane and a grid map. The grid map is scanned and queried to obtain a plurality of blade occupancy points, and the blade occupancy points are clustered to obtain the blade position. The hub position is determined according to the blade position, and the blade angle is obtained according to the hub position and the blade position. The orbital motion trajectory of the UAV is updated according to the fitting plane and the hub position, and the step of obtaining the first point cloud data of the target blade by the lidar carried by the UAV is returned until the blade angle obtained currently and the blade angle obtained in the previous round meet the preset convergence condition, and the blade angle obtained currently is output as the target blade angle. In the embodiment of the present invention, the blade position and the hub position are estimated according to the point cloud data of the blade, the blade angle is determined according to the blade position and the hub position, and the orbital motion trajectory of the UAV is updated according to the hub position, so as to obtain the point cloud data again for the estimation of the blade position and the hub position until the blade angle obtained currently and the blade angle obtained in the previous round meet the preset convergence condition, and the final blade angle is obtained, thereby improving the accuracy of the blade angle detection of the wind turbine. In addition, in the embodiment of the present invention, the straight line where the blade is located is fitted according to the point cloud data of the blade, the detection target space coordinates are determined according to the UAV position and the straight line where the blade is located, the shooting angle of the pan-tilt camera is adjusted according to the detection target space coordinates and the detection target image coordinates are determined, the target circular area is determined in the image data captured by the pan-tilt camera according to the detection target image coordinates, and the exposure value of the pan-tilt camera is adjusted according to the average gray value of the target circular area, thereby improving the quality of the image of the wind turbine blade captured. Description of the Drawings

[0059] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following introduces the drawings required to be used in the embodiments of the present invention. It should be understood that the drawings introduced below only facilitate the clear expression of some embodiments of the technical solutions in the present invention, and those skilled in the art can also obtain other drawings based on these drawings without creative efforts.

[0060] Figure 1 It is a flowchart of the steps of a method for detecting the blade of a wind turbine provided by an embodiment of the present invention;

[0061] Figure 2 It is a schematic diagram of the UAV detection platform and the corresponding space coordinate system provided by an embodiment of the present invention;

[0062] Figure 3 It is a side view of the lidar position arrangement provided by an embodiment of the present invention;

[0063] Figure 4 It is a top view of the lidar position arrangement provided by an embodiment of the present invention;

[0064] Figure 5 Schematic diagram of the spatial position relationship between the drone and the blade provided by the embodiment of the present invention;

[0065] Figure 6 Block diagram of the structure of a wind turbine blade detection system provided by the embodiment of the present invention;

[0066] Figure 7 Block diagram of the structure of a wind turbine blade detection device provided by the embodiment of the present invention. Detailed implementation manners

[0067] The embodiments of the present invention will be described in detail below. The examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals indicate the same or similar elements or elements with the same or similar functions throughout. The embodiments described below by referring to the drawings are exemplary and are only used to explain the present invention and should not be construed as a limitation to the present invention. For the step numbers in the following embodiments, they are only set for the convenience of explanation and illustration, and no limitation is imposed on the order between the steps. The execution order of each step in the embodiments can be adjusted adaptively according to the understanding of those skilled in the art.

[0068] In the description of the present invention, the meaning of "a plurality of" is two or more. If the first and second are described, it is only for the purpose of distinguishing technical features and should not be construed as indicating or implying relative importance or implicitly indicating the quantity of the indicated technical features or implicitly indicating the sequence of the indicated technical features. In addition, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field of the present invention.

[0069] Unmanned aerial vehicles (UAVs) are crucial in the detection of wind turbine blades. However, there are still some unsolved problems in the field of autonomous wind turbine blade detection based on UAVs. First, autonomous detection tasks require long-distance sensing capabilities and high-performance computing capabilities. Unfortunately, typical commercial UAVs are limited by their limited sensing range and computing resources. Second, existing detection methods rely on prior knowledge of the blade angle or limit the detection to specific blade angles. This limits the diversity of this application in different scenarios. Third, the influence of sunlight during the shooting process may lead to uneven exposure, resulting in abnormal brightness and loss of details on the blade. This poses a major challenge for subsequent processing.

[0070] Conducting inspection experiments in an operating wind farm poses challenges in personnel deployment and potential power generation losses. Schäfer et al. conducted experiments in a simulated environment using a Pelican drone equipped with a Hokuyo UTM-30LX lidar. However, bridging the gap between simulation and reality remains a daunting challenge. Parlange et al. conducted experiments on a small-scale outdoor wind turbine model using a Parrot Bebop 2 drone equipped with a GoPro camera. However, the differences between the small-scale model and actual wind turbines limit the practical applicability of the algorithm.

[0071] In an operating wind farm, a stable and reliable drone platform is the foundation of an effective inspection procedure. Complex automated inspection algorithms rely on remote sensing capabilities and real-time decision-making, which pose significant challenges in platform design. Car et al. introduced a custom drone equipped with a Velodyne VLP-16 lidar. However, a custom-designed platform may lack the necessary reliability and stability. In contrast, Castelar Wembers et al. configured a commercial DJI Matrice300RTK drone with a DJI Zenmuse H20T gimbal camera and a Hokuyo UST-20LX lidar. Using a commercial DJI platform demonstrated impressive stability.

[0072] During the blade angle detection process, the wind turbine blades must be fixed at a specific angle. Normally, these blades rotate passively in response to the wind. Some wind turbines can only be fixed at specific angles (such as a positive Y-shape or an inverted Y-shape), which requires a time-consuming locking procedure that depends on wind conditions. The method proposed by Castelar Wembers et al. is designed for blades in an inverted Y-setting and requires precise blade angles. The method of et al. involves presetting the blade angle and requires manual intervention.

[0073] Some modern wind turbines offer the flexibility to be fixed at various angles, simplifying the locking process but potentially leaving the blades fixed at arbitrary angles. In such cases, automated blade angle detection is crucial for inspection automation. Visual methods are commonly used for this purpose. Stokkeland et al. and Parlange et al. applied traditional visual techniques using the Hough transform to detect blade lines and determine the blade angle. Guo et al. introduced a deep learning-based visual method to determine the blade angle by identifying the blade tip. However, vision-based methods are vulnerable to background interference and may lead to instability. Currently, there is a lack of research on using point clouds or grid maps for blade angle detection.

[0074] The images of wind turbine blades captured by drones are crucial for subsequent damage detection. During the image capture process, changes in sunlight and camera angles may result in significant brightness differences between the blade and the background area. However, currently popular commercial gimbal cameras focus more on overall image exposure rather than on specific regions of interest. This approach may lead to overexposure or underexposure of the blade area, potentially masking key details and complicating subsequent damage detection. Implementing real-time automatic exposure adjustment during the image capture process is expected to optimize the uneven brightness and alleviate the challenges of subsequent tasks. As far as is known to the present invention, there is no research on automatically adjusting exposure during the image capture process in the field of wind turbine blade detection.

[0075] To address these challenges, the present invention proposes an automated UAV-based wind turbine blade detection method, which includes three key components. First, the present invention proposes a detection platform equipped with a long-range lidar and an on-board computer, aiming to meet the requirements of multi-sensor integration and high computing power to achieve automated detection. Second, the present invention introduces a blade angle detection algorithm based on a grid map, which achieves superior accuracy in detecting the blade at any angle based on the grid map generated by high-precision lidar. Finally, the present invention proposes a projection-based blade exposure adjustment algorithm to ensure that the brightness and details of the blade surface remain optimal under different lighting conditions during the image capture process. Extensive tests have been carried out in operating wind farms, demonstrating the robust reliability of the platform and algorithms of the present invention and confirming their effectiveness in actual detection scenarios.

[0076] First, the UAV detection platform proposed by the present invention is introduced and described, as Figure 2 shown in the schematic diagram of the UAV detection platform and the corresponding space coordinate system provided by the embodiment of the present invention. The UAV detection platform of the present invention includes a UAV body and a gimbal camera 1, a lidar 2, and an on-board computer 3 provided on the UAV body. Among them, the space coordinate system of the UAV body is O b -x b y b z b , the space coordinate system of the gimbal camera is O c -x c y c z c , the space coordinate system of the lidar is O l -x l y l z lFor the drone, the present invention selects DJI Matrice 300 RTK due to its stability and safety in industrial applications. It also has an Onboard SDK, which simplifies the secondary development process. To capture high-quality detection images, the present invention selects DJI Zenmuse H20T, which is a gimbal camera with a three-axis gimbal and a three-lens system. As the lidar function, the present invention selects Livox MID-360 to obtain long-range and high-precision point cloud data. Finally, as the on-board computer, the present invention selects NUC 11 Pro Kit NUC11TNKi5, which provides sufficient computing power, a compact size, convenient installation, and multiple interfaces to facilitate data exchange between the various hardware elements.

[0077] In Figure 2 , the placement of the components within the platform of the present invention is also shown: the gimbal camera is mounted below the front of the drone body, while the lidar and the on-board computer are located on the top. This configuration is carefully designed to avoid interfering with the normal flight of the drone and the battery loading. In addition, the lidar is located in the front on the top, and the on-board computer is located behind it. To establish a connection between the on-board computer and the drone, the present invention uses a DJI SDK extension module. The drone provides power to the on-board computer and the lidar through a voltage regulation module. Data cables are used to ensure smooth communication between these components. Finally, the present invention adds a protective housing to prevent outdoor dust intrusion and protect the computer and hardware interfaces.

[0078] As Figure 3 shown is a side view of the lidar position arrangement provided by the embodiment of the present invention, and as Figure 4 shown is a top view of the lidar position arrangement provided by the embodiment of the present invention. The placement method of the lidar is shown in Figure 3 and 4 . When the lidar is horizontally positioned, the vertical field of view (FOV) range is from -7 degrees to 52 degrees. To ensure symmetry when the forward vertical FOV is φ = 60°, the present invention sets the pitch angle of the lidar to α = -23°. In the horizontal dimension, the present invention focuses on the point cloud in the front area, so the horizontal FOV is set to θ = 60°.

[0079] The grid map-based blade angle detection algorithm and the projection-based blade exposure adjustment algorithm of the embodiments of the present invention will be described in detail below.

[0080] Referring to Figure 1 , the embodiments of the present invention provide a method for detecting wind turbine blades, which specifically includes the following steps:

[0081] S101. Obtain the first point cloud data of the target blade by means of the lidar carried by the unmanned aerial vehicle;

[0082] S102. Perform plane fitting and grid map conversion on the first point cloud data to obtain the fitted plane and the grid map;

[0083] S103. Conduct a scan query on the grid map to obtain multiple blade occupancy points, and cluster the blade occupancy points to obtain the blade position;

[0084] S104. Determine the hub position according to the blade position, and obtain the blade angle according to the hub position and the blade position;

[0085] S105. Update the orbital motion trajectory of the unmanned aerial vehicle according to the fitted plane and the hub position, and return to the step of obtaining the first point cloud data of the target blade by means of the lidar carried by the unmanned aerial vehicle until the blade angle obtained currently and the blade angle obtained in the previous round meet the preset convergence condition, and output the currently obtained blade angle as the target blade angle.

[0086] In the embodiment of the present invention, the blade position and the hub position are estimated based on the point cloud data of the blade, the blade angle is determined according to the blade position and the hub position, and the orbital motion trajectory of the unmanned aerial vehicle is updated according to the hub position, so as to obtain the point cloud data again for the estimation of the blade position and the hub position until the blade angle obtained currently and the blade angle obtained in the previous round meet the preset convergence condition, and the final blade angle is obtained, thereby improving the accuracy of the blade angle detection of the wind turbine.

[0087] Further as an optional implementation manner, performing plane fitting and grid map conversion on the first point cloud data to obtain the fitted plane and the grid map specifically includes:

[0088] S1021. Perform plane fitting on the first point cloud data by means of the random sample consensus algorithm to obtain the fitted plane and the corresponding unit normal vector;

[0089] S1022. Determine the map resolution according to the size of the wind turbine, and perform grid map conversion on the first point cloud data according to the map resolution to obtain the grid map.

[0090] Specifically, first perform plane fitting and grid map conversion on the acquired point cloud.

[0091] The unmanned aerial vehicle hovers in front of the wind turbine, facing the main shaft. The lidar is used to capture the point cloud of the main shaft and the blade root, denoted as Considering the unique geometric shape of the wind turbine, the present invention uses the random sample consensus algorithm to perform plane fitting to obtain the fitted plane and its unit normal vector v n .

[0092] The present invention starts the orbital motion of the UAV in front of the wind turbine to expand the sensing range of the point cloud. The trajectory of this orbital motion is defined as C d :

[0093] C d ={p|p∈Π d ,‖p - p c ‖ = r c |} (1)

[0094] where Π d represents the orbital plane, such that r c represents the orbital radius. p c ∈Π d represents the calculated orbital center, and the calculation formula is as shown in (2):

[0095] p c = p h - v n ·d c (2)

[0096] where p h represents the main axis position. In particular, p h has a preset initial estimate value and is continuously optimized in subsequent processing. d c represents the distance between the UAV and the main axis, which is set to 20 meters.

[0097] Considering that the point cloud is disordered and high - density, directly processing it may lead to significant computational overhead. Given that the grid map provides an efficient representation and fast query ability for the point cloud, the present invention will be converted into a grid map To adapt to the size of the wind turbine, the present invention sets the resolution of M to r

[0098] Further as an optional implementation manner, the grid map is scanned and queried to obtain multiple blade occupancy points, and the blade occupancy points are clustered to obtain the blade positions, which specifically include:[[]]

[0099] S1031. Determine the vertical scanning direction according to the unit normal vector, and perform vertical scanning on the grid map according to the vertical scanning direction to obtain multiple first sampling points;

[0100] S1032. Determine the circular scanning trajectory according to the first sampling points on the fitting plane, and perform circular scanning on the grid map according to the circular scanning trajectory to obtain multiple second sampling points;

[0101] S1033. Query the grid status corresponding to the second sampling point in the grid map. When the grid status is occupied, determine the second sampling point as a blade occupancy point;

[0102] S1034. Cluster the blade occupancy points to obtain three sets of blade occupancy points;

[0103] S1035. Determine the corresponding blade position according to the average position of the blade occupancy points in each set of blade occupancy points.

[0104] Specifically, the blades are identified by scanning and querying, and then clustering is performed to determine the blade positions.

[0105] The present invention utilizes both vertical scanning and circular scanning. It should be noted that the term "scanning" here refers to an algorithmic process rather than the physical movement of the drone. During the entire scanning process, the drone maintains the aforementioned orbital motion.

[0106] In vertical scanning, scan along v n Scan, sample points p with a step size of Δd s , thereby forming a set Q s , which is defined as shown in the following formula (3):

[0107] Q s ={p s |p s =p c +v n d c +nΔd, n∈N, nΔd<d b} (3)

[0108] where d c represents the distance from p c to the initial scanning position, and d b represents the scanning range. The present invention constructs a plane Π s with v n as the normal vector based on p s . On Π d , a circle C s is constructed with p s as the center and r s as the radius, as shown in the following formula (4). Circular scanning will be performed on C s .

[0109] C s ={p|p∈Π d ,‖p - p s ‖ = r s} (4)

[0110] In circular scanning, the present invention samples points p on C s with a step size of Δlr . In , query the status of the grid containing p r . If the result shows "occupied", mark this position as part of the blade and designate it as the hitting point. All the hitting points of the circular scans are then aggregated into the set Q t , and then clustered into different groups. Successful clustering involves dividing into 3 groups, with each group representing a blade. If the clustering fails, the circular scan is repeated at the next sampling point of the vertical scan until successful clustering is achieved. After clustering, the present invention calculates the average value of each group to obtain the estimated blade position Q b = {p b1 , p b2 , p b3}.

[0111] Further as an optional implementation manner, determine the hub position according to the blade position, and obtain the blade angle according to the hub position and the blade position, which specifically includes:

[0112] S1041. Construct a target triangle according to the positions of three blades, and use the Fermat point of the target triangle as the hub position;

[0113] S1042. Determine the corresponding blade direction vector according to the directed connection line from the hub position to the blade position;

[0114] S1043. Determine the corresponding blade angle according to the blade direction vector.

[0115] Further as an optional implementation manner, update the orbital motion trajectory of the drone according to the fitting plane and the hub position, which specifically includes:

[0116] S1051. Determine the orbital plane according to the fitting plane, so that the orbital plane is parallel to the fitting plane;

[0117] S1052. Use the projection position of the hub position on the orbital plane as the orbital center position;

[0118] S1053. Determine the updated orbital motion trajectory on the orbital plane according to the preset orbital radius and the orbital center position.

[0119] Specifically, the embodiment of the present invention utilizes the Fermat point to optimize the hub position, and then calculates the blade angle.

[0120] Utilize the Fermat point principle to determine the hub position p h . The Fermat point has unique properties when related to a triangle: for a triangle with all three interior angles less than 120 degrees, connecting its Fermat point to the three vertices forms three line segments, and the angles between them are 120 degrees.

[0121] Consider the 120 degrees between the turbine blades and utilize the obtained Q b Form a triangle △B. Given the geometric configuration of the turbine blades, all interior angles of △B are less than 120 degrees. Therefore, the hub position of the wind turbine corresponds to the Fermat point of △B. The present invention can determine the Fermat point of △B as the best estimate of the hub position p h . At the same time, this p h will be used to re-determine the center of the orbital trajectory in the next iteration.

[0122] Connect p h and Q b to determine the blade direction vector V b ={v b1 , v b2 , v b3}, and calculate the blade angle A b ={a b1 , a b2 , a b3}. When the obtained in the current iteration and the obtained in the previous iteration satisfy the following formula (5), the algorithm convergence criterion of the present invention is satisfied, and thus the optimal A b is obtained:

[0123]

[0124] where ε A is a preset threshold.

[0125] The algorithm flow of the blade angle detection algorithm based on the grid map in the embodiment of the present invention is as follows:

[0126]

[0127] Further as an optional implementation manner, the unmanned aerial vehicle is also equipped with a pan-tilt camera, and the wind turbine blade detection method further includes the following steps:

[0128] S201. Obtain the second point cloud data of the target blade through the lidar;

[0129] S202. Perform linear fitting on the second point cloud data to obtain the straight line where the blade is located, and determine the detection target space coordinates according to the position of the unmanned aerial vehicle and the straight line where the blade is located;

[0130] S203. Adjust the shooting angle of the pan-tilt camera according to the detection target space coordinates, and determine the corresponding detection target image coordinates according to the detection target space coordinates;

[0131] S204. Set the exposure value of the pan-tilt camera and obtain the image data of the target blade through the pan-tilt camera;

[0132] S205. Convert the image data into a grayscale image, and determine the target circular region in the grayscale image according to the detected target image coordinates;

[0133] S206. Determine the average grayscale value of the target circular region, adjust the exposure value of the pan-tilt camera according to the average grayscale value, and return to the step of obtaining the image data of the target blade through the pan-tilt camera until the average grayscale value meets the preset threshold range, and output the currently captured image data as the target blade image data.

[0134] Specifically, in the embodiment of the present invention, the straight line where the blade is located is fitted according to the point cloud data of the blade, the detection target space coordinates are determined according to the position of the unmanned aerial vehicle and the straight line where the blade is located, the shooting angle of the pan-tilt camera is adjusted according to the detection target space coordinates, the detection target image coordinates are determined, the target circular region is determined in the image data captured by the pan-tilt camera according to the detection target image coordinates, and the exposure value of the pan-tilt camera is adjusted according to the average grayscale value of the target circular region, thereby improving the quality of the captured image of the wind turbine blade.

[0135] The blade exposure adjustment algorithm based on projection proposed by the present invention aims to enhance the image quality by dynamically adjusting the exposure value during the image capture process. It locates the blade area through projection, optimizes the brightness within this area, and preserves key details.

[0136] As Figure 5 shown is a schematic diagram of the spatial position relationship between the unmanned aerial vehicle and the blade provided by the embodiment of the present invention. In Figure 5 during the detection flight, the present invention uses lidar to capture the point cloud of the blade In view of the unique shape of the blade, the present invention applies the random sample consensus algorithm to perform straight line fitting and obtain the blade straight line The position of the unmanned aerial vehicle is represented as p d . Establish a straight line l t , as the perpendicular line from p d to , where w p f is the intersection point, satisfying the following formula (6):

[0137]

[0138] The direction vector of l t is represented by v t . By designating w p f as the detection target, the present invention adjusts the pan-tilt to align with the direction of v t . This alignment ensures that the detection target always remains within the captured image.

[0139] According to the perspective projection principle, i p f is deduced to be w p f represented in the image frame, referring to Equation (7) below:

[0140]

[0141] where and respectively represent the detection targets in the image frame and the world frame. represents the internal matrix of the gimbal camera, which can be obtained from DJI documentation. is the external matrix, representing the transformation from the world frame to the camera frame, referring to Equation (8) below:

[0142]

[0143] where c R b and c t b respectively represent the rotation and translation between the camera frame and the body frame. b R w and b t w are respectively determined by the attitude and position of the drone.

[0144] The present invention converts the RGB image captured by the gimbal camera into a grayscale image, denoted as G. A circular region C i p f centered at f is established, and the grayscale values within this region are analyzed. The definition of C f is as follows in Equation (9):

[0145] C f ={ i p| i p∈G,|| i p - i p f ||≤r f} (9)

[0146] where r f represents the radius of C f . Subsequently, the average grayscale value g within C f is calculated as shown in Equation (10) below:

[0147]

[0148] where |C f | represents the number of pixels within C f .

[0149] Finally, the present invention uses g to evaluate the brightness level of the blade area. To ensure that g remains within the appropriate range [g min , g max , the present invention adjusts the current exposure value u to v by means of a step factor k g . Subsequently, the camera exposure value is set to the adjusted v.

[0150] Further as an optional implementation manner, the threshold range includes an upper threshold and a lower threshold, and the exposure value of the pan-tilt camera is adjusted according to the average gray value, which specifically includes:

[0151] S2061. When the average gray value is greater than the upper threshold, the exposure value is decreased according to a preset step factor;

[0152] S2062. When the average gray value is less than the upper threshold, the exposure value is increased according to the step factor.

[0153] The algorithm flow of the blade exposure adjustment algorithm based on projection in the embodiment of the present invention is as follows:

[0154]

[0155] The method steps of the embodiment of the present invention are described above. It can be recognized that in the embodiment of the present invention, the blade position and the hub position are estimated according to the point cloud data of the blade, the blade angle is determined according to the blade position and the hub position, the orbital operation trajectory of the unmanned aerial vehicle is updated according to the hub position, so as to obtain the point cloud data again for the estimation of the blade position and the hub position until the blade angle obtained currently and the blade angle obtained in the previous round meet the preset convergence condition, and the final blade angle is obtained, improving the accuracy of the blade angle detection of the wind turbine; in addition, in the embodiment of the present invention, the straight line where the blade is located is fitted according to the point cloud data of the blade, the detection target space coordinates are determined according to the position of the unmanned aerial vehicle and the straight line where the blade is located, the shooting angle of the pan-tilt camera is adjusted according to the detection target space coordinates and the detection target image coordinates are determined, the target circular area is determined in the image data captured by the pan-tilt camera according to the detection target image coordinates, and the exposure value of the pan-tilt camera is adjusted according to the average gray value of the target circular area, thereby improving the quality of the image of the wind turbine blade captured.

[0156] The following further describes the embodiment of the present invention in combination with the test experiment of an operating wind farm.

[0157] The present invention has been experimentally tested in operating wind farms. So far, the platform and algorithms proposed by the present invention have been repeatedly tested on five different models of wind turbines in three wind farms. The UAV detection platform has completed more than 100 flights. The blade angle detection algorithm has been tested under more than 80 different blade angles. The blade exposure adjustment algorithm has been evaluated under more than 60 different lighting conditions.

[0158] Extensive experiments have confirmed the stability and reliability of the platform of the present invention, and confirmed its compatibility with the sensor requirements of the algorithms. The platform has proven its ability to perform detections under challenging conditions, such as operating at an altitude of 1000 meters and in strong winds of 8 m / s. Next, the present invention will present the results of these two algorithms.

[0159] The present invention compares the experimental results of the blade angle (Our method) with the vision-based method in the prior art. Stokkeland et al. proposed a vision-based method that uses the Hough line transform to detect lines and a voting algorithm to identify blade lines. The present invention adjusted this method to adapt to the detection distance of the present invention while retaining its key voting algorithm. The average angle error (MeanAngle Error) and success rate of the two methods are compared as shown in Table 1 below.

[0160] Table 1

[0161]

[0162] It can be seen that the method of the present invention exhibits an average angle error of 1.15 degrees, exceeding the 2.14-degree error of the vision-based method. In addition, the method of the present invention has a success rate of 98.6%, significantly superior to the 69.2% success rate of the vision-based method. These findings highlight the excellent robustness and accuracy of the method proposed by the present invention compared to the vision-based method.

[0163] The platform proposed by the present invention performs inspections along the wind turbine blade, capturing images at set time intervals from the blade root to the tip. The average value μ, standard deviation σ, and entropy H of the gray values within the blade region are used as indicators to evaluate the method of the present invention. Specifically, μ reflects the brightness, and σ and H represent the complexity of the details. Generally, there is underexposure on the lower blade surface, while overexposure is common on the upper surface. The present invention conducts experiments to evaluate these situations. The means (Mean), standard deviations (Std Dev), and entropies (Entropy) of the finally obtained original images and adjusted images in underexposure and overexposure scenarios are shown in Table 2 below.

[0164] Table 2

[0165]

[0166] It can be seen that in the case of underexposure, the blade brightness increases from 25.48 to 133.27, and the richness of blade details quantified by the standard deviation and entropy increases by 218.85% and 56.19% respectively. On the contrary, in the case of overexposure, the blade brightness is adjusted from 245.64 to 150.85. The richness of blade details measured by the standard deviation and entropy increases by 187.73% and 40.65% respectively. This highlights the robustness and versatility of the method of the present invention in underexposure and overexposure scenarios. The method of the present invention ensures that the brightness falls within an optimal range, effectively retaining the complex details within the blade region.

[0167] Referring to Figure 6 , an embodiment of the present invention provides a wind turbine blade detection system, including:

[0168] A point cloud data acquisition module, configured to acquire first point cloud data of a target blade through a lidar carried by a drone;

[0169] A point cloud data processing module, configured to perform plane fitting and grid map conversion on the first point cloud data to obtain a fitted plane and a grid map;

[0170] A scanning and querying module, configured to scan and query the grid map to obtain multiple blade occupancy points, and cluster the blade occupancy points to obtain the blade position;

[0171] A blade angle determination module, configured to determine the hub position according to the blade position, and obtain the blade angle according to the hub position and the blade position;

[0172] The UAV trajectory update module is used to update the orbital motion trajectory of the UAV according to the fitted plane and the hub position, and return the step of obtaining the first point cloud data of the target blade through the lidar carried by the UAV, until the blade angle obtained currently and the blade angle obtained in the previous round meet the preset convergence condition, and output the currently obtained blade angle as the target blade angle.

[0173] The content in the above method embodiments is applicable to the system embodiments. The functions specifically implemented by the system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those of the above method embodiments.

[0174] Referring to Figure 7 , an embodiment of the present invention provides a wind turbine blade detection device, including:

[0175] At least one processor;

[0176] At least one memory for storing at least one program;

[0177] When the above at least one program is executed by the above at least one processor, the above at least one processor implements the above-mentioned wind turbine blade detection method.

[0178] The content in the above method embodiments is applicable to the device embodiments. The functions specifically implemented by the device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those of the above method embodiments.

[0179] An embodiment of the present invention also provides a computer-readable storage medium, in which a program executable by a processor is stored. The program executable by the processor is used to execute the above-mentioned wind turbine blade detection method when executed by the processor.

[0180] A computer-readable storage medium of an embodiment of the present invention can execute a wind turbine blade detection method provided by an embodiment of the method of the present invention, can execute any combination of implementation steps of the method embodiment, and has the corresponding functions and beneficial effects of the method.

[0181] An embodiment of the present invention also discloses a computer program product or a computer program. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes Figure 1 the method shown.

[0182] In some alternative embodiments, the functions / operations recited in the block diagrams may not occur in the order presented in the operational illustrations. For example, depending on the functions / operations involved, two blocks shown in succession may actually be executed substantially simultaneously or the blocks may sometimes be executed in the reverse order. Further, the embodiments presented and described in the flowcharts of the present invention are provided by way of example in order to provide a more thorough understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are envisioned in which the order of various operations is altered and in which sub-operations described as part of a larger operation are performed independently.

[0183] Moreover, although the present invention has been described in the context of functional modules, it should be understood that, unless otherwise stated to the contrary, one or more of the above-described functions and / or features may be integrated in a single physical device and / or software module, or one or more functions and / or features may be implemented in separate physical devices or software modules. It should also be understood that a detailed discussion of the actual implementation of each module is not necessary for an understanding of the present invention. Rather, the actual implementation of the module will be understood within the ordinary skill of an engineer, given the attributes, functions, and internal relationships of the various functional modules in the devices disclosed herein. Thus, those skilled in the art can implement the present invention as set forth in the claims without undue experimentation. It should also be understood that the particular concepts disclosed are illustrative only and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0184] If the above functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or a part of this technical solution, can be embodied in the form of a software product stored in a storage medium, including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the above methods in various embodiments of the present invention. The foregoing storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs.

[0185] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a definable sequence list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other systems that can fetch and execute instructions from the instruction execution system, apparatus, or device), or used in combination with these instruction execution systems, apparatuses, or devices. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

[0186] More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection part (electronic device) having one or more wirings, a portable computer diskette (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can even be paper or other suitable media on which the above program can be printed, because the above program can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpretation, or other appropriate processing as necessary, and then stored in a computer memory.

[0187] It should be understood that various parts of the present invention can be implemented by hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits having logic gate circuits for implementing logical functions on data signals, application specific integrated circuits having appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.

[0188] In the above description of this specification, the descriptions referring to the terms "one embodiment / example", "another embodiment / example", or "certain embodiments / examples", etc. mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0189] Although embodiments of the present invention have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the claims and their equivalents.

[0190] The above is a specific description of the preferred embodiments of the present invention, but the present invention is not limited to the above embodiments. Those skilled in the art can also make various equivalent deformations or substitutions without departing from the spirit of the present invention, and these equivalent deformations or substitutions are all included within the scope defined by the claims of this application.

Claims

1. A method for detecting a wind turbine blade, characterized in that, Including the following steps: Obtain the first point cloud data of the target blade through the lidar carried by the drone; Perform plane fitting and grid map conversion on the first point cloud data to obtain a fitted plane and a grid map; Perform a scan query on the grid map to obtain multiple blade occupancy points, and cluster the blade occupancy points to obtain the blade position; Determine the hub position according to the blade position, and obtain the blade angle according to the hub position and the blade position; Update the orbital motion trajectory of the drone according to the fitted plane and the hub position, and return to the step of obtaining the first point cloud data of the target blade through the lidar carried by the drone until the blade angle obtained currently and the blade angle obtained in the previous round meet the preset convergence condition, and output the currently obtained blade angle as the target blade angle; The drone is also equipped with a pan-tilt camera, and the wind turbine blade detection method further includes the following steps: Obtain the second point cloud data of the target blade through the lidar; Perform line fitting on the second point cloud data to obtain the line where the blade is located, and determine the detection target space coordinates according to the drone position and the line where the blade is located; Adjust the shooting angle of the pan-tilt camera according to the detection target space coordinates, and determine the corresponding detection target image coordinates according to the detection target space coordinates; Set the exposure value of the pan-tilt camera, and obtain the image data of the target blade through the pan-tilt camera; Convert the image data into a grayscale image, and determine the target circular area in the grayscale image according to the detection target image coordinates; Determine the average grayscale value of the target circular area, adjust the exposure value of the pan-tilt camera according to the average grayscale value, and return to the step of obtaining the image data of the target blade through the pan-tilt camera until the average grayscale value meets the preset threshold range, and output the currently captured image data as the target blade image data; 2. A method for detecting a wind turbine blade according to claim 1, wherein, The performing plane fitting and grid map conversion on the first point cloud data to obtain a fitted plane and a grid map specifically includes: Perform plane fitting on the first point cloud data through the random sample consensus algorithm to obtain the fitted plane and the corresponding unit normal vector; Determine the map resolution according to the size of the wind turbine, and perform grid map conversion on the first point cloud data according to the map resolution to obtain the grid map; 3. A method for detecting a wind turbine blade according to claim 2, characterized in that, The performing a scan query on the grid map to obtain multiple blade occupancy points, and clustering the blade occupancy points to obtain the blade position specifically includes: Determine the vertical scan direction according to the unit normal vector, and perform a vertical scan on the grid map according to the vertical scan direction to obtain multiple first sampling points; Determine a circular scan trajectory on the fitted plane according to the first sampling points, and perform a circular scan on the grid map according to the circular scan trajectory to obtain multiple second sampling points; Query the grid status corresponding to the second sampling points in the grid map, and when the grid status is occupied, determine the second sampling points as the blade occupancy points; Cluster the blade occupancy points to obtain three sets of blade occupancy points; Determine the corresponding blade positions according to the average positions of the blade occupancy points in each set of blade occupancy points.

4. A method for detecting a wind turbine blade according to claim 3, characterized in that, Determine the hub position according to the blade positions, and obtain the blade angles according to the hub position and the blade positions. Specifically, it includes: Construct a target triangle based on the three blade positions, and use the Fermat point of the target triangle as the hub position; Determine the corresponding blade direction vector according to the directed connection line from the hub position to the blade position; Determine the corresponding blade angle according to the blade direction vector.

5. A method for detecting a wind turbine blade according to claim 1, characterized in that, Update the orbital motion trajectory of the drone according to the fitted plane and the hub position. Specifically, it includes: Determine the orbital plane according to the fitted plane, such that the orbital plane is parallel to the fitted plane; Take the projection position of the hub position on the orbital plane as the orbital center position; Determine the updated orbital motion trajectory on the orbital plane according to a preset orbital radius and the orbital center position.

6. A method for detecting a wind turbine blade according to claim 1, characterized in that, The threshold range includes an upper threshold and a lower threshold. Adjust the exposure value of the pan-tilt camera according to the average gray value. Specifically, it includes: When the average gray value is greater than the upper threshold, decrease the exposure value according to a preset step factor; When the average gray value is less than the upper threshold, increase the exposure value according to the step factor.

7. A wind turbine blade detection system, characterized in that, It includes: A first point cloud data acquisition module, configured to acquire first point cloud data of a target blade through a lidar carried by the drone; A first point cloud data processing module, configured to perform plane fitting and grid map conversion on the first point cloud data to obtain a fitted plane and a grid map; A scanning and querying module, configured to scan and query the grid map to obtain a plurality of blade occupancy points, and cluster the blade occupancy points to obtain blade positions; A blade angle determination module, configured to determine the hub position according to the blade positions, and obtain the blade angles according to the hub position and the blade positions; A drone trajectory update module, configured to update the orbital motion trajectory of the drone according to the fitted plane and the hub position, and return to the step of acquiring the first point cloud data of the target blade through the lidar carried by the drone, until the blade angles obtained currently and the blade angles obtained in the previous round meet a preset convergence condition, and output the currently obtained blade angle as the target blade angle; The drone is also equipped with a pan-tilt camera. The wind turbine blade detection system further includes: A second point cloud data acquisition module, configured to acquire second point cloud data of the target blade through the lidar; A second point cloud data processing module, configured to perform line fitting on the second point cloud data to obtain the line where the blade is located, and determine the detection target space coordinates according to the drone position and the line where the blade is located; A shooting angle adjustment module, configured to adjust the shooting angle of the pan-tilt camera according to the detection target space coordinates, and determine the corresponding detection target image coordinates according to the detection target space coordinates; The blade image acquisition module is configured to set the exposure value of the pan-tilt camera and acquire the image data of the target blade through the pan-tilt camera; The image conversion module is configured to convert the image data into a grayscale image and determine a target circular region in the grayscale image according to the detected target image coordinates; The exposure value update module is configured to determine the average grayscale value of the target circular region, adjust the exposure value of the pan-tilt camera according to the average grayscale value, and return to the step of acquiring the image data of the target blade through the pan-tilt camera until the average grayscale value meets the preset threshold range, and output the currently captured image data as the target blade image data.

8. A wind turbine blade detection device, characterized in that, Comprising: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements a wind turbine blade detection method according to any one of claims 1 to 6.

9. A computer-readable storage medium storing a program executable by a processor, characterized in that, The program executable by the processor, when executed by the processor, is used to execute a wind turbine blade detection method according to any one of claims 1 to 6.

Images