Wind driven generator blade detection method, system and device and storage medium

Abstract

The invention discloses a wind driven generator blade detection method, system and device and a storage medium. First point cloud data of a target blade are acquired through a laser radar carried by an unmanned aerial vehicle; performing plane fitting and grid map conversion on the first point cloud data to obtain a fitting plane and a grid map; scanning and querying the grid map to obtain a plurality of blade occupation points, and clustering the blade occupation points to obtain blade positions; determining a hub position according to the blade position, and obtaining a blade angle according to the hub position and the blade position; and updating the orbital motion trajectory of the unmanned aerial vehicle according to the fitting plane and the hub position, and returning to the step of obtaining the first point cloud data of the target blade through the laser radar carried by the unmanned aerial vehicle until the currently obtained blade angle and the blade angle obtained in the last round meet a preset convergence condition. The method improves the accuracy of wind driven generator blade angle detection, also improves the quality of wind driven generator blade images, and can be applied to the technical field of machine vision.

Description

Technical Field

[0001] The present invention relates to the field of machine vision technology, and in particular to a method, system, device and storage medium for detecting blades of a wind turbine generator. Background Art

[0002] In recent years, driven by the continuous advancement of renewable energy technology, wind energy has become an important source of electricity. As of 2022, the installed wind power capacity in the world has surged to an impressive 906 GW. Wind farms are strategically located in hilly, mountainous and coastal areas, taking advantage of the abundant wind resources to provide considerable energy potential for efficient wind turbine installations. Despite these advances, severe weather conditions such as strong winds, lightning, hail and rain pose a threat to wind turbine blades, which can cause damage. Regular inspection of these blades is essential to detect and correct problems in time, thereby avoiding interference with the normal operation of the generator.

[0003] Traditional methods of inspecting wind turbine blades have relied on manual techniques involving the use of ground-based telescopes or personnel suspended from the generator using safety ropes. However, these methods are time-consuming, labor-intensive, inefficient, and present safety risks. The introduction of drone technology has revolutionized these inspection methods, enabling skilled remote pilots to efficiently conduct comprehensive assessments. But this requires a proficient grasp of both flying and photography. The emergence of drone-based automated inspection systems takes full advantage of automation and artificial intelligence to autonomously guide the operation of drones during flight and image capture. These technological advances can not only reduce labor costs, but also improve inspection efficiency and image quality.

[0004] Given the potential benefits of drone-based automated wind turbine blade inspection in maintenance, the field faces significant challenges. Although researchers have proposed various approaches to address these issues, several unresolved issues remain:

[0005] 1) Limited supply of advanced drone equipment: Commercial drones that integrate high-definition gimbal cameras, high-precision lidar, and powerful computing units are scarce. While custom platforms exist, they often lack the stability and safety that are critical for effective wind turbine inspections.

[0006] 2) Constraints on blade angle detection: Existing detection methods are often restricted to specific blade angles, limiting their versatility and adaptability. Some methods require preset blade angles, hindering automation. Vision-based blade angle detection methods are susceptible to interference, affecting accuracy and robustness.

[0007] 3) Lack of automatic exposure adjustment: Exposure adjustment techniques employed by commercial pan-tilt cameras typically rely on overall image brightness, limiting their utility in specific inspection scenarios. Existing inspection methods often overlook the importance of adjusting exposure to varying lighting conditions during image capture. This oversight can result in under- or over-exposure of leaf areas in images, especially in challenging lighting conditions, thereby compromising visibility of critical details and hindering subsequent damage assessment.

[0008] The above problems need to be solved urgently. Summary of the invention

[0009] The purpose of the present invention is to solve one of the technical problems existing in the prior art to at least a certain extent.

[0010] Therefore, an object of an embodiment of the present invention is to provide a method for detecting blades of a wind turbine generator, which improves the accuracy of detecting the angle of the blades of the wind turbine generator and also improves the quality of the image of the blades of the wind turbine generator.

[0011] Another object of an embodiment of the present invention is to provide a wind turbine blade detection system.

[0012] In order to achieve the above technical objectives, the technical solutions adopted by the embodiments of the present invention include:

[0013] In a first aspect, an embodiment of the present invention provides a method for detecting blades of a wind turbine generator, comprising the following steps:

[0014] The first point cloud data of the target blade is obtained through the laser radar carried by the drone;

[0015] Performing plane fitting and grid map conversion on the first point cloud data to obtain a fitting plane and a grid map;

[0016] Scan and query the grid map to obtain a plurality of leaf occupation points, and cluster the leaf occupation points to obtain leaf positions;

[0017] Determine the hub position according to the blade position, and obtain the blade angle according to the hub position and the blade position;

[0018] The orbital motion trajectory of the UAV is updated according to the fitting plane and the hub position, and the step of obtaining the first point cloud data of the target blade through the lidar carried by the UAV is returned to, until the current blade angle and the blade angle obtained in the previous round meet the preset convergence conditions, and the current blade angle is output as the target blade angle.

[0019] Further, in one embodiment of the present invention, performing plane fitting and grid map conversion on the first point cloud data to obtain a fitting plane and a grid map specifically includes:

[0020] Performing plane fitting on the first point cloud data by using a random sampling consensus algorithm to obtain the fitting plane and the corresponding unit normal vector;

[0021] The map resolution is determined according to the size of the wind turbine, and the first point cloud data is converted into a grid map according to the map resolution to obtain the grid map.

[0022] Further, in one embodiment of the present invention, scanning and querying the grid map to obtain a plurality of leaf occupation points, and clustering the leaf occupation points to obtain leaf positions specifically includes:

[0023] Determine a vertical scanning direction according to the unit normal vector, and vertically scan the grid map according to the vertical scanning direction to obtain a plurality of first sampling points;

[0024] Determine a circular scanning trajectory on the fitting plane according to the first sampling points, and perform a circular scanning on the grid map according to the circular scanning trajectory to obtain a plurality of second sampling points;

[0025] querying a grid state corresponding to the second sampling point in the grid map, and when the grid state is occupied, determining that the second sampling point is the leaf occupied point;

[0026] Clustering the leaf occupied points to obtain three leaf occupied point sets;

[0027] The corresponding leaf position is determined according to the average position of the leaf occupied points in each of the leaf occupied point sets.

[0028] Further, in one embodiment of the present invention, determining the hub position according to the blade position, and obtaining the blade angle according to the hub position and the blade position specifically includes:

[0029] Construct a target triangle according to the three blade positions, and use the Fermat point of the target triangle as the hub position;

[0030] Determine a corresponding blade direction vector according to a directed connection line from the hub position to the blade position;

[0031] The corresponding blade angle is determined according to the blade direction vector.

[0032] Further, in one embodiment of the present invention, the updating of the orbital motion trajectory of the UAV according to the fitting plane and the hub position specifically includes:

[0033] Determine the orbital plane according to the fitting plane, so that the orbital plane is parallel to the fitting plane;

[0034] Taking the projection position of the hub position on the track plane as the track center position;

[0035] The updated orbital motion trajectory is determined on the orbital plane according to the preset orbital radius and the orbital center position.

[0036] Furthermore, in one embodiment of the present invention, the drone is also equipped with a pan-tilt camera, and the wind turbine blade detection method further includes the following steps:

[0037] Acquire second point cloud data of the target blade by using the laser radar;

[0038] Performing straight line fitting on the second point cloud data to obtain a straight line where the blade is located, and determining the detection target spatial coordinates according to the position of the UAV and the straight line where the blade is located;

[0039] Adjusting the shooting angle of the pan / tilt camera according to the detection target space coordinates, and determining the corresponding detection target image coordinates according to the detection target space coordinates;

[0040] Setting the exposure value of the pan-tilt camera, and acquiring the image data of the target blade through the pan-tilt camera;

[0041] Converting the image data into a grayscale image, and determining a target circular area in the grayscale image according to the detection target image coordinates;

[0042] Determine the average grayscale value of the target circular area, adjust the exposure value of the gimbal camera according to the average grayscale value, and return to the step of obtaining the image data of the target leaf through the gimbal camera until the average grayscale value meets the preset threshold range, and output the image data currently captured as the target leaf image data.

[0043] Further, in one embodiment of the present invention, the threshold range includes an upper threshold limit and a lower threshold limit, and adjusting the exposure value of the gimbal camera according to the average grayscale value specifically includes:

[0044] When the average grayscale value is greater than the upper threshold value, reducing the exposure value according to a preset step factor;

[0045] When the average grayscale value is less than the upper threshold value, the exposure value is increased according to the step factor.

[0046] In a second aspect, an embodiment of the present invention provides a wind turbine blade detection system, comprising:

[0047] A point cloud data acquisition module, used to acquire first point cloud data of a target blade through a laser radar carried by a UAV;

[0048] A point cloud data processing module, used for performing plane fitting and grid map conversion on the first point cloud data to obtain a fitting plane and a grid map;

[0049] A scanning query module, used for scanning and querying the grid map to obtain a plurality of leaf occupation points, and clustering the leaf occupation points to obtain leaf positions;

[0050] A blade angle determination module, used to determine the hub position according to the blade position, and obtain the blade angle according to the hub position and the blade position;

[0051] The UAV trajectory update module is used to update the orbital motion trajectory of the UAV according to the fitting plane and the hub position, and return to the step of obtaining the first point cloud data of the target blade through the laser radar carried by the UAV, until the current blade angle and the blade angle obtained in the previous round meet the preset convergence conditions, and the current blade angle is output as the target blade angle.

[0052] In a third aspect, an embodiment of the present invention provides a wind turbine blade detection device, comprising:

[0053] at least one processor;

[0054] at least one memory for storing at least one program;

[0055] When the at least one program is executed by the at least one processor, the at least one processor implements the above-mentioned wind turbine blade detection method.

[0056] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, in which a program executable by a processor is stored. When the program executable by the processor is executed by the processor, it is used to execute the above-mentioned method for detecting blades of a wind turbine.

[0057] The advantages and beneficial effects of the present invention will be partly given in the following description, partly become apparent from the following description, or be understood through the practice of the present invention:

[0058] The embodiment of the present invention obtains first point cloud data of a target blade through a laser radar carried by an unmanned aerial vehicle, performs plane fitting and grid map conversion on the first point cloud data to obtain a fitting plane and a grid map, scans and queries the grid map to obtain multiple blade occupation points, clusters the blade occupation points to obtain the blade position, determines the hub position according to the blade position, obtains the blade angle according to the hub position and the blade position, updates the orbital motion trajectory of the unmanned aerial vehicle according to the fitting plane and the hub position, and returns to the step of obtaining the first point cloud data of the target blade through the laser radar carried by the unmanned aerial vehicle, until the currently obtained blade angle and the blade angle obtained in the previous round meet the preset convergence conditions, and the currently obtained blade angle is output as the target blade angle. The embodiment of the present invention estimates the blade position and the hub position according to the point cloud data of the blade, determines the blade angle according to the blade position and the hub position, and updates the orbit trajectory of the drone according to the hub position, so as to obtain the point cloud data again for estimating the blade position and the hub position, until the current blade angle and the blade angle obtained in the previous round meet the preset convergence conditions, and the final blade angle is obtained, thereby improving the accuracy of wind turbine blade angle detection; in addition, the embodiment of the present invention obtains the straight line where the blade is located according to the point cloud data of the blade, determines the detection target spatial coordinates according to the drone position and the straight line where the blade is located, adjusts the shooting angle of the gimbal camera according to the detection target spatial coordinates and determines the detection target image coordinates, determines the target circular area in the image data captured by the gimbal camera according to the detection target image coordinates, and adjusts the exposure value of the gimbal camera according to the average grayscale value of the target circular area, thereby improving the quality of the captured wind turbine blade image. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the technical solution in the embodiments of the present invention, the following introduction is made to the drawings required for use in the embodiments of the present invention. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solution of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0060] Figure 1 A flow chart of the steps of a wind turbine blade detection method provided by an embodiment of the present invention;

[0061] Figure 2 A schematic diagram of a drone detection platform and a corresponding spatial coordinate system provided by an embodiment of the present invention;

[0062] Figure 3 A side view of the laser radar position arrangement provided by an embodiment of the present invention;

[0063] Figure 4 A top view of the laser radar position arrangement provided by an embodiment of the present invention;

[0064] Figure 5 A schematic diagram of the spatial position relationship between the drone and the blade provided in an embodiment of the present invention;

[0065] Figure 6 A structural block diagram of a wind turbine blade detection system provided by an embodiment of the present invention;

[0066] Figure 7 A structural block diagram of a wind turbine blade detection device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0067] The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and are not to be construed as limitations of the present invention. For the step numbers in the following embodiments, they are only provided for the convenience of explanation, and the order between the steps is not limited in any way, and the execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.

[0068] In the description of the present invention, the meaning of "a plurality" is two or more than two. If there is a description of "a first" or "a second", it is only used for the purpose of distinguishing technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features. In addition, unless otherwise defined, all technical and scientific terms used in this document have the same meaning as those commonly understood by those skilled in the art.

[0069] Unmanned aerial vehicles (UAVs) are crucial in wind turbine blade inspection. However, there are still some unresolved issues in the field of UAV-based autonomous wind turbine blade inspection. First, autonomous inspection tasks require long-range perception capabilities and high-performance computing capabilities. Unfortunately, typical commercial UAVs are limited by limited perception distance and computing resources. Second, existing inspection methods rely on prior knowledge of blade angles or restrict inspection to specific blade angles. This limits the versatility of this application in different scenarios. Third, the influence of sunlight during shooting may cause uneven exposure, resulting in abnormal brightness and loss of details on the blades. This poses significant challenges for subsequent processing.

[0070] Conducting inspection experiments within operational wind farms faces challenges with staff deployment and potential power generation losses. Sch{"a}fer et al. conducted experiments in a simulated environment using a Pelican drone equipped with a Hokuyo UTM-30LX lidar. However, bridging the gap between simulation and reality remains a daunting challenge. Parlange et al. conducted experiments on a small outdoor wind turbine model using a Parrot Bebop 2 drone equipped with a GoPro camera. However, the differences between the small-scale model and the actual wind turbine limit the practical applicability of the proposed algorithm.

[0071] A stable and reliable drone platform is fundamental to an effective inspection program within an operating wind farm. Complex automated inspection algorithms rely on remote perception capabilities and real-time decision making, which presents significant challenges in platform design. Car et al. introduced a custom drone equipped with a Velodyne VLP-16 lidar. However, custom-designed platforms may lack the necessary reliability and stability. In contrast, Castelar Wembers et al. equipped a commercial DJI Matrice300RTK drone with a DJI Zenmuse H20T gimbal camera and a Hokuyo UST-20LX lidar. The commercial DJI platform demonstrated impressive stability.

[0072] During blade angle detection, wind turbine blades must be fixed at a specific angle. Normally, these blades rotate passively in response to wind force. Some wind turbines can only be fixed at specific angles (such as a positive Y-shape or an inverted Y-shape), which requires a time-consuming locking procedure that depends on wind conditions. The method proposed by Castelar Wembers et al. is designed for blades in an inverted Y-shape setting, which requires precise blade angles. et al.’s method involves presetting the blade angle and requires manual intervention.

[0073] Some modern wind turbines offer the flexibility to be fixed at various angles, simplifying the locking process but potentially leaving the blades fixed at arbitrary angles. In this case, automated blade angle detection is essential for detection automation. Visual methods are often used for this purpose. Stokkeland et al. and Parlange et al. applied traditional visual techniques using Hough transform to detect blade lines and determine blade angles. Guo et al. introduced a deep learning-based visual method to determine blade angles by identifying blade tips. However, vision-based methods are susceptible to background interference and may lead to instability. Currently, there is a lack of research on blade angle detection using point clouds or raster maps.

[0074] Wind turbine blade images captured by drones are critical for subsequent damage detection. During the image capture process, changes in sunlight and camera angles may cause significant brightness differences between the blades and background areas. However, currently popular commercial gimbal cameras focus more on overall image exposure rather than focusing on specific areas of interest. This approach may result in over- or under-exposure of blade areas, which may obscure critical details and complicate subsequent damage detection. Implementing real-time automatic exposure adjustment during image capture is expected to optimize uneven brightness and alleviate the challenges of subsequent tasks. To the best of the knowledge of the present invention, there is no research on automatically adjusting exposure during image capture in the field of wind turbine blade inspection.

[0075] To address these challenges, the present invention proposes an automated UAV-based wind turbine blade inspection method, which includes three key components. First, the present invention proposes a detection platform equipped with a long-range lidar and an onboard computer, which is designed to meet the needs of multi-sensor integration and high computing power to achieve automated inspection. Secondly, the present invention introduces a grid map-based blade angle detection algorithm, which achieves superior accuracy in detecting arbitrary angles of the blade based on the grid map generated by the high-precision lidar. Finally, the present invention proposes a projection-based blade exposure adjustment algorithm to ensure that the brightness and details of the blade surface remain optimal under different lighting conditions during image capture. Extensive testing in operating wind farms has demonstrated the robustness and reliability of the platform and algorithms of the present invention, confirming their effectiveness in actual inspection scenarios.

[0076] First, the UAV detection platform proposed by the present invention is introduced. Figure 2 FIG. 1 is a schematic diagram of a drone detection platform and a corresponding spatial coordinate system provided by an embodiment of the present invention. The drone detection platform of the present invention includes a drone body and a pan / tilt camera 1, a laser radar 2, and an onboard computer 3 arranged on the drone body. The spatial coordinate system of the drone body is b -x b y b z b , the spatial coordinate system of the gimbal camera is O c -x c y c z c , the spatial coordinate system of the laser radar is O l -x l y l z l. For the drone, the present invention selected the DJI Matrice 300RTK for its stability and safety in industrial applications. It also has an Onboard SDK to simplify the secondary development process. To capture high-quality inspection images, the present invention selected the DJI Zenmuse H20T, a gimbal camera with a three-axis gimbal and a three-lens system. As a lidar function, the present invention selected the Livox MID-360 to obtain long-distance, high-precision point cloud data. Finally, as an onboard computer, the present invention selected NUC 11Pro Kit NUC11TNKi5, which provides sufficient computing power, compact size, convenient installation and multiple interfaces to facilitate data exchange between various hardware elements.

[0077] exist Figure 2 The placement of the components within the platform of the present invention is also shown: the gimbal camera is installed at the front and bottom of the drone body, while the lidar and onboard computer are located at the top. This configuration is carefully designed to avoid interfering with the normal flight and battery loading of the drone. In addition, the lidar is located at the front of the top and the onboard computer is located behind it. In order to establish a connection between the onboard computer and the drone, the present invention uses the DJIOSDK expansion module. The drone provides power to the onboard computer and lidar through the voltage regulation module. The data cable is used to ensure smooth communication between these components. Finally, the present invention adds a protective shell to prevent the intrusion of outdoor dust and protect the computer and hardware interfaces.

[0078] like Figure 3 FIG. 1 is a side view of the laser radar position arrangement provided by an embodiment of the present invention, as shown in FIG. Figure 4 FIG. 1 is a top view of the laser radar position arrangement provided by an embodiment of the present invention. Figure 3 and 4 The placement of the LiDAR is shown in FIG. When the LiDAR is positioned horizontally, its vertical field of view (FOV) ranges from -7 degrees to 52 degrees. To ensure symmetry when the forward vertical FOV is φ = 60°, the pitch angle of the LiDAR is set to α = -23°. In the horizontal dimension, the focus of the present invention is on the point cloud of the front area, so the horizontal FOV is set to θ = 60°.

[0079] The following is a detailed description of a blade angle detection algorithm based on a grid map and a blade exposure adjustment algorithm based on projection according to an embodiment of the present invention.

[0080] Reference Figure 1 The embodiment of the present invention provides a method for detecting blades of a wind turbine generator, which specifically includes the following steps:

[0081] S101, obtaining first point cloud data of the target blade by using a laser radar carried by the UAV;

[0082] S102, performing plane fitting and grid map conversion on the first point cloud data to obtain a fitting plane and a grid map;

[0083] S103, scanning and querying the grid map to obtain a plurality of leaf occupation points, and clustering the leaf occupation points to obtain leaf positions;

[0084] S104, determining the hub position according to the blade position, and obtaining the blade angle according to the hub position and the blade position;

[0085] S105. Update the orbital motion trajectory of the UAV according to the fitting plane and the hub position, and return to the step of obtaining the first point cloud data of the target blade through the laser radar carried by the UAV, until the current blade angle and the blade angle obtained in the previous round meet the preset convergence conditions, and output the current blade angle as the target blade angle.

[0086] The embodiment of the present invention estimates the blade position and the hub position based on the point cloud data of the blade, determines the blade angle based on the blade position and the hub position, and updates the orbital trajectory of the UAV based on the hub position, thereby acquiring point cloud data again for estimating the blade position and the hub position, until the current blade angle and the blade angle obtained in the previous round meet the preset convergence conditions, and the final blade angle is obtained, thereby improving the accuracy of wind turbine blade angle detection.

[0087] As an optional implementation, plane fitting and grid map conversion are performed on the first point cloud data to obtain a fitting plane and a grid map, which specifically includes:

[0088] S1021, performing plane fitting on the first point cloud data by using a random sampling consensus algorithm to obtain a fitting plane and a corresponding unit normal vector;

[0089] S1022: Determine a map resolution according to the size of the wind turbine, and perform raster map conversion on the first point cloud data according to the map resolution to obtain a raster map.

[0090] Specifically, plane fitting and grid map conversion are first performed on the acquired point cloud.

[0091] The drone hovers in front of the wind turbine, facing the main shaft. The point cloud of the main shaft and blade root is captured using lidar, represented as Considering the unique geometric shape of wind turbines, the present invention uses a random sampling consensus algorithm to Perform plane fitting to obtain the fitting plane and its unit normal vector v n .

[0092] The present invention starts the orbital motion of the drone in front of the wind turbine to expand the sensing range of the point cloud. The trajectory of this orbital motion is defined as C d :

[0093] C d ={p|p∈Π d ,‖pp c ‖ = r c |} (1)

[0094] Among them, d represents the orbital plane, so that r c represents the orbital radius. c ∈Π d represents the calculated orbital center, and the calculation formula is shown in (2):

[0095] p c =p h -v n ·d c (2)

[0096] Among them, p h represents the spindle position. In particular, p h There is a preset initial estimate, which is continuously optimized in subsequent processing. c Indicates the distance between the drone and the main axis, set to 20 meters.

[0097] Considering that point clouds are disordered and high-density, direct processing may result in significant computational overhead. Convert to a raster map To adapt to the size of the wind turbine, the present invention will The resolution is set to r M =1 meter.

[0098] As an optional implementation, the grid map is scanned and queried to obtain a plurality of leaf occupation points, and the leaf occupation points are clustered to obtain leaf positions, which specifically includes:

[0099] S1031, determining a vertical scanning direction according to a unit normal vector, and vertically scanning the grid map according to the vertical scanning direction to obtain a plurality of first sampling points;

[0100] S1032, determining a circular scanning trajectory on the fitting plane according to the first sampling point, and performing a circular scanning on the grid map according to the circular scanning trajectory to obtain a plurality of second sampling points;

[0101] S1033, querying the grid state corresponding to the second sampling point in the grid map, and when the grid state is occupied, determining that the second sampling point is a leaf occupied point;

[0102] S1034, clustering the leaf occupied points to obtain three leaf occupied point sets;

[0103] S1035. Determine the corresponding leaf position according to the average position of the leaf occupation points in each leaf occupation point set.

[0104] Specifically, leaves are identified through scanning and querying, followed by clustering to determine leaf locations.

[0105] The present invention utilizes both vertical scanning and circular scanning. It should be noted that the term "scanning" here refers to the algorithmic process, not the physical movement of the drone. During the entire scanning process, the drone maintains the aforementioned orbital motion.

[0106] In the vertical scan, along v n Scan, sampling point p with step size Δd s , thus forming a set Q s , which is defined as shown in the following formula (3):

[0107] Q s ={p s |p s =p c +v n d c +nΔd,n∈N,nΔd <d b} (3)

[0108] Among them, d c Indicates that from p c Distance to the initial position of the scan, d b Indicates the scanning range. The present invention is based on p s With v n Establish plane Π for normal vector s In Π d On, with p s Centered, r s Construct a circle for radius C s , as shown in the following formula (4). s Perform a circular scan on the

[0109] C s ={p|p∈Π d ,‖pp s ‖ = r s} (4)

[0110] In the circular scanning, the present invention uses a step size Δl to scan C s Upsampling point pr .exist In the query, the query contains p r If the result shows "occupied", the position is identified as part of the blade and is designated as the hit point. The hit points of all circular scans are then summarized into a set Q t , and then clustered into different groups. Successful clustering involves classification into 3 groups, each representing a leaf. If clustering fails, the circular scan is repeated at the next sampling point of the vertical scan until successful clustering is achieved. After clustering, the present invention calculates the average value of each group to obtain the estimated leaf position Q b ={p b1 ,p b2 ,p b3}.

[0111] As an optional implementation, the hub position is determined according to the blade position, and the blade angle is obtained according to the hub position and the blade position, which specifically includes:

[0112] S1041, constructing a target triangle according to the positions of the three blades, and using the Fermat point of the target triangle as the hub position;

[0113] S1042, determining a corresponding blade direction vector according to a directed connection line from the hub position to the blade position;

[0114] S1043. Determine a corresponding blade angle according to the blade direction vector.

[0115] As an optional implementation, updating the orbital motion trajectory of the drone according to the fitting plane and the hub position specifically includes:

[0116] S1051, determining the orbital plane according to the fitting plane, so that the orbital plane is parallel to the fitting plane;

[0117] S1052, taking the projection position of the hub position on the track plane as the track center position;

[0118] S1053: Determine an updated orbital motion trajectory on the orbital plane according to the preset orbital radius and orbital center position.

[0119] Specifically, the embodiment of the present invention optimizes the hub position using Fermat points and then calculates the blade angle.

[0120] Determine the hub position p using the Fermat point principle h Fermat points have a unique property when they are associated with triangles: for a triangle with three interior angles less than 120 degrees, the three line segments connecting its Fermat point and the three vertices have angles of 120 degrees between them.

[0121] Considering 120 degrees between turbine blades, using the obtained Q b A triangle △B is formed. In view of the geometric configuration of the turbine blades, all the internal angles of △B are less than 120 degrees. Therefore, the hub position of the wind turbine corresponds to the Fermat point of △B. The present invention can determine the Fermat point of △B as the hub position p h At the same time, this p h Will be used to re-center the orbital trajectory in the next iteration.

[0122] Connect h and Q b To determine the blade direction vector V b = {v b1 ,v b2 ,v b3}, and calculate the blade angle A b ={a b1 ,a b2 ,a b3}. When the current iteration obtains and the previous iteration When the following formula (5) is satisfied, the convergence criterion of the algorithm of the present invention is satisfied, thereby obtaining the optimal A b :

[0123]

[0124] where ε A is a preset threshold.

[0125] The algorithm flow of the blade angle detection algorithm based on the grid map of the embodiment of the present invention is as follows:

[0126]

[0127] As an optional implementation, the drone is also equipped with a gimbal camera, and the wind turbine blade detection method further includes the following steps:

[0128] S201, acquiring second point cloud data of the target blade by using a laser radar;

[0129] S202, performing straight line fitting on the second point cloud data to obtain the straight line where the blade is located, and determining the detection target spatial coordinates according to the position of the UAV and the straight line where the blade is located;

[0130] S203, adjusting the shooting angle of the pan / tilt camera according to the detection target space coordinates, and determining the corresponding detection target image coordinates according to the detection target space coordinates;

[0131] S204, setting the exposure value of the pan-tilt camera, and acquiring image data of the target leaf through the pan-tilt camera;

[0132] S205, converting the image data into a grayscale image, and determining a target circular area in the grayscale image according to the detection target image coordinates;

[0133] S206, determine the average grayscale value of the target circular area, adjust the exposure value of the gimbal camera according to the average grayscale value, and return to the step of obtaining the image data of the target leaf through the gimbal camera until the average grayscale value meets the preset threshold range, and output the image data currently captured as the target leaf image data.

[0134] Specifically, an embodiment of the present invention obtains a straight line where the blade is located according to the point cloud data of the blade, determines the spatial coordinates of the detection target according to the position of the drone and the straight line where the blade is located, adjusts the shooting angle of the gimbal camera according to the detection target spatial coordinates and determines the detection target image coordinates, determines the target circular area in the image data captured by the gimbal camera according to the detection target image coordinates, and adjusts the exposure value of the gimbal camera according to the average grayscale value of the target circular area, thereby improving the quality of the captured image of the wind turbine blade.

[0135] The projection-based leaf exposure adjustment algorithm proposed in this paper aims to enhance image quality by dynamically adjusting the exposure value during image capture. It locates the leaf area through projection, optimizes the brightness in the area, and retains key details.

[0136] like Figure 5 FIG. 1 is a schematic diagram showing the spatial position relationship between the drone and the blades provided in an embodiment of the present invention. Figure 5 In the detection flight process, the present invention uses a laser radar to capture the point cloud of the blade In view of the unique shape of the blade, the present invention applies a random sampling consensus algorithm to Perform straight line fitting and obtain the blade straight line The position of the drone is denoted as p d . Create a straight line l t , as from p d arrive The vertical line of w p f is the intersection point, satisfying the following formula (6):

[0137]

[0138] l t The direction vector is given by v t By w p f Assigned as the detection target, the present invention adjusts the pan / tilt to match v t This alignment ensures that the detection target always remains within the captured image.

[0139] According to the perspective projection principle, i p f It is derived as w p f The representation in the image frame is as follows:

[0140]

[0141] in and Represent the detected targets in the image frame and the world frame respectively. Represents the internal matrix of the gimbal camera, which can be obtained from DJI documentation. is the external matrix, which represents the transformation from the world frame to the camera frame, refer to the following formula (8):

[0142]

[0143] in c R b and c t b Represent the rotation and translation between the camera frame and the body frame respectively. b R w and b t w They are determined by the attitude and position of the UAV respectively.

[0144] The present invention converts the RGB image captured by the PTZ camera into a grayscale image, denoted as G. i p f The circular area C is the center f , and analyze the gray value in this area. f The definition of is as follows (9):

[0145] C f ={ i p| i p∈G,|| i p- i p f ||≤r f} (9)

[0146] where r f Represents C f Then, calculate C f The average gray value g in is as shown in the following formula (10):

[0147]

[0148] Where |C f | means C f The number of pixels within.

[0149] Finally, the present invention uses g to evaluate the brightness level of the leaf area. To ensure that g remains in the appropriate range [g min ,g max ], the present invention uses the step factor k g Adjust the current exposure value u to v. Then, set the camera exposure value to the adjusted v.

[0150] As an optional implementation, the threshold range includes an upper threshold limit and a lower threshold limit, and the exposure value of the gimbal camera is adjusted according to the average gray value, which specifically includes:

[0151] S2061, when the average gray value is greater than the upper threshold value, reducing the exposure value according to a preset step factor;

[0152] S2062: When the average grayscale value is less than the upper threshold value, increase the exposure value according to the step factor.

[0153] The algorithm flow of the projection-based leaf exposure adjustment algorithm of the embodiment of the present invention is as follows:

[0154]

[0155] The method steps of the embodiment of the present invention are described above. It can be recognized that the embodiment of the present invention estimates the blade position and the hub position according to the point cloud data of the blade, determines the blade angle according to the blade position and the hub position, and updates the orbital trajectory of the drone according to the hub position, thereby obtaining point cloud data again for the estimation of the blade position and the hub position, until the current blade angle and the blade angle obtained in the previous round meet the preset convergence conditions, and the final blade angle is obtained, thereby improving the accuracy of the wind turbine blade angle detection; in addition, the embodiment of the present invention obtains the straight line where the blade is located according to the point cloud data of the blade, determines the detection target spatial coordinates according to the drone position and the straight line where the blade is located, adjusts the shooting angle of the gimbal camera according to the detection target spatial coordinates and determines the detection target image coordinates, determines the target circular area in the image data captured by the gimbal camera according to the detection target image coordinates, and adjusts the exposure value of the gimbal camera according to the average grayscale value of the target circular area, thereby improving the quality of the captured wind turbine blade image.

[0156] The embodiments of the present invention are further described below in conjunction with test experiments in an operating wind farm.

[0157] The invention has been experimented on operating wind farms. To date, the platform and algorithm proposed in the invention have been tested multiple times on 5 different types of wind turbines in 3 wind farms. The drone detection platform has completed more than 100 flights. The blade angle detection algorithm has been tested under more than 80 different blade angles. The blade exposure adjustment algorithm has been evaluated under more than 60 different lighting conditions.

[0158] Extensive experiments have confirmed the stability and reliability of the platform, confirming its compatibility with the algorithm's sensor requirements. The platform has demonstrated its ability to perform detection under challenging conditions, such as operating at an altitude of 1000 meters and in strong winds of 8m / s. Next, the results of the two algorithms will be presented.

[0159] The present invention compares the experimental results of blade angle (Our method) with the vision-based method in the prior art. Stokkeland et al. proposed a vision-based method that uses Hough line transform to detect lines and uses a voting algorithm to identify blade lines. The present invention adjusts this method to adapt to the detection distance of the present invention while retaining its key voting algorithm. The final comparison of the mean angle error (MeanAngle Error) and success rate (Success Rate) of the two methods is shown in Table 1 below.

[0160] Table 1

[0161]

[0162] It can be seen that the method of the present invention exhibits an average angular error of 1.15 degrees, which exceeds the 2.14 degree error of the vision-based method. In addition, the method of the present invention has a success rate of 98.6%, which is significantly better than the 69.2% success rate of the vision-based method. These findings highlight the superior robustness and accuracy of the method proposed in the present invention relative to the vision-based method.

[0163] The platform proposed in the present invention detects along the blades of wind turbines, capturing images at set time intervals from the root to the tip of the blade. The mean μ, standard deviation σ and entropy H of the grayscale values ​​in the blade area are used as indicators to evaluate the method of the present invention. Specifically, μ reflects the brightness, and σ and H represent the complexity of the details. Usually, the lower blade surface is underexposed, while the upper surface is often overexposed. The present invention conducted experiments to evaluate these situations. The mean (Mean), standard deviation (Std Dev) and entropy (Entropy) of the original image (Original Image) and the adjusted image (Adjusted Image) in the underexposure (Underexposure) and overexposure (Overexposure) scenes are shown in Table 2 below.

[0164] Table 2

[0165]

[0166] It can be seen that in the case of underexposure, the leaf brightness increases from 25.48 to 133.27, and the richness of leaf details quantified by standard deviation and entropy increases by 218.85% and 56.19%, respectively. On the contrary, in the case of overexposure, the leaf brightness is adjusted from 245.64 to 150.85. The richness of leaf details measured by standard deviation and entropy increases by 187.73% and 40.65%, respectively. This highlights the robustness and versatility of the method of the present invention in underexposure and overexposure scenarios. The method of the present invention ensures that the brightness falls within an optimal range, effectively retaining the complex details in the leaf area.

[0167] Reference Figure 6 The embodiment of the present invention provides a wind turbine blade detection system, comprising:

[0168] A point cloud data acquisition module, used to acquire first point cloud data of a target blade through a laser radar carried by a UAV;

[0169] A point cloud data processing module, used for performing plane fitting and grid map conversion on the first point cloud data to obtain a fitting plane and a grid map;

[0170] A scanning query module is used to scan and query the grid map to obtain multiple leaf occupation points, and cluster the leaf occupation points to obtain the leaf positions;

[0171] A blade angle determination module is used to determine the hub position according to the blade position, and obtain the blade angle according to the hub position and the blade position;

[0172] The UAV trajectory update module is used to update the orbital motion trajectory of the UAV according to the fitting plane and the hub position, and return to the step of obtaining the first point cloud data of the target blade through the laser radar carried by the UAV, until the current blade angle and the blade angle obtained in the previous round meet the preset convergence conditions, and the current blade angle is output as the target blade angle.

[0173] The contents of the above method embodiments are all applicable to the present system embodiments. The functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0174] Reference Figure 7 The embodiment of the present invention provides a wind turbine blade detection device, comprising:

[0175] at least one processor;

[0176] at least one memory for storing at least one program;

[0177] When the at least one program is executed by the at least one processor, the at least one processor implements the wind turbine blade detection method.

[0178] The contents of the above method embodiments are all applicable to the present device embodiments. The functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0179] An embodiment of the present invention further provides a computer-readable storage medium, in which a program executable by a processor is stored. When the program executable by the processor is executed by the processor, it is used to execute the above-mentioned wind turbine blade detection method.

[0180] A computer-readable storage medium according to an embodiment of the present invention can execute a wind turbine blade detection method provided by an embodiment of the present invention, can execute any combination of implementation steps of the method embodiment, and has the corresponding functions and beneficial effects of the method.

[0181] The embodiment of the present invention also discloses a computer program product or a computer program, wherein the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes Figure 1 The method shown.

[0182] In some selectable embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the above-mentioned boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided by way of example, for the purpose of providing a more comprehensive understanding of technology. The disclosed method is not limited to the operation and logic flow presented herein. Selectable embodiments are expected, wherein the order of various operations is changed and the sub-operation of a part for which is described as a larger operation is performed independently.

[0183] In addition, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise specified to the contrary, one or more of the above-mentioned functions and / or features can be integrated into a single physical device and / or software module, or one or more functions and / or features can be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the present invention. More specifically, in view of the properties, functions and internal relationships of the various functional modules in the device disclosed herein, the actual implementation of the module will be understood within the conventional skills of the engineer. Therefore, those skilled in the art can implement the present invention set forth in the claims without excessive experimentation using ordinary techniques. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0184] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the above methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0185] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0186] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and editable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the above-mentioned program is printed, since the above-mentioned program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or processing in other suitable ways as necessary, and then stored in a computer memory.

[0187] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0188] In the above description of this specification, the description with reference to the terms "one embodiment / example", "another embodiment / example" or "certain embodiments / examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0189] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

[0190] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the above embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A method for detecting blades of a wind turbine generator, characterized in that: The following steps are involved: The first point cloud data of the target blade is obtained through the laser radar carried by the drone; Performing plane fitting and grid map conversion on the first point cloud data to obtain a fitting plane and a grid map; Scan and query the grid map to obtain a plurality of leaf occupation points, and cluster the leaf occupation points to obtain leaf positions; Determine the hub position according to the blade position, and obtain the blade angle according to the hub position and the blade position; The orbital motion trajectory of the UAV is updated according to the fitting plane and the hub position, and the step of obtaining the first point cloud data of the target blade through the lidar carried by the UAV is returned to, until the current blade angle and the blade angle obtained in the previous round meet the preset convergence conditions, and the current blade angle is output as the target blade angle.

2. A wind turbine blade detection method according to claim 1, characterized in that: The performing plane fitting and grid map conversion on the first point cloud data to obtain a fitting plane and a grid map specifically includes: Performing plane fitting on the first point cloud data by using a random sampling consensus algorithm to obtain the fitting plane and the corresponding unit normal vector; The map resolution is determined according to the size of the wind turbine, and the first point cloud data is converted into a grid map according to the map resolution to obtain the grid map.

3. A wind turbine blade detection method according to claim 2, characterized in that: The scanning and querying of the grid map to obtain a plurality of leaf occupation points and clustering the leaf occupation points to obtain leaf positions specifically includes: Determine a vertical scanning direction according to the unit normal vector, and vertically scan the grid map according to the vertical scanning direction to obtain a plurality of first sampling points; Determine a circular scanning trajectory on the fitting plane according to the first sampling points, and perform a circular scanning on the grid map according to the circular scanning trajectory to obtain a plurality of second sampling points; querying a grid state corresponding to the second sampling point in the grid map, and when the grid state is occupied, determining that the second sampling point is the leaf occupied point; Clustering the leaf occupied points to obtain three leaf occupied point sets; The corresponding leaf position is determined according to the average position of the leaf occupied points in each of the leaf occupied point sets.

4. A wind turbine blade detection method according to claim 3, characterized in that: Determining the hub position according to the blade position, and obtaining the blade angle according to the hub position and the blade position specifically includes: Construct a target triangle according to the three blade positions, and use the Fermat point of the target triangle as the hub position; Determine a corresponding blade direction vector according to a directed connection line from the hub position to the blade position; The corresponding blade angle is determined according to the blade direction vector.

5. A wind turbine blade detection method according to claim 1, characterized in that: The updating of the orbital motion trajectory of the UAV according to the fitting plane and the hub position specifically includes: Determine the orbital plane according to the fitting plane, so that the orbital plane is parallel to the fitting plane; Taking the projection position of the hub position on the track plane as the track center position; The updated orbital motion trajectory is determined on the orbital plane according to the preset orbital radius and the orbital center position.

6. A wind turbine blade detection method according to any one of claims 1 to 5, characterized in that: The drone is also equipped with a pan-tilt camera, and the wind turbine blade detection method further includes the following steps: Acquire second point cloud data of the target blade by using the laser radar; Performing straight line fitting on the second point cloud data to obtain a straight line where the blade is located, and determining the detection target spatial coordinates according to the position of the UAV and the straight line where the blade is located; Adjusting the shooting angle of the pan / tilt camera according to the detection target space coordinates, and determining the corresponding detection target image coordinates according to the detection target space coordinates; Setting the exposure value of the pan-tilt camera, and acquiring the image data of the target blade through the pan-tilt camera; Converting the image data into a grayscale image, and determining a target circular area in the grayscale image according to the detection target image coordinates; Determine the average grayscale value of the target circular area, adjust the exposure value of the gimbal camera according to the average grayscale value, and return to the step of obtaining the image data of the target leaf through the gimbal camera until the average grayscale value meets the preset threshold range, and output the image data currently captured as the target leaf image data.

7. A wind turbine blade detection method according to claim 6, characterized in that: The threshold range includes an upper threshold limit and a lower threshold limit, and the step of adjusting the exposure value of the gimbal camera according to the average grayscale value specifically includes: When the average grayscale value is greater than the upper threshold value, reducing the exposure value according to a preset step factor; When the average grayscale value is less than the upper threshold value, the exposure value is increased according to the step factor.

8. A wind turbine blade detection system, characterized in that: include: A point cloud data acquisition module, used to acquire first point cloud data of a target blade through a laser radar carried by a UAV; A point cloud data processing module, used for performing plane fitting and grid map conversion on the first point cloud data to obtain a fitting plane and a grid map; A scanning query module, used for scanning and querying the grid map to obtain a plurality of leaf occupation points, and clustering the leaf occupation points to obtain leaf positions; A blade angle determination module, used to determine the hub position according to the blade position, and obtain the blade angle according to the hub position and the blade position; The UAV trajectory update module is used to update the orbital motion trajectory of the UAV according to the fitting plane and the hub position, and return to the step of obtaining the first point cloud data of the target blade through the laser radar carried by the UAV, until the current blade angle and the blade angle obtained in the previous round meet the preset convergence conditions, and the current blade angle is output as the target blade angle.

9. A wind turbine blade detection device, characterized in that: include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements a wind turbine blade detection method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a program executable by a processor, characterized in that: The program executable by the processor is used to execute a wind turbine blade detection method according to any one of claims 1 to 7 when executed by the processor.

Images