Fan blade dynamic autonomous inspection method based on unmanned aerial vehicle

Abstract

The invention discloses a fan blade dynamic autonomous inspection method based on an unmanned aerial vehicle. The method comprises the following steps: controlling the unmanned aerial vehicle to shoot a panoramic top view of a fan; determining the orientation of the fan according to the panoramic top view, controlling the unmanned aerial vehicle to fly to the front of the fan for shooting according to the orientation of the fan, extracting the pixel coordinates of the hub center point and the pixel coordinates of the blade tip in the image, and further planning the inspection track and the number of inspection points of the unmanned aerial vehicle. The method is carried out under the working condition that the draught fan is not stopped, and therefore economic losses caused by stopping of the draught fan are avoided; the inspection process is full-automatic, the inspection efficiency is high, the shooting direction of the camera is parallel to the orientation of the fan, the center line of the shot blade and the axis of an image coordinate system can be kept horizontal, the collected image can fully cover the fan blade, and the image collection quality is high.

Description

Technical Field

[0001] The present invention relates to the technical fields of wind turbines, unmanned aerial vehicles, autonomous inspection technologies, wind farm operation and maintenance, and in particular to a wind turbine blade inspection method based on unmanned aerial vehicles. Background Art

[0002] As a clean and renewable energy source, wind energy has attracted the attention of the whole society and has become one of the main substitutes for traditional fossil energy. In recent years, the construction of wind farms has continued to deepen, and the installed capacity of wind turbines has continued to increase. Wind turbine blades are an important component of wind turbine energy conversion, converting wind energy into mechanical energy. At the same time, they are the main load-bearing components of wind turbines and play a key role in the safe operation of the entire wind turbine. The working environment of the blades is particularly harsh. In the natural environment, the long-term continuous operation will have a great impact on the operation of the blades due to the external climate, especially typhoons, thunderstorms, ice and snow, sandstorms and other severe weather conditions may cause damage to the blades at any time, thereby causing economic losses to wind power generation. Therefore, daily inspection of wind turbine blades is an important part of the wind power maintenance and management process. Ensuring that the blades are in a healthy state is an essential basic work to ensure the safe operation of the wind turbine.

[0003] The commonly used inspection methods for wind turbine blades are mainly divided into manual telescope observation and manual drone observation. Both methods have certain disadvantages, such as heavy workload for inspectors, easy fatigue of inspectors, low safety, low efficiency of blade inspection, etc. At the same time, the inspection process has strict requirements on the inspection precision instruments and the professional technical level of the inspectors.

[0004] The application of UAV autonomous inspection technology aims to realize the detection and monitoring of wind power equipment through the intelligent, automated and semi-automated robot technology. It can improve the safety, reliability and comprehensiveness of inspections and provide technical support for the efficient, reliable and stable operation of wind power generation. UAV autonomous inspection technology mainly uses UAVs for viewpoint planning and high-altitude aerial photography of wind power equipment to obtain its full range of image information. UAV viewpoint planning technology can obtain detailed information and data of wind power equipment, such as the damage of wind turbine blades, by taking high-definition aerial photos of wind power equipment, so as to monitor and analyze wind power equipment and provide strong support for the inspection and maintenance of wind power equipment.

[0005] At present, there are a large number of relevant literatures at home and abroad that study the application of drone inspection technology in wind power equipment, and have proposed a variety of autonomous drone inspection technologies for wind turbine blades.

[0006] 1. In the article entitled "Analysis, Design and Research on Autonomous Inspection System of Wind Turbine Blades by UAVs in Wind Farms", the author studied the autonomous inspection system of wind turbine blades by UAVs. First, the core functions that need to be realized by the autonomous inspection system of UAVs are explained, and then the hardware equipment composition and software functional modules of the autonomous inspection system are analyzed and designed. The key technologies required to achieve the autonomous inspection goal of UAVs are studied from the aspects of wind turbine shutdown posture parameter calculation, autonomous inspection path planning and visual servo gimbal tracking. Finally, the detection content and practical application of blade image collection by UAVs are explained. This system is of great significance for realizing autonomous and intelligent inspection of wind turbine blades in wind farms, but this method is designed after the wind turbine is shut down, and inspection can only be carried out when the wind turbine is shut down.

[0007] 2. Zeng Fanchun et al. established a wind turbine model in a patent application with application number CN202210666133.7 and titled “A method for automatic trajectory planning of unmanned aerial vehicles for wind turbine blade inspection”. The wind turbine model was gridded according to the inspection requirements to obtain a wind turbine surface graphics model. The candidate track point area was planned based on each grid patch of the model to ensure that the selected track point in the track point area can see at least one complete patch area, so as to obtain a complete wind turbine surface image; track point sampling was performed based on the candidate track point area to complete track point optimization; track line planning was completed; although this method can effectively inspect wind turbine blades, it is necessary to identify the shutdown state. At the same time, the accuracy of the grid directly affects the comprehensiveness of the inspection and the efficiency of the trajectory planning. If the grid is too sparse, the inspection points may be unevenly distributed and key surface areas may be missed. If the grid is too dense, the computational complexity will increase, and high requirements will be placed on real-time performance and computing resources, which may not meet the needs of fast planning.

[0008] 3. In the article titled "Transfer learning-based fault detection in wind turbine blades using radarplots and deep learning models", the authors Jaikrishna M. Arjun et al. proposed using transfer learning methods to monitor wind turbine blades and identify fault conditions. The study used a good blade condition and four faulty blade conditions: bending, loose hub-blade connection, erosion, and pitch angle distortion. The vibration signals of each blade condition were collected and converted into radar plots, which were then input and analyzed using a pre-trained deep learning model. Hyperparameters including optimizer, training-test split ratio, batch size, epoch, and learning rate were examined to determine the optimal configuration for each network. These findings highlight the potential of using deep learning models for wind turbine monitoring and fault detection, significantly improving the efficiency and reliability of wind turbines, but the method is extremely complex and difficult to apply to actual scenarios. Summary of the invention

[0009] In view of this, the purpose of the present invention is to provide a dynamic autonomous inspection method for wind turbine blades based on a drone, so as to solve the technical problem of autonomously inspecting wind turbine blades by a drone when the wind turbine is in operation and improving the inspection efficiency.

[0010] The present invention provides a method for dynamic autonomous inspection of wind turbine blades based on an unmanned aerial vehicle, comprising the following steps:

[0011] 1) According to the coordinates of the center of the wind turbine tower in the world coordinate system (X t0 , Y t0 ), tower center height H and wind turbine blade length L, control the drone to fly to the panoramic overlooking shooting point (X t0 , Y t0， H+D) to capture a panoramic bird's-eye view of the wind turbine, where D satisfies the following panoramic bird's-eye view constraints:

[0012]

[0013] Where D is the minimum safe distance between the UAV and the wind turbine in vertical height, θ l is the horizontal field of view of the gimbal camera carried by the drone, θ v The vertical field of view of the gimbal camera on the drone;

[0014] 2) Determine the direction of the fan, including the following steps:

[0015] The fan blades are obtained by denoising the panoramic top view of the fan. The Hough linear transformation detection method is used to fit the straight line representing the fan blades, and the angle γ between the straight line and the north direction of the image coordinate system is calculated. According to the perpendicular relationship between the orientation of the fan body and the straight line representing the blades, the angle β between the orientation of the fan and the north direction of the image coordinate system is calculated:

[0016] β=90-γ (2)

[0017] And according to the angle α between the true north of the image coordinate system and the true north of the world coordinate system, the angle between the wind turbine direction and the true north of the world coordinate system is calculated.

[0018]

[0019] 3) Control the UAV to fly in the direction of the wind turbine to the front of the wind turbine, then control the UAV to descend vertically to the front of the center of the wind turbine hub, and then control the UAV to fly in the direction of the wind turbine to adjust the distance d between the UAV and the center of the wind turbine hub to be greater than or equal to the set minimum safety distance;

[0020] 4) Control the shooting direction of the gimbal camera carried by the drone to be parallel to the direction of the wind turbine, continuously shoot the front of the wind turbine, and extract the pixel coordinates of the hub center point and the blade tip in the image, find the image of the blade tip on the same horizontal line as the hub center, and obtain the blade tip pixel coordinates (u, v) in the image, and measure the distance Z from the drone to the wind turbine through the laser lightning carried by the drone c , perform inverse operation according to the following conversion formula (4) from the world coordinate system to the pixel coordinate system, convert the coordinates of the blade tip in the pixel coordinate system to the world coordinate system, and then obtain the world coordinates (Xw, Yw, Zw) of the wind turbine blade tip;

[0021]

[0022] 5) Plan the inspection trajectory of the drone:

[0023] According to the known coordinates of the center of the wind turbine hub (Xw0, Yw0, Zw0) and the coordinates of the tip of the wind turbine blade (Xw1, Yw1, Zw1) in the world coordinate system, the straight line equation of the blade determined by these two points in the world coordinate system is constructed as follows:

[0024]

[0025] The UAV is designed to fly on the windward and leeward sides of the wind turbine in a straight line parallel to equation (5), and the inspection trajectory equation of the UAV on the windward side of the wind turbine is obtained as follows:

[0026]

[0027] The inspection trajectory equation of the UAV on the leeward side of the wind turbine is:

[0028]

[0029] 6) Determine the number of inspection points to be set along the inspection track according to the following method:

[0030] When the drone is facing the wind turbine, take a photo of the wind turbine blades, find the tip coordinates (u, v) and hub center coordinates (u0, v0) of any horizontal wind turbine blade in the pixel coordinate system, and calculate the slope k of the blade and the angle θ with the horizontal axis of the coordinate system based on the coordinates of the two points:

[0031]

[0032] The length L of the blade at each inspection point in the image coordinate system is calculated according to the following formula: i :

[0033] L i =|x / cosθ| (9)

[0034] Where x is the horizontal imaging width of the camera;

[0035]

[0036] The number of inspection points of the drone on the windward or leeward side of the blade is:

[0037]

[0038] 8) Control the drone to fly to each inspection point in turn along the inspection trajectory, and set the time t to trigger the gimbal camera on the drone to start inspection and shooting at the current inspection point:

[0039] t=t0+T1-T (12)

[0040] Where T is the camera trigger delay, T1 is the shooting interval, T1 = 120 / ω, ω is the angular velocity of the fan blade rotation, and ω is determined by the following method:

[0041] After the drone stably hovers at the current inspection point, the timing starts when the laser radar carried by the drone detects the fan blade for the first time, and stops when the fan blade is detected for the fourth time. The total time t1 taken for the fan blade to rotate one circle is obtained by the time difference between the two timings, ω=360 / t1;

[0042] t0 is the time when the laser lightning detects the wind blade after the angular velocity of the wind blade rotation is determined;

[0043] The PTZ camera is controlled to take a photo of each blade at least once at each inspection point.

[0044] Furthermore, the wind turbine blade dynamic autonomous inspection method based on a drone also includes step 9):

[0045] Process the currently captured leaf image, transform the leaf centerline in the pixel coordinate system to the image coordinate system, and calculate the slope k of the leaf centerline l , then according to the slope k l Image capture quality evaluation:

[0046] If the blade centerline slope k l The value is positive, and k l If it is greater than the set threshold, the PTZ camera trigger time is too advanced, that is, the PTZ camera took a picture of the blade in advance, then increase the camera trigger delay and take the picture again;

[0047] If the blade centerline slope k l The value is negative, and |k l |If it is greater than the set threshold, the trigger time of the gimbal camera is too delayed, that is, the gimbal camera takes pictures of the blades with a delay, then reduce the camera trigger delay and take the picture again.

[0048] Beneficial effects of the present invention:

[0049] 1. This UAV-based dynamic autonomous inspection method for wind turbine blades is carried out without stopping the wind turbine, thus avoiding economic losses caused by wind turbine shutdown.

[0050] 2. This UAV-based dynamic autonomous inspection method for wind turbine blades is fully automated and efficient. The camera shooting direction is parallel to the wind turbine direction, and can ensure that the center line of the captured blade is horizontal with the axis of the image coordinate system, so that the collected image can fully cover the wind turbine blades, and the image acquisition quality is high. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 This is a flowchart of the dynamic inspection of wind turbine blades based on drones.

[0052] Figure 2 This is a schematic diagram for calculating the fan orientation angle.

[0053] Figure 3 A schematic diagram of the coordinate system position relationship. DETAILED DESCRIPTION

[0054] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0055] The wind turbine blade dynamic autonomous inspection method based on a drone in this embodiment includes the following steps:

[0056] 1) According to the coordinates of the center of the wind turbine tower in the world coordinate system (X t0, Y t0 ), tower center height H and wind turbine blade length L, control the drone to fly to the panoramic overlooking shooting point (X t0 , Y t0， H+D) to capture a panoramic bird's-eye view of the wind turbine, where D satisfies the following panoramic bird's-eye view constraints:

[0057]

[0058] Where D is the minimum safe distance between the UAV and the wind turbine in vertical height, θ l is the horizontal field of view of the gimbal camera carried by the drone, θ v It is the vertical field of view of the gimbal camera carried by the drone.

[0059] In a specific implementation, the origin of the world coordinate system can be set at the intersection of the tower centerline and the horizontal ground. At this time, the coordinates of the wind turbine tower center in the world coordinate system (X t0 , Y t0 ) is (0,0).

[0060] 2) Determining the direction of the wind turbine is the basis for determining the shooting direction of the gimbal camera carried by the drone. In this embodiment, determining the direction of the wind turbine includes the following steps:

[0061] The fan blades are obtained by denoising the panoramic top view of the fan. The Hough linear transformation detection method is used to fit the straight line representing the fan blades, and the angle γ between the straight line and the north direction of the image coordinate system is calculated. According to the perpendicular relationship between the orientation of the fan body and the straight line representing the blades, the angle β between the orientation of the fan and the north direction of the image coordinate system is calculated:

[0062] β=90-γ (2)

[0063] And according to the angle α between the true north of the image coordinate system and the true north of the world coordinate system, the angle between the wind turbine direction and the true north of the world coordinate system is calculated.

[0064]

[0065] 3) Control the UAV to fly in the direction of the wind turbine to the front of the wind turbine, then control the UAV to descend vertically to the front of the center of the wind turbine hub, and then control the UAV to fly in the direction of the wind turbine to adjust the distance d between the UAV and the center of the wind turbine hub to be greater than or equal to the set minimum safety distance; the minimum safety distance is generally set to 40m.

[0066] 4) Control the shooting direction of the gimbal camera carried by the drone to be parallel to the direction of the wind turbine, continuously shoot the front of the wind turbine, and extract the pixel coordinates of the hub center point and the blade tip in the image, find the image of the blade tip on the same horizontal line as the hub center, and obtain the blade tip pixel coordinates (u, v) in the image, and measure the distance Z from the drone to the wind turbine through the laser lightning carried by the drone c , perform inverse operation according to the following conversion formula (4) from the world coordinate system to the pixel coordinate system, convert the coordinates of the blade tip in the pixel coordinate system to the world coordinate system, and then obtain the world coordinates (Xw, Yw, Zw) of the wind turbine blade tip;

[0067]

[0068] 5) Plan the inspection trajectory of the drone:

[0069] According to the known coordinates of the center of the wind turbine hub (Xw0, Yw0, Zw0) and the coordinates of the tip of the wind turbine blade (Xw1, Yw1, Zw1) in the world coordinate system, the straight line equation of the blade determined by these two points in the world coordinate system is constructed as follows:

[0070]

[0071] The UAV is designed to fly on the windward and leeward sides of the wind turbine in a straight line parallel to equation (5), and the inspection trajectory equation of the UAV on the windward side of the wind turbine is obtained as follows:

[0072]

[0073] The inspection trajectory equation of the UAV on the leeward side of the wind turbine is:

[0074]

[0075] 6) Determine the number of inspection points to be set along the inspection track according to the following method:

[0076] When the drone is facing the wind turbine, take a photo of the wind turbine blades, find the tip coordinates (u, v) and hub center coordinates (u0, v0) of any horizontal wind turbine blade in the pixel coordinate system, and calculate the slope k of the blade and the angle θ with the horizontal axis of the coordinate system based on the coordinates of the two points:

[0077]

[0078] The length L of the blade at each inspection point in the image coordinate system is calculated according to the following formula: i :

[0079] L i =|x / cosθ| (9)

[0080] Where x is the horizontal imaging width of the camera;

[0081]

[0082] The number of inspection points of the drone on the windward or leeward side of the blade is:

[0083]

[0084] 8) Control the drone to fly to each inspection point in turn along the inspection trajectory, and set the time t to trigger the gimbal camera on the drone to start inspection and shooting at the current inspection point:

[0085] t=t0+T1-T (12)

[0086] Where T is the camera trigger delay, T1 is the shooting interval, T1 = 120 / ω, ω is the angular velocity of the fan blade rotation, and ω is determined by the following method:

[0087] After the drone stably hovers at the current inspection point, the timing starts when the laser radar carried by the drone detects the fan blade for the first time, and stops when the fan blade is detected for the fourth time. The total time t1 taken for the fan blade to rotate one circle is obtained by the time difference between the two timings, ω=360 / t1;

[0088] t0 is the time when the laser lightning detects the wind blade after the angular velocity of the wind blade rotation is determined;

[0089] The PTZ camera is controlled to take a photo of each blade at least once at each inspection point.

[0090] In order to make the collected image cover the blades more perfectly, the blades in the image should be kept level with the x-axis of the image as much as possible. Therefore, as an improvement to the above embodiment, the dynamic autonomous inspection method of wind turbine blades based on drones also includes step 9):

[0091] Process the currently captured leaf image, transform the leaf centerline in the pixel coordinate system to the image coordinate system, and calculate the slope k of the leaf centerline l , then according to the slope k l Image capture quality evaluation:

[0092] From the geometric relationship, we know that the blade slope k l The larger the absolute value, the more the center line of the leaf deviates from the x-axis of the image center, that is, the less leaf coverage is collected in the image; on the contrary, k l The smaller the absolute value of is, the closer it is to 0, and the closer the center line of the leaf is to the center x-axis of the image, that is, the more leaf coverage is collected in the image, so:

[0093] If the blade centerline slope k lThe value is positive, and k l If it is greater than the set threshold, the PTZ camera trigger time is too advanced, that is, the PTZ camera took a picture of the blade in advance, then increase the camera trigger delay and take the picture again;

[0094] If the blade centerline slope k l The value is negative, and |k l |If it is greater than the set threshold, the trigger time of the gimbal camera is too delayed, that is, the gimbal camera takes pictures of the blades with a delay, then reduce the camera trigger delay and take the picture again.

[0095] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution of the present invention, which should be included in the scope of the claims of the present invention.

Claims

1. A dynamic autonomous inspection method for wind turbine blades based on drones, characterized in that: Includes steps: 1) According to the coordinates of the center of the wind turbine tower in the world coordinate system (X t0 , Y t0 ), tower center height H and wind turbine blade length L, control the drone to fly to the panoramic overlooking shooting point (X t0 , Y t0， H+D) to capture a panoramic bird's-eye view of the wind turbine, where D satisfies the following panoramic bird's-eye view constraints: Where D is the minimum safe distance between the UAV and the wind turbine in vertical height, θ l is the horizontal field of view of the gimbal camera carried by the drone, θ v The vertical field of view of the gimbal camera on the drone; 2) Determine the direction of the fan, including the following steps: The fan blades are obtained by denoising the panoramic top view of the fan. The Hough linear transformation detection method is used to fit the straight line representing the fan blades, and the angle γ between the straight line and the north direction of the image coordinate system is calculated. According to the perpendicular relationship between the orientation of the fan body and the straight line representing the blades, the angle β between the orientation of the fan and the north direction of the image coordinate system is calculated: β=90-γ (2) And according to the angle α between the true north of the image coordinate system and the true north of the world coordinate system, the angle between the wind turbine direction and the true north of the world coordinate system is calculated. 3) Control the UAV to fly in the direction of the wind turbine to the front of the wind turbine, then control the UAV to descend vertically to the front of the center of the wind turbine hub, and then control the UAV to fly in the direction of the wind turbine to adjust the distance d between the UAV and the center of the wind turbine hub to be greater than or equal to the set minimum safety distance; 4) Control the shooting direction of the gimbal camera carried by the drone to be parallel to the direction of the wind turbine, continuously shoot the front of the wind turbine, and extract the pixel coordinates of the hub center point and the blade tip in the image, find the image of the blade tip on the same horizontal line as the hub center, and obtain the blade tip pixel coordinates (u, v) in the image, and measure the distance Z from the drone to the wind turbine through the laser lightning carried by the drone c , perform inverse operation according to the following conversion formula (4) from the world coordinate system to the pixel coordinate system, convert the coordinates of the blade tip in the pixel coordinate system to the world coordinate system, and then obtain the world coordinates (Xw, Yw, Zw) of the wind turbine blade tip; 5) Plan the inspection trajectory of the drone: According to the known coordinates of the center of the wind turbine hub (Xw0, Yw0, Zw0) and the coordinates of the tip of the wind turbine blade (Xw1, Yw1, Zw1) in the world coordinate system, the straight line equation of the blade determined by these two points in the world coordinate system is constructed as follows: The UAV is designed to fly on the windward and leeward sides of the wind turbine in a straight line parallel to equation (5), and the inspection trajectory equation of the UAV on the windward side of the wind turbine is obtained as follows: The inspection trajectory equation of the UAV on the leeward side of the wind turbine is: 6) Determine the number of inspection points to be set along the inspection track according to the following method: When the drone is facing the wind turbine, take a photo of the wind turbine blades, find the tip coordinates (u, v) and hub center coordinates (u0, v0) of any horizontal wind turbine blade in the pixel coordinate system, and calculate the slope k of the blade and the angle θ with the horizontal axis of the coordinate system based on the coordinates of the two points: The length L of the blade at each inspection point in the image coordinate system is calculated according to the following formula: i : L i =|x / cosθ| (9) Where x is the horizontal imaging width of the camera; The number of inspection points of the drone on the windward or leeward side of the blade is: 8) Control the drone to fly to each inspection point in turn along the inspection trajectory, and set the time t to trigger the gimbal camera on the drone to start inspection and shooting at the current inspection point: t=t0+T1-T (12) Where T is the camera trigger delay, T1 is the shooting interval, T1 = 120 / ω, ω is the angular velocity of the fan blade rotation, and ω is determined by the following method: After the drone stably hovers at the current inspection point, the timing starts when the laser radar carried by the drone detects the fan blade for the first time, and stops when the fan blade is detected for the fourth time. The total time t1 taken for the fan blade to rotate one circle is obtained by the time difference between the two timings, ω=360 / t1; t0 is the time when the laser lightning detects the wind blade after the angular velocity of the wind blade rotation is determined; The PTZ camera is controlled to take a photo of each blade at least once at each inspection point.

2. The method for dynamic autonomous inspection of wind turbine blades based on a drone according to claim 1 is characterized in that: Also includes step 9): Process the currently captured leaf image, transform the leaf centerline in the pixel coordinate system to the image coordinate system, and calculate the slope k of the leaf centerline l , then according to the slope k l Image capture quality evaluation: If the blade centerline slope k l The value is positive, and k l If it is greater than the set threshold, the PTZ camera trigger time is too advanced, that is, the PTZ camera took a picture of the blade in advance, then increase the camera trigger delay and take the picture again; If the blade centerline slope k l The value is negative, and |k l |If it is greater than the set threshold, the trigger time of the gimbal camera is too delayed, that is, the gimbal camera takes pictures of the blades with a delay, then reduce the camera trigger delay and take the picture again.

Images