Fan blade tip visual positioning method based on multi-parameter correction

Abstract

The invention provides a fan blade tip visual positioning method based on multi-parameter correction, and relates to the technical field of visual localization, and the method comprises the steps: firstly controlling an unmanned aerial vehicle to reach a top view point through the position information of a fan, carrying out the target detection and feature extraction of the fan through an algorithm, and according to the position relation between a hub and a cabin, carrying out the target detection and feature extraction; a redundancy calculation mechanism for acquiring yaw information of the fan is designed; secondly, the unmanned aerial vehicle conducts pose correction based on the top view by means of visual feedback, and therefore calculation is conducted and the unmanned aerial vehicle goes to a fan front view point; then, primary positioning of blade tips is completed through a front view, and three correction parameters of a blade phase angle, a wind wheel elevation angle and a blade taper angle are measured and calculated; and finally, constructing a fan simplified model and combining the corrected parameters to realize accurate positioning. According to the invention, precision guarantee can be provided for fan inspection route planning.

Description

Technical Field

[0001] The invention relates to the technical field of visual positioning, and in particular to a fan blade tip visual positioning method based on multi-parameter correction. Background Art

[0002] my country is vigorously developing new energy sources such as wind energy. By the end of 2023, China's wind power installed capacity has reached about 440 million kilowatts. However, with the rapid growth of wind power installed capacity, the operation and maintenance of wind power equipment has become increasingly important. In order to improve the efficiency and quality of operation and maintenance, wind farms have begun to widely adopt a combination of intelligent robots and manual inspections. Among them, drones have shown great potential due to their high efficiency and convenience. In the actual operation and maintenance process of wind farms, especially when the wind turbine is shut down, the randomness of the yaw angle and phase angle poses a huge challenge to the autonomous inspection task of drones. Among them, blade tip positioning is the primary technical difficulty in achieving autonomous inspection. As a key part of the wind turbine blade, the state of the blade tip directly affects the operating safety and efficiency of the wind turbine. Therefore, accurately and quickly locating the blade tip position is crucial for drones to perform subsequent inspection tasks. However, in the face of random changes in the yaw angle and phase angle, the accuracy and efficiency of blade tip positioning in existing technologies still need to be improved, which has become a technical bottleneck restricting the wider application of drones in the field of wind power inspection. Therefore, developing a technology that can adapt to random changes in yaw angle and phase angle and achieve efficient and accurate blade tip positioning is of great significance for promoting the intelligent operation and maintenance of wind power and improving operation and maintenance efficiency.

[0003] Relevant scholars at home and abroad have conducted certain research in this field: Castelar WC et al. proposed a solution based on light detection and ranging, by using a drone equipped with a 2D-LiDAR sensor to fly around the blade and perform SVD fitting, and realize blade tip positioning according to point cloud information. In this solution, the drone needs to enter the working area of ​​the wind turbine, which is less safe and time-consuming; the prior art provides a wind turbine blade image acquisition and defect detection system, which extracts the edge orientation of the wind turbine cabin through the drone, and obtains the cabin heading angle by subtracting it from the direction of the drone head, and realizes blade tip positioning in combination with the physical parameters of the wind turbine, but this method is only applicable to the "inverted Y-shaped" blade state; the prior art also provides a wind turbine blade tip positioning method based on drone vision, which uses the principle of homography transformation relationship between the front view and side view of the wind turbine to fit the three blades in the image into an ellipse, and calculates the yaw of the wind turbine according to the relationship between the major and minor axis lengths, thereby realizing blade tip positioning. This method ignores the existence of elevation angle and cone angle, so it can only achieve rough positioning; the prior art also proposes a method based on Mask The R-CNN deep learning method for positioning wind blades uses a deep learning algorithm to extract information from the wind turbine panorama, and finally calculates the blade tip coordinates through the drone position information and the basic information of the unit to achieve positioning, but this method has a slow detection speed.

[0004] Existing wind turbine blade tip positioning technologies based on drones are mainly divided into two categories. The first method relies on laser point cloud data and builds a complete three-dimensional model of the wind turbine to accurately locate each component of the wind turbine. The second method uses visual recognition technology to quickly locate the blade tip based on a simplified wind turbine model. At present, visual positioning is the mainstream method.

[0005] The method for positioning laser point cloud data has the following disadvantages:

[0006] (1) Poor real-time performance. Since a complete model needs to be built to achieve positioning, it is impossible to detect the wind turbine in real time.

[0007] (2) Model non-reusability. Since the posture of the wind turbine is random each time it stops, if laser point cloud positioning is used, it is necessary to first use a drone to circle the wind turbine to collect point clouds, and then transmit them to the ground end for data processing, model analysis and construction before the next inspection can be carried out. The entire process is large, complex, time-consuming and labor-intensive.

[0008] The visual positioning method has the following disadvantages:

[0009] (1) Slightly poor versatility. Among the existing visual positioning methods and technologies, some require the fan to be stopped in advance to an "inverted Y shape" before it can be visually positioned. However, stopping the fan in an inverted Y shape requires a lot of time to adjust.

[0010] (2) Ignoring the rotor pitch angle and blade cone angle. The current mainstream visual positioning methods still have certain limitations in practical applications. In particular, they often ignore the influence of two key parameters of the wind turbine: the rotor pitch angle and blade cone angle. Ignoring these two parameters will lead to large errors in calculating the blade tip position, thus affecting the accuracy and reliability of inspections.

[0011] (3) Lack of exception handling mechanism. Some visual positioning technologies do not yet have a complete exception handling mechanism and only rely on the wind turbine hub or cabin to calculate the yaw angle. Once the hub or cabin is blocked, an abnormal state will occur, causing the drone to fall into a reciprocating adjustment cycle. The adjustment time can be as long as 8 minutes, which takes up a lot of time, or the pixel coordinates cannot be updated, and the calculated correction point will continue to deviate significantly from the wind turbine. Summary of the invention

[0012] In order to overcome the shortcomings of the prior art, the purpose of the present invention is to provide a fan blade tip visual positioning method based on multi-parameter correction. The present invention uses a top view of the fan to locate the direction of the fan. While maintaining a certain safety distance, the fan yaw information can be accurately and quickly obtained. The redundant calculation mechanism therein can avoid the large errors caused by possible occlusion relationships between fan components and is more stable.

[0013] To achieve the above object, the present invention provides the following solutions:

[0014] A fan blade tip visual positioning method based on multi-parameter correction, comprising:

[0015] Collecting image information of a target wind turbine, and performing data annotation and model training on the image information to obtain a trained target detection model; the image information includes a panoramic view and a partial view of the target wind turbine;

[0016] Determine the top view shooting point of the UAV according to the structural parameters of the target wind turbine;

[0017] A first image to be tested is photographed by a drone at the top view shooting point, and the first image to be tested is input into the target detection model to obtain first pixel information of the target wind turbine structure, and yaw information of the target wind turbine is calculated according to the first pixel information of the target wind turbine structure; the yaw information includes a yaw angle of the wind turbine;

[0018] Performing position correction and attitude correction on the posture of the UAV according to the yaw information;

[0019] Control the drone to move to a shooting point facing the wind turbine according to the corrected posture, shoot a second image to be tested at the shooting point facing the wind turbine through the drone, input the second image to be tested into the target detection model, obtain second pixel information of the target wind turbine structure, and determine the blade phase angle according to the second pixel information;

[0020] Based on a preset simplified wind turbine model, the wind rotor pitch angle and the blade cone angle are determined according to the blade phase angle; the simplified wind turbine model includes: an ideal wind turbine model, a phase angle-pitch angle model and a phase angle-pitch angle-cone angle model;

[0021] The blade tip is determined according to the blade phase angle, the wind wheel pitch angle and the blade cone angle, and the final corrected coordinates of the blade tip are determined according to the blade tip, the blade phase angle and a coordinate system conversion matrix including the wind turbine yaw angle.

[0022] Preferably, the annotation categories of the data annotation include: wind turbine as a whole, hub, nacelle, wind turbine blades and tower.

[0023] Preferably, the target detection model is constructed based on the lightweight YOLOv5 algorithm.

[0024] Preferably, determining the top view shooting point of the drone according to the structural parameters of the target wind turbine includes:

[0025] Obtain the structural parameters of the target wind turbine and the field of view parameters of the airborne camera; the structural parameters include: the center height h of the wind turbine hub and the center coordinates of the wind turbine tower ( W x t , W y t );in, W x t is the horizontal coordinate value of the center coordinate of the wind turbine tower, W y t is the vertical coordinate value of the center coordinate of the wind turbine tower; the field of view parameters include: horizontal field of view angle θ hfov and vertical field angle θ vfov ;

[0026] Determine the horizontal coordinates of the wind turbine tower center coordinates as the hub center rough coordinates;

[0027] Determine the top view shooting point according to the hub center rough coordinates and the field of view parameters of the onboard camera ( W x t , W y t ,h+d); the formula of the constraint condition satisfied by the top view shooting point is: Where d is the straight-line distance between the drone and the hub center, d s is the minimum safe distance between the UAV and the wind turbine, d vfov and d hfov The minimum image shooting distance to cover the entire wind turbine under fixed horizontal and vertical field of view, h cam and w cam are the height and width of the field of view, and L is the length of the blade.

[0028] Preferably, a first image to be tested is photographed by a drone at the top view shooting point, and the first image to be tested is input into the target detection model to obtain first pixel information of the target wind turbine structure, and the yaw information of the target wind turbine is calculated according to the first pixel information of the target wind turbine structure, including:

[0029] After controlling the drone to fly to the position of the overhead view shooting point, use the onboard camera carried by the drone to shoot the first image to be tested;

[0030] Performing target detection on the first image to be tested by using the target detection model to extract first pixel information of the target wind turbine structure; the first pixel information includes: pixel information of the hub, the nacelle and the blades;

[0031] When both the hub and the nacelle are visible, the hub-nacelle centerline method is used to construct the vector according to the pixel coordinate values ​​of the hub center point O and the nacelle center point M extracted by detection. Unit vector in the negative direction of v-axis and u-axis of pixel coordinate system and

[0032] The wind turbine yaw angle yaw is calculated based on the positional relationship between the wind wheel axis and the u-axis and v-axis in the pixel coordinate system. f for:

[0033] When the hub and the nacelle are not visible, the blade slope weight method is used. After obtaining the pre-selected frame of the blade through detection and extraction, Hough line detection is performed on the image in the frame to obtain j detection line segments;

[0034] Calculate the pixel length l of each detection line segment j , and the length coefficient of each blade and the slope k of the detection line segment j The product weighted value of is fitted to a straight line with a slope of k0, and the axis of the wind rotor is the perpendicular line of the straight line, then the yaw angle of the wind turbine yaw can be obtained f for: Wherein, N is a set of positive integers.

[0035] Preferably, performing position correction and attitude correction on the position and attitude of the UAV according to the yaw information includes:

[0036] Obtain the pixel coordinates of the center point of the identified wheel hub through target detection (u hub ,v hub ), let the center point coordinates of the pixel plane be (u0, v0), and determine the pixel difference between the center point pixel coordinates and the center point coordinates of the pixel plane as the correction values ​​△u and △v of the drone in the pixel coordinate system; the calculation formula is: Among them, u hub is the horizontal coordinate value of the pixel coordinate of the center point of the wheel hub, v hub is the vertical coordinate value of the pixel coordinate of the center point of the hub, u0 is the horizontal coordinate value of the coordinate of the center point of the pixel plane, and v0 is the vertical coordinate value of the coordinate of the center point of the pixel plane;

[0037] The correction values ​​△x and △y in the world coordinate system are calculated by coordinate system conversion. When △x and △y are both less than 1m, it is considered that the UAV is directly above the center of the wheel hub and no position correction is required. The current horizontal coordinate of the UAV in the world coordinate system ( W x0, W y0) is the precise horizontal coordinate of the hub center; where, W x0 is the horizontal coordinate value of the current world coordinate system of the drone. W y0 is the ordinate value of the current world coordinate system of the drone;

[0038] The yaw angle of the wind turbine calculated at the current drone position f As the reference value, the updated yaw angle of the drone is obtained. d for:

[0039] After obtaining the updated value, the drone performs yaw correction and sets the yaw angle error threshold to ±5°, that is, when ||yaw d -yaw f When |-π|≤0.0873, it is considered that the UAV and the wind turbine are facing each other, and no attitude angle correction is required. f Determine the precise wind turbine yaw information;

[0040] Repeat the steps of position correction and attitude correction until the position and attitude of the drone are within the allowable range of the error threshold.

[0041] Preferably, the drone is controlled to move to a shooting point facing the wind turbine according to the corrected posture, and a second image to be tested is shot by the drone at the shooting point facing the wind turbine, and the second image to be tested is input into the target detection model to obtain second pixel information of the target wind turbine structure, and the blade phase angle is determined according to the second pixel information, including:

[0042] Control the drone to maintain the corrected posture and fly to a shooting point facing the wind turbine, and use an onboard camera carried by the drone to shoot the second image to be tested;

[0043] Performing target detection on the second image to be tested by using the target detection model, extracting the wind turbine with the largest area within the field of view of the second image to be tested, and extracting second pixel information of the target wind turbine structure from the wind turbine image of the wind turbine with the largest area; the second pixel information includes pixel information of the hub and blades;

[0044] Based on the relative position relationship between the blade and the hub, the pixel coordinates of the i-th blade tip (u i ,v i ), and obtain the center line of the i-th blade; where u iis the horizontal coordinate of the pixel coordinate of the i-th leaf tip, v i is the vertical coordinate of the pixel coordinate of the i-th leaf tip;

[0045] The measured phase angle β of the i-th blade is calculated by the slope i , the phase angle weight method is used to calculate the measured phase angle to obtain the final blade phase angle β i '; The blade phase angle β i The calculation formula of ' is:

[0046] Preferably, based on a preset simplified wind turbine model, determining the rotor pitch angle and the blade cone angle according to the blade phase angle includes:

[0047] The phase angle-elevation angle model is used to determine the wind rotor plane as a circular plane that has rotated;

[0048] Based on the least square method, an ellipse is fitted to the circular plane according to the pixel coordinates of the blade tip and the center coordinates of the hub to obtain an ellipse surface;

[0049] Calculate the elevation angle α of the fan according to the elliptical surface;

[0050] The phase angle-elevation angle-cone angle model is used to determine the cone angle γ, elevation angle α, and blade phase angle β i ' and tilt angle θ i The relationship between cone angle γ, elevation angle α, blade phase angle β i ' and tilt angle θ i The relationship formula is: i =arctan(cos(β i ')·tan(-α))+γ;

[0051] The pixel coordinates of the leaf tip (u i ,v i ) to find the pixel length uv l i , and converted to the projection length in the world coordinate system W l i , we get the formula cos(|θ i |)= W l i / L; where L is the blade length;

[0052] Simultaneous formula θ i =arctan(cos(β i ')·tan(-α))+γ and cos(|θ i |)= W l i / L, and obtain the equations corresponding to the three leaves;

[0053] Based on the equations corresponding to the three blades, the elevation angle α', cone angle γ' and tilt angle θ with the minimum error are fitted by the least squares method. i '; where θ i The positive and negative values ​​of the elevation angle and cone angle are determined by the initial guess values ​​during the fitting process. The calculated demarcation phase angle β bound To determine; the boundary phase angle β bound The formula is: θ i The positive and negative determination formula is:

[0054] The elevation angle α' and the cone angle γ' that are fitted with the minimum error are determined as the wind wheel elevation angle and the blade cone angle.

[0055] Preferably, the blade tip is determined according to the blade phase angle, the rotor pitch angle and the blade cone angle, and the final corrected coordinates of the blade tip are determined according to the blade tip, the blade phase angle and a coordinate system conversion matrix including the yaw angle of the wind turbine, including:

[0056] When the phase angle-elevation angle model is used, the blade cone angle is 0°, and the calculated blade tilt angle θ i The formula is: i =arctan(cos(β i ')·tan(-α));

[0057] When the phase angle-elevation angle-cone angle model is used, the calculated blade tilt angle θ is i The formula is: i =arctan(cos(β i ')·tan(-α))+γ;

[0058] The final corrected coordinate P of the blade tip is determined based on the obtained blade tip, the blade phase angle, and the coordinate system conversion matrix including the yaw angle of the wind turbine. i ”=( W x i ”, W y i ”, W z i ")for: in, is the transformation matrix from the world coordinate system to the wind turbine coordinate system, W x" is the X-axis coordinate of the corrected coordinates, W y" is the Y-axis coordinate of the corrected coordinates, W z" is the Z-axis coordinate of the corrected coordinates.

[0059] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0060] The present invention provides a method for visual positioning of a wind turbine blade tip based on multi-parameter correction, comprising: collecting image information of a target wind turbine, and performing data annotation and model training on the image information to obtain a trained target detection model; the image information includes a panoramic view and a local view of the target wind turbine; determining a top view shooting point of a drone according to structural parameters of the target wind turbine; shooting a first image to be tested by a drone at the top view shooting point, and inputting the first image to be tested into the target detection model to obtain first pixel information of the target wind turbine structure, and calculating the yaw information of the target wind turbine according to the first pixel information of the target wind turbine structure; the yaw information includes a yaw angle of the wind turbine; performing position correction and attitude correction on the posture of the drone according to the yaw information ; According to the corrected attitude, the drone is controlled to move to the wind turbine front view shooting point, and the second image to be tested is shot by the drone at the wind turbine front view shooting point, and the second image to be tested is input into the target detection model to obtain the second pixel information of the target wind turbine structure, and the blade phase angle is determined according to the second pixel information; Based on the preset simplified model of the wind turbine, the wind rotor pitch angle and the blade cone angle are determined according to the blade phase angle; The simplified model of the wind turbine includes: an ideal model of the wind turbine, a phase angle-elevation angle model and a phase angle-elevation angle-cone angle model; The blade tip is determined according to the blade phase angle, the wind rotor pitch angle and the blade cone angle, and the final corrected coordinates of the blade tip are determined according to the blade tip, the blade phase angle and the coordinate system conversion matrix containing the wind turbine yaw angle. , The present invention uses the wind turbine top view to locate the wind turbine direction, and can accurately and quickly obtain the wind turbine yaw information while maintaining a certain safety distance. The redundant calculation mechanism can avoid the large error caused by the possible occlusion relationship between the wind turbine components, and has more stability. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0062] Figure 1 A flow chart of a method provided by an embodiment of the present invention;

[0063] Figure 2 The model and geometric parameter diagram of the wind turbine provided by the embodiment of the present invention; wherein, Figure 2 (a) is the front view of the “inverted Y-shaped” reverse propeller of the wind turbine; Figure 2 (b) is the side view of the “inverted Y-shaped” wind turbine with feathered propellers;

[0064] Figure 3 An overall flow chart of the solution provided by an embodiment of the present invention;

[0065] Figure 4 Schematic diagram of two yaw angle measurement methods provided in embodiments of the present invention; Figure 4 (a) is the hub-nacelle centerline method, Figure 4 (b) is the blade slope weight method;

[0066] Figure 5 A schematic diagram of posture correction provided by an embodiment of the present invention;

[0067] Figure 6 A simplified model diagram of a wind turbine provided by an embodiment of the present invention;

[0068] Figure 7 A schematic diagram of the relationship between the cone angle, elevation angle, phase angle and tilt angle provided in an embodiment of the present invention;

[0069] Figure 8 A schematic diagram of the coordinate system conversion relationship provided by an embodiment of the present invention; wherein, Figure 8 (a) is a schematic diagram of the pixel-world coordinate system transformation relationship; Figure 8 (b) is a schematic diagram of the transformation relationship between the fan and other coordinate systems. DETAILED DESCRIPTION

[0070] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0071] The purpose of the present invention is to provide a fan blade tip visual positioning method based on multi-parameter correction, which can avoid the large errors caused by possible occlusion relationships between fan components and has greater stability.

[0072] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0073] Figure 1 A flow chart of a method provided by an embodiment of the present invention, such as Figure 1 As shown, the present invention provides a fan blade tip visual positioning method based on multi-parameter correction, comprising:

[0074] Step 100: collecting image information of the target wind turbine, and performing data annotation and model training on the image information to obtain a trained target detection model; the image information includes a panoramic view and a local view of the target wind turbine;

[0075] Step 200: determining a top view shooting point of the UAV according to the structural parameters of the target wind turbine;

[0076] Step 300: photographing a first image to be tested by a drone at a top view shooting point, and inputting the first image to be tested into a target detection model to obtain first pixel information of a target wind turbine structure, and calculating yaw information of the target wind turbine according to the first pixel information of the target wind turbine structure; the yaw information includes a yaw angle of the wind turbine;

[0077] Step 400: Correct the position and attitude of the UAV according to the yaw information;

[0078] Step 500: Control the drone to move to the wind turbine front view shooting point according to the corrected posture, and shoot a second image to be tested at the wind turbine front view shooting point by the drone, and input the second image to be tested into the target detection model to obtain second pixel information of the target wind turbine structure, and determine the blade phase angle according to the second pixel information;

[0079] Step 600: Based on a preset simplified wind turbine model, determine the wind rotor pitch angle and the blade cone angle according to the blade phase angle; the simplified wind turbine model includes: an ideal wind turbine model, a phase angle-pitch angle model and a phase angle-pitch angle-cone angle model;

[0080] Step 700: determine the blade tip according to the blade phase angle, the rotor pitch angle and the blade cone angle, and determine the final corrected coordinates of the blade tip according to the blade tip, the blade phase angle and a coordinate system conversion matrix including the wind turbine yaw angle.

[0081] Specifically, the geometric parameter diagram of the wind turbine is as follows: Figure 2 As shown, the wind turbine is roughly composed of four parts: a hub, a tower, a nacelle, and blades. The wind turbine parameters involved in the present invention include the rotor axis, the blade phase angle (the angle between the blade centerline and the phase angle reference line, hereinafter referred to as the "phase angle"), the rotor elevation angle (the angle between the rotor axis and the horizontal plane, hereinafter referred to as the "elevation angle"), the blade cone angle (the inclination angle of the blade centerline relative to the plane perpendicular to the rotor axis, hereinafter referred to as the "cone angle"), the blade warp angle (the angle between the blade centerline and the vertical plane, hereinafter referred to as the "warp angle"), etc.

[0082] Specifically, Figure 3 As shown, the steps of the solution of this embodiment are as follows:

[0083] Step 1: Dataset creation

[0084] 1-1 Image acquisition

[0085] Use high-definition visible light cameras to capture panoramic and local images of wind turbines;

[0086] 1-2 Data Annotation

[0087] The labeled data set includes five categories: wind turbine as a whole, hub, nacelle, wind turbine blades, and tower;

[0088] 1-3 Data Training

[0089] Configure the lightweight YOLOv5 algorithm and perform data set training.

[0090] Step 2: Orient the top view

[0091] 2-1 Algorithm Configuration

[0092] The lightweight YOLOv5 algorithm and data set described in step 1 are configured into the drone; exemplary, since this method needs to be deployed on the airborne computing board on the drone side, its computing power is limited, so in the target detection module, the present invention adopts an improved lightweight YOLOv5 algorithm, which can be replaced by YOLOv8 Nano and YOLOv8n algorithms.

[0093] 2-2 Determine the overlooking shooting point

[0094] In the wind farm, for the center height h of the wind turbine hub and the center coordinates of the wind turbine tower ( W x t , W y t ) has no accurate value, so the horizontal coordinate of the wind turbine tower center is regarded as the rough coordinate of the hub center. According to the known hub center coordinate position and the horizontal field of view angle θ of the airborne camera hfov and vertical field angle θ vfov , set a top view shooting point ( W x t , W y t ,h+d), formula (1) is the constraint condition that the point needs to satisfy, where d is the straight-line distance between the drone and the hub center, d s is the minimum safe distance between the UAV and the wind turbine, d vfov and d hfov The minimum image shooting distance to cover the entire wind turbine under fixed horizontal and vertical field of view, h cam and w cam are the height and width of the field of view, and L is the length of the blade.

[0095]

[0096] 2-3 performs target detection and feature extraction on the image obtained in 2-2.

[0097] After the drone flies to the set point, it uses the onboard camera to detect the wind turbine through the YOLOv5 algorithm, and extracts the pixel information of the hub, nacelle, and blades based on the pre-trained model to obtain the yaw information of the wind turbine.

[0098] 2-4 Calculate wind turbine yaw information

[0099] Among the current mainstream algorithms, many algorithms rely on the hub or cabin of the wind turbine to calculate the yaw angle of the wind turbine. However, once the hub or cabin of the wind turbine is blocked or the pixel ratio is too small to be recognized, the pixel information cannot be updated in time, and eventually the UAV deviates from the adjusted position. Therefore, in view of this situation, the present invention discusses the visibility of the hub and cabin of the wind turbine in the top view in the process of obtaining the yaw information of the wind turbine, and develops a set of redundant calculation mechanisms for the yaw angle, such as Figure 4 As shown in the figure (the green background in the figure is a simulation of the grass in the simulation environment, which is more in line with the actual situation), the specific mechanism is as follows:

[0100] 1) When both the hub and the nacelle are visible, use the hub-nacelle centerline method (e.g. Figure 4 (a) shows that, according to the pixel coordinate values ​​of the hub center point O and the cabin center point M extracted by detection, the vector Unit vector in the negative direction of v-axis and u-axis of pixel coordinate system and According to the positional relationship between the wind wheel axis and the u-axis and v-axis in the pixel coordinate system, the wind turbine yaw angle yaw can be calculated. f for:

[0101]

[0102] 2) On the contrary, the blade slope weight method (such as Figure 4 (b)). After obtaining the pre-selected frame of the leaf through detection and extraction, this method performs Hough line detection on the image in the frame to obtain j detected line segments, and then calculates their pixel length l j , and the length coefficient of each blade and its slope k j The product weighted value of is fitted to a straight line with a slope of k0, and the axis of the wind wheel is the perpendicular line of the straight line, then we can get:

[0103]

[0104] Step 3: Correct the drone’s posture

[0105] After completing step 2, based on the yaw information of the wind turbine obtained in steps 2-4, position correction is performed to ensure that the drone is directly above the center of the hub and facing the wind turbine. At this time, the horizontal coordinates of the center of the hub and the precise yaw information of the wind turbine can be determined; and in view of the visual errors that may be caused by YOLOv5, the correction buffer time (hovering time after each correction) and output limit are set for the posture corrector. The buffer time is set to 5s, the correction of the yaw angle is limited to (-π,π], and the correction value of the displacement is limited to ±10m. In the buffer state, the posture of the next correction is calculated synchronously.

[0106] 3-1 Position Correction

[0107] like Figure 5 As shown in the figure, the pixel coordinates of the center point of the identified wheel hub are obtained through target detection (u hub ,v hub ), let the coordinates of the center point of the pixel plane be (u0, v0), and the pixel difference between the two points is the correction value △u and △v of the drone in the pixel coordinate system:

[0108]

[0109] The correction values ​​△x and △y in the world coordinate system are calculated by coordinate system conversion. When both △x and △y are less than 1m, it is considered that the UAV is directly above the center of the wheel hub and no correction is required. The current horizontal coordinate of the UAV in the world coordinate system ( W x0, W y0) is the exact horizontal coordinate of the hub center.

[0110] 3-2 Posture Correction

[0111] The yaw angle of the wind turbine calculated at the current drone position f As the reference value, the updated yaw angle of the drone can be obtained. d for:

[0112]

[0113] After obtaining the updated value, the drone performs yaw correction and sets the yaw angle error threshold to ±5°, that is, when ||yaw d -yaw f When |-π|≤0.0873, it is considered that the UAV and the wind turbine are facing each other and no attitude angle correction is required. f This is the accurate wind turbine yaw information. Repeat steps 3-1 and 3-2 until the position and posture of the drone are within the allowable error range.

[0114] Step 4: Obtain fan parameters

[0115] 4-1 Construction of simplified fan model

[0116] The comparison of three simplified fan models is shown in the figure below: Figure 6 As shown, Figure 6 In the figure, α is the elevation angle, γ is the cone angle, model 411 is the currently common ideal model, model 412 is the phase angle-elevation angle model, and model 413 is the phase angle-elevation angle-cone angle model.

[0117] 4-2 Blade phase angle acquisition

[0118] The drone keeps the attitude angle obtained in step 3 unchanged and flies to the shooting point facing the wind turbine. The wind turbine with the largest area in the field of view is extracted through target detection. The pixel information of the hub and blades is extracted from the wind turbine image. Based on the relative position relationship between the blade and the hub, the pixel coordinates (u i ,v i ), and obtain the center line of the i-th blade (the line connecting the i-th blade tip and the blade root), and calculate the measured phase angle β of the i-th blade through the slope i In order to reduce the visual error, this paper uses the phase angle weight method to calculate the final phase angle β i 'As shown in formula (6):

[0119]

[0120] 4-3 Obtaining the wind rotor pitch angle and blade cone angle

[0121] 1) When model 412 is used, the cone angle is considered to be 0°, the wind wheel plane can be approximately regarded as a rotated circular plane, and the front view is an elliptical surface, which can be obtained by the pixel coordinates of the three blade tip points (u i ,v i ) and the hub center coordinates are obtained by least squares fitting. Figure 7 It can be seen that at this time, the projected length of the blade at the position of the phase angle of 0° is equal to the actual length (i.e., the major axis of the ellipse), and at the position of the phase angle of 90°, the projected length of the blade is equal to the actual length multiplied by the cosine value of the elevation angle (i.e., the minor axis of the ellipse). uv m, short axis uv n is converted to the world coordinate system W m and W n, so the elevation angle α of the fan can be calculated.

[0122] 2) When using model 413, cone angle γ, elevation angle α, phase angle β i ' and tilt angle θ i The relationship is as Figure 6 As shown, we can see that the relationship is:

[0123] θi =arctan(cos(β i ')·tan(-α))+γ (7)

[0124] The pixel coordinates of the three leaf tip points (u i ,v i ) to find its pixel length uv l i , and converted to the projection length in the world coordinate system W l i , we can know that:

[0125] cos(|θ i |)= W l i / L (8)

[0126] Combining formulas (7) and (8), we get the equations corresponding to the three blades. Based on these three equations, we fit the minimum error of the elevation angle α', cone angle γ' and tilt angle θ by the least squares method. i '. Among them, θ i The positive and negative values ​​of the elevation angle and cone angle are determined by the initial guess values ​​during the fitting process. The calculated demarcation phase angle β bound To determine:

[0127]

[0128] Step 5: Multi-parameter correction to achieve precise positioning

[0129] 5-1 Calculation of blade warp angle

[0130] 1) When the 412 model is used, the blade cone angle is 0°. Figure 7 The warp angle θ can be calculated i for:

[0131] θ i =arctan(cos(β i ')·tan(-α)) (11)

[0132] 2) When using the 413 model, the warping angle can be calculated as θ i :

[0133] θ i =arctan(cos(β i ')·tan(-α))+γ (12)

[0134] Here is still θ i , because in the original formula θ i =arctan(cos(β iIn the equation ')·tan(-α))+γ, for the phase angle-elevation angle model, the blade cone angle γ is 0, so there is no such term here.

[0135] 5-2 Blade tip coordinate calculation

[0136] According to the three parameters obtained and the coordinate system transformation matrix including the wind turbine yaw angle, the final blade tip correction coordinate P″ can be obtained. i =( W x″ i , W y″ i , W z″ i )for:

[0137]

[0138] Furthermore, the coordinate system transformation relationship diagram is as follows: Figure 8 As shown in Figure 2, after the drone completes target detection and feature extraction, Figure 8 As shown in (a), the coordinate transformation from the pixel coordinate system to the world coordinate system can be realized through the world coordinate system {W}, the body coordinate system {D}, the camera coordinate system {C}, the image coordinate system {xy}, and the pixel coordinate system {uv}. The transformation relationship is shown in formula (14):

[0139] After the drone completes target detection and feature extraction, Figure 8 As shown in (a), the coordinate transformation from the pixel coordinate system to the world coordinate system can be realized through the world coordinate system {W}, the body coordinate system {D}, the camera coordinate system {C}, the image coordinate system {xy}, and the pixel coordinate system {uv}. The transformation relationship is:

[0140]

[0141] Among them, (u, v) is the point in the pixel coordinate system, C Z is the scaling factor; f is the focal length, dx and dy represent the width of a pixel in the x and y directions respectively, and (u0, v0) is the center of the pixel plane; is the rotation matrix and translation matrix from the body coordinate system to the camera coordinate system, Determined by the gimbal attitude angle; The rotation matrix and translation matrix from the world coordinate system to the body coordinate system are determined by the current quaternion and position of the drone respectively; ( W X, W Y, W Z) is the coordinate point in the world coordinate system.

[0142] When processing images, define the fan coordinate system {F} that complies with the "right-hand rule", such as Figure 8As shown in (b), the hub center is taken as the coordinate origin, the horizontal direction from the nacelle to the hub is the positive direction of the x-axis, and the vertical upward is the positive direction of the z-axis. The conversion relationship is:

[0143]

[0144] in, is the transformation matrix from the wind turbine coordinate system to the aircraft coordinate system, which is determined by the posture of the UAV and the wind turbine; It is the transformation matrix from the world coordinate system to the fan coordinate system.

[0145] Specifically, the key processes of this embodiment include:

[0146] 1. First, a top view of the wind turbine is obtained through a drone, and information on components such as the hub, nacelle, and blades is obtained through target detection and feature extraction. When the nacelle and hub are visible, the hub-nacelle centerline method is used to determine the yaw angle of the wind turbine by relying on the center line vector of the pre-selected boxes of the two components; when the nacelle and hub are not fully visible, the blade slope method is used to detect the line segments in the blade pre-selected box through Hough line detection and calculate the average slope; after obtaining the yaw angle, the drone performs posture correction based on the visual image feedback, so that the drone is finally located directly above the wind turbine and facing the wind turbine.

[0147] 2. Construction of simplified wind turbine model: The previous simplified wind turbine model ignored the cone angle and elevation angle, and the simplified wind turbine model was a vertical plane. This embodiment proposes two simplified wind turbine models (introducing the elevation angle, introducing the cone angle and elevation angle).

[0148] 3. Methods for calculating the wind turbine rotor elevation angle and blade cone angle: For the elevation angle model, the elevation angle can be obtained by fitting an ellipse using the least squares method through the front view; for the cone angle elevation angle model, the elevation angle and cone angle can be fitted using the least squares method based on the ratio of the projection values ​​of the three blades to the actual values.

[0149] 4. Multi-parameter corrected blade tip positioning method: The actual and precise positions of the three blade tips of the fan can be solved by relying on the fan yaw angle, blade phase angle, wind wheel pitch angle, blade cone angle and the actual length of the fan blade.

[0150] In summary, the present invention proposes a visual positioning method for the tip of a wind turbine blade based on multi-parameter correction. First, the UAV uses the position information of the wind turbine to reach the overlooking point, and performs target detection and feature extraction on the wind turbine through the YOLOv5 algorithm. According to the positional relationship between the hub and the nacelle, a redundant calculation mechanism for obtaining the yaw information of the wind turbine is designed; secondly, the UAV relies on visual feedback based on the overhead view to perform posture correction, thereby solving and heading to the front view point of the wind turbine; then, the front view is used to complete the preliminary positioning of the blade tip and calculate the three correction parameters of the blade phase angle, wind wheel pitch angle, and blade cone angle; finally, a simplified model of the wind turbine is constructed and combined with the correction parameters to achieve precise positioning. Experiments have shown that compared with existing methods, the measurement error of the wind turbine yaw information can be reduced by 51.39%, and the average error of the three-axis positioning of the blade tip can be reduced by 57.57%, providing accuracy guarantee for the inspection route planning of the wind turbine.

[0151] The beneficial effects of the present invention are as follows:

[0152] (1) The present invention uses a top view of the wind turbine to locate the direction of the wind turbine. While maintaining a certain safety distance, it can accurately and quickly obtain the wind turbine yaw information. The redundant calculation mechanism can avoid the large errors caused by the possible obstruction relationship between the wind turbine components, and it is more stable.

[0153] (2) The present invention adopts a multi-parameter fusion method to accurately locate the blade tip by measuring the wind turbine yaw angle, wind rotor pitch angle, blade phase angle, and blade cone angle. The error can reach sub-meter level, providing accuracy guarantee for subsequent inspection path planning.

[0154] (3) The simplified models of the three types of fans constructed by the present invention can be flexibly applied to fans with horizontal axes and inclined axes, and have good universality.

[0155] (4) The present invention adopts the lightweight YOLOv5 algorithm to perform target detection and feature extraction on the entire wind turbine, which not only meets the accuracy requirements but also has good real-time performance, thus saving a lot of time in the inspection process.

[0156] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0157] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A fan blade tip visual positioning method based on multi-parameter correction, characterized in that: include: Collecting image information of a target wind turbine, and performing data annotation and model training on the image information to obtain a trained target detection model; the image information includes a panoramic view and a partial view of the target wind turbine; Determine the top view shooting point of the UAV according to the structural parameters of the target wind turbine; A first image to be tested is photographed by a drone at the top view shooting point, and the first image to be tested is input into the target detection model to obtain first pixel information of the target wind turbine structure, and yaw information of the target wind turbine is calculated according to the first pixel information of the target wind turbine structure; the yaw information includes a yaw angle of the wind turbine; Performing position correction and attitude correction on the posture of the UAV according to the yaw information; Control the drone to move to a shooting point facing the wind turbine according to the corrected posture, shoot a second image to be tested at the shooting point facing the wind turbine through the drone, input the second image to be tested into the target detection model, obtain second pixel information of the target wind turbine structure, and determine the blade phase angle according to the second pixel information; Based on a preset simplified wind turbine model, the wind rotor pitch angle and the blade cone angle are determined according to the blade phase angle; The simplified wind turbine model includes: an ideal wind turbine model, a phase angle-elevation angle model and a phase angle-elevation angle-cone angle model; The blade tip is determined according to the blade phase angle, the wind wheel pitch angle and the blade cone angle, and the final corrected coordinates of the blade tip are determined according to the blade tip, the blade phase angle and a coordinate system conversion matrix including the wind turbine yaw angle.

2. The fan blade tip visual positioning method based on multi-parameter correction according to claim 1 is characterized in that: The annotation categories of the data annotation include: wind turbine as a whole, hub, nacelle, wind turbine blades and tower.

3. The fan blade tip visual positioning method based on multi-parameter correction according to claim 1 is characterized in that: The target detection model is built based on the lightweight YOLOv5 algorithm.

4. The fan blade tip visual positioning method based on multi-parameter correction according to claim 1 is characterized in that: Determining the top view shooting point of the drone according to the structural parameters of the target wind turbine includes: Obtain the structural parameters of the target wind turbine and the field of view parameters of the airborne camera; the structural parameters include: the center height h of the wind turbine hub and the center coordinates of the wind turbine tower ( W x t , W y t );in, W x t is the horizontal coordinate value of the center coordinate of the wind turbine tower, W y t is the vertical coordinate value of the center coordinate of the wind turbine tower; the field of view parameters include: horizontal field of view angle θ hfov and vertical field angle θ vfov ; Determine the horizontal coordinates of the wind turbine tower center coordinates as the hub center rough coordinates; Determine the top view shooting point according to the hub center rough coordinates and the field of view parameters of the onboard camera ( W x t , W y t ,h+d); the formula of the constraint condition satisfied by the top view shooting point is: Where d is the straight-line distance between the drone and the hub center, d s is the minimum safe distance between the UAV and the wind turbine, d vfov and d hfov The minimum image shooting distance to cover the entire wind turbine under fixed horizontal and vertical field of view, h cam and w cam are the height and width of the field of view, and L is the length of the blade.

5. The fan blade tip visual positioning method based on multi-parameter correction according to claim 1 is characterized in that: A first image to be tested is photographed by a drone at the top view shooting point, and the first image to be tested is input into the target detection model to obtain first pixel information of a target wind turbine structure, and yaw information of the target wind turbine is calculated according to the first pixel information of the target wind turbine structure, including: After controlling the drone to fly to the position of the overhead view shooting point, use the onboard camera carried by the drone to shoot the first image to be tested; Performing target detection on the first image to be tested by using the target detection model to extract first pixel information of the target wind turbine structure; the first pixel information includes: pixel information of the hub, the nacelle and the blades; When both the hub and the nacelle are visible, the hub-nacelle centerline method is used to construct the vector according to the pixel coordinate values ​​of the hub center point O and the nacelle center point M extracted by detection. Unit vector in the negative direction of v-axis and u-axis of pixel coordinate system and The wind turbine yaw angle yaw is calculated based on the positional relationship between the wind wheel axis and the u-axis and v-axis in the pixel coordinate system. f for: When the hub and the nacelle are not visible, the blade slope weight method is used. After obtaining the pre-selected frame of the blade through detection and extraction, Hough line detection is performed on the image in the frame to obtain j detection line segments; Calculate the pixel length l of each detection line segment j , and the length coefficient of each blade and the slope k of the detection line segment j The product weighted value of is fitted to a straight line with a slope of k0, and the axis of the wind rotor is the perpendicular line of the straight line, then the yaw angle of the wind turbine yaw can be obtained f for: Wherein, N is a set of positive integers.

6. The fan blade tip visual positioning method based on multi-parameter correction according to claim 1 is characterized in that: The position and attitude of the UAV are corrected according to the yaw information, including: Obtain the pixel coordinates of the center point of the identified wheel hub through target detection (u hub ,v hub ), let the center point coordinates of the pixel plane be (u0, v0), and determine the pixel difference between the center point pixel coordinates and the center point coordinates of the pixel plane as the correction values ​​△u and △v of the drone in the pixel coordinate system; the calculation formula is: Among them, u hub is the horizontal coordinate value of the pixel coordinate of the center point of the wheel hub, v hub is the vertical coordinate value of the pixel coordinate of the center point of the hub, u0 is the horizontal coordinate value of the coordinate of the center point of the pixel plane, and v0 is the vertical coordinate value of the coordinate of the center point of the pixel plane; The correction values ​​△x and △y in the world coordinate system are calculated by coordinate system conversion. When △x and △y are both less than 1m, it is considered that the UAV is directly above the center of the wheel hub and no position correction is required. The current horizontal coordinate of the UAV in the world coordinate system ( W x0, W y0) is the precise horizontal coordinate of the hub center; where, W x0 is the horizontal coordinate value of the current world coordinate system of the drone. W y0 is the ordinate value of the current world coordinate system of the drone; The yaw angle of the wind turbine calculated at the current drone position f As the reference value, get the updated yaw value of the drone d for: After obtaining the updated value, the drone performs yaw correction and sets the yaw angle error threshold to ±5°, that is, when ||yaw d -yaw f When |-π|≤0.0873, it is considered that the UAV and the wind turbine are facing each other, and no attitude angle correction is required. f Determine the precise wind turbine yaw information; Repeat the steps of position correction and attitude correction until the position and attitude of the drone are within the allowable range of the error threshold.

7. The fan blade tip visual positioning method based on multi-parameter correction according to claim 1 is characterized in that: The drone is controlled to move to a shooting point facing the wind turbine according to the corrected posture, and a second image to be tested is shot by the drone at the shooting point facing the wind turbine, and the second image to be tested is input into the target detection model to obtain second pixel information of the target wind turbine structure, and the blade phase angle is determined according to the second pixel information, including: Control the drone to maintain the corrected posture and fly to a shooting point facing the wind turbine, and use an onboard camera carried by the drone to shoot the second image to be tested; Performing target detection on the second image to be tested by using the target detection model, extracting the wind turbine with the largest area within the field of view of the second image to be tested, and extracting second pixel information of the target wind turbine structure from the wind turbine image of the wind turbine with the largest area; the second pixel information includes pixel information of the hub and blades; Based on the relative position relationship between the blade and the hub, the pixel coordinates of the i-th blade tip (u i ,v i ), and obtain the center line of the i-th blade; where u i is the horizontal coordinate of the pixel coordinate of the i-th leaf tip, v i is the vertical coordinate of the pixel coordinate of the i-th leaf tip; The measured phase angle β of the i-th blade is calculated by the slope i , the phase angle weight method is used to calculate the measured phase angle to obtain the final blade phase angle β i '; The blade phase angle β i The calculation formula of ' is:

8. The fan blade tip visual positioning method based on multi-parameter correction according to claim 7 is characterized in that: Based on a preset simplified wind turbine model, the rotor pitch angle and the blade cone angle are determined according to the blade phase angle, including: The phase angle-elevation angle model is used to determine the wind rotor plane as a circular plane that has rotated; Based on the least square method, an ellipse is fitted to the circular plane according to the pixel coordinates of the blade tip and the center coordinates of the hub to obtain an ellipse surface; Calculate the elevation angle α of the fan according to the elliptical surface; The phase angle-elevation angle-cone angle model is used to determine the cone angle γ, elevation angle α, and blade phase angle β i ' and tilt angle θ i The relationship between cone angle γ, elevation angle α, blade phase angle β i ' and tilt angle θ i The relationship formula is: i =arctan(cos(β i ')·tan(-α))+γ; The pixel coordinates of the leaf tip (u i ,v i ) to find the pixel length uv l i , and converted to the projection length in the world coordinate system W l i , we get the formula cos(|θ i |)= W l i / L; where L is the blade length; Simultaneous formula θ i =arctan(cos(β i ')·tan(-α))+γ and cos(|θ i |)= W l i / L, and obtain the equations corresponding to the three leaves; Based on the equations corresponding to the three blades, the elevation angle α', cone angle γ' and tilt angle θ with the minimum error are fitted by the least squares method. i '; where θ i The positive and negative values ​​of the elevation angle and cone angle are determined by the initial guess values ​​during the fitting process. The calculated demarcation phase angle β bound To determine; the boundary phase angle β bound The formula is: θ i The positive and negative determination formula is: The elevation angle α' and the cone angle γ' that are fitted with the minimum error are determined as the wind wheel elevation angle and the blade cone angle.

9. The fan blade tip visual positioning method based on multi-parameter correction according to claim 8 is characterized in that: The blade tip is determined according to the blade phase angle, the wind rotor pitch angle and the blade cone angle, and the final corrected coordinates of the blade tip are determined according to the blade tip, the blade phase angle and a coordinate system conversion matrix including the wind turbine yaw angle, including: When the phase angle-elevation angle model is used, the blade cone angle is 0°, and the calculated blade tilt angle θ i The formula is: i =arctan(cos(β i ')·tan(-α)); When the phase angle-elevation angle-cone angle model is used, the calculated blade tilt angle θ is i The formula is: i =arctan(cos(β i ')·tan(-α))+γ; The final blade tip correction coordinate P" is determined based on the obtained blade tip, blade phase angle, and coordinate system conversion matrix including the wind turbine yaw angle. i =( W x″ i , W y″ i , W z″ i )for: in, is the transformation matrix from the world coordinate system to the wind turbine coordinate system, W x" is the X-axis coordinate of the corrected coordinates, W y" is the Y-axis coordinate of the corrected coordinates, W z" is the Z-axis coordinate of the corrected coordinates.

Images