A wind turbine nacelle orientation recognition calculation method and system based on unmanned aerial vehicle vision

Abstract

The application provides a fan nacelle orientation identification calculation method and system based on unmanned aerial vehicle vision, relates to the unmanned aerial vehicle application field, and comprises the following steps: pre-configuring the position relationship between an unmanned aerial vehicle and a fan nacelle, obtaining image data of the fan nacelle, and performing size normalization processing on the image data to obtain a first image; a YOLO-OBB detection model is used to perform screening processing on the first image to obtain a second image, and based on the second image, the angle of the fan nacelle in a world coordinate system is calculated. The application can process more extensive wind farm scenes, the calculation of the fan nacelle orientation can be realized by detecting the nacelle and the hub, the fan nacelle orientation can also be calculated by detecting the nacelle and the blade, the error caused by the change of the blade posture is reduced, seamless connection with a flight path planning module, a direction calibration module and the like is facilitated, and the real-time and stability requirements of unmanned aerial vehicle inspection are met.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) applications, and more specifically, to a method and system for wind turbine nacelle orientation recognition and calculation based on UAV vision. Background Technology

[0002] The nacelle of a wind turbine is located at the top of the tower and is an important component for the operation and maintenance of the wind turbine. For UAVs in tasks such as wind turbine inspection, nacelle orientation identification and positioning, obtaining the precise shape and specific location of the nacelle in the image is a key step for subsequent measurement and flight control.

[0003] In existing technologies, nacelle shape extraction typically relies on traditional image processing methods, such as grayscale thresholding, edge detection, and morphological operations. These methods have several limitations. Traditional image segmentation methods are highly sensitive to factors such as lighting conditions, shooting angle, and background complexity. During outdoor wind turbine inspections, uncontrollable lighting variations, such as backlighting, shadows, strong reflections, and background interference from the sky, blades, and clouds, can lead to unstable segmentation results and even errors. Because nacelles of different models, colors, and materials vary significantly in appearance, traditional algorithms often require adjustments to algorithm parameters, such as thresholds and filter kernel sizes, for specific turbine models, resulting in poor generalization ability across different scenes and turbine types. Furthermore, when wind turbine blades pass over the nacelle or at certain junctions, traditional methods are prone to contour discontinuities and local gaps, severely impacting the accuracy of subsequent orientation calculations and position estimations based on the nacelle shape.

[0004] Furthermore, some methods choose not to directly extract the nacelle outline from images. Instead, they fly a drone directly above the nacelle to capture images from a top-down perspective, obtaining the nacelle's planar shape and fitting a straight line representing the turbine blades to calculate the turbine's orientation. However, this approach also has significant drawbacks. In complex wind fields, accurately flying the drone directly above the nacelle's center point requires precise nacelle GPS positioning and navigation planning, and the drone's own GPS drift can also affect accuracy.

[0005] No effective solutions have yet been proposed to address the problems in the relevant technologies. Summary of the Invention

[0006] In view of this, the present invention provides a method and system for wind turbine nacelle orientation recognition calculation based on UAV vision, in order to solve the aforementioned problems.

[0007] To solve the above problems, the specific technical solution adopted by the present invention is as follows:

[0008] According to one aspect of the present invention, a method for wind turbine nacelle orientation recognition and calculation based on UAV vision is provided, comprising the following steps:

[0009] S1. Pre-configure the positional relationship between the UAV and the wind turbine nacelle; the positional relationship includes the yaw angle of the UAV gimbal being the same as the yaw angle of the UAV body and the pitch angle of the UAV gimbal pointing downwards at 90°.

[0010] S2. Based on the world coordinate system, acquire image data of the wind turbine nacelle, and perform size normalization processing on the image data to obtain the first image; and use the YOLO-OBB detection model to filter the first image to obtain the second image;

[0011] S3. Based on the second image, divide the wind turbine nacelle into a rotating rectangle and use the rotating rectangle to calculate the position of the wind turbine nacelle in the world coordinate system.

[0012] S4. Based on the second image, divide the rotating rectangle of the wind turbine nacelle, the rotating rectangle of the hub, and the rotating rectangle of the three blades, and use the rotating rectangle of the wind turbine nacelle, the rotating rectangle of the hub, or the rotating rectangle of the three blades to calculate the orientation angle of the wind turbine nacelle in the pixel coordinate system.

[0013] S5. Based on the orientation angle of the wind turbine nacelle in the pixel coordinate system, the angle of the wind turbine nacelle in the world coordinate system is calculated.

[0014] Preferably, the process of acquiring image data of the wind turbine nacelle based on the world coordinate system, performing size normalization on the image data to obtain a first image, and then using the YOLO-OBB detection model to filter the first image to obtain a second image includes:

[0015] S21. Modify the height and width of the data image to the preset data, and normalize the pixel values ​​to the pixel range to determine the first image;

[0016] S22. Use a convolutional neural network algorithm to extract the feature map of the first image, and use a detection head to predict the feature map to obtain the candidate target result;

[0017] S23. The non-maximum suppression algorithm is used to filter the subsequent target results to obtain the second image.

[0018] Preferably, the step of dividing the wind turbine nacelle into a rotating bounding box based on the second image and calculating the position of the wind turbine nacelle in the world coordinate system using the rotating bounding box dataset includes:

[0019] S31. Using the drone's flight altitude and wheel hub height, calculate the mapping distance of the wheel hub on the optical axis in the camera coordinate system;

[0020] S32. Extract the center point of the rotating rectangle of the wind turbine nacelle, calculate the position of the center point of the wind turbine nacelle in the camera coordinate system using the camera intrinsic parameter matrix and optical axis mapping distance, and transform the center point of the wind turbine nacelle in the camera coordinate system to the UAV rigid body coordinate system.

[0021] S33. Based on the yaw angle of the UAV body, calculate the coordinates of the center point of the wind turbine nacelle in the world coordinate system with the UAV as the origin.

[0022] Preferably, the expression for calculating the position of the wind turbine nacelle center point in the camera coordinate system is:

[0023] ;

[0024] In the formula, Center point of the cabin The coordinates of the center point in the pixel coordinate system. d The distance mapped to the optical axis. Camera intrinsic parameter matrix The reverse, It is calculated Coordinates in the camera coordinate system;

[0025] The expression for the transformation of the wind turbine nacelle center point into the UAV rigid body coordinate system is as follows:

[0026] ;

[0027] In the formula, For the conversion The coordinates of the point in the rigid body coordinate system of the UAV.

[0028] Preferably, the expression for calculating the coordinates of the wind turbine nacelle center point in the world coordinate system is:

[0029] ;

[0030] In the formula, The yaw angle of the drone body. It is The coordinates of a point in the rigid body coordinate system of the UAV are transformed to obtain the coordinates in the world coordinate system.

[0031] Preferably, the step of dividing the wind turbine nacelle and hub into rotating rectangular frames based on the second image, and calculating the orientation angle of the wind turbine nacelle in the pixel coordinate system using the rotating rectangular frames of the wind turbine nacelle and hub, includes:

[0032] S41. Extract the center points of the rotating rectangular frames of the wind turbine nacelle and the hub, respectively.

[0033] S42. Construct a first vector pointing from the center point of the wind turbine nacelle to the center point of the hub in the pixel coordinate system, and use the first vector and the X-axis to calculate the orientation angle of the wind turbine nacelle in the pixel coordinate system.

[0034] Preferably, the step of dividing the wind turbine nacelle into rotating rectangular frames and three blades based on the second image, and calculating the orientation angle of the wind turbine nacelle in the pixel coordinate system using the rotating rectangular frames of the wind turbine nacelle and three blades, includes:

[0035] S41' Extract the coordinates of the midpoints of the two short sides of the rotating rectangle of the wind turbine nacelle and the coordinates of the four vertices of the rotating rectangle of the three blades;

[0036] S42' Based on the midpoints of the two short sides, construct the second and third vectors of the wind turbine nacelle in the pixel coordinate system. According to the coordinates of the four vertices, take the coordinates of the vertex closest to the center point of the wind turbine nacelle to construct the nearest vector. Calculate the angles between the second vector and the nearest vector and between the third vector and the nearest vector respectively to obtain a vector group. Use the second or third vector in the vector group to calculate the orientation angle of the wind turbine nacelle in the pixel coordinate system.

[0037] Preferably, the calculation of the angle of the wind turbine nacelle in the world coordinate system based on the orientation angle of the wind turbine nacelle in the pixel coordinate system includes:

[0038] S51. Based on the orientation angle of the wind turbine nacelle in the pixel coordinate system, calculate the angle of the F-axis in the rigid body coordinate system of the UAV.

[0039] S52. Calculate the angle between the yaw angle of the UAV body and the F-axis in the UAV rigid body coordinate system to obtain the angle of the wind turbine nacelle in the world coordinate system.

[0040] Preferably, the formula for calculating the angle of the F-axis in the rigid body coordinate system of the UAV is:

[0041] ;

[0042] In the formula, mod represents the modulo operation, indicating The remainder when divided by 360; α The orientation angle of the wind turbine nacelle in the pixel coordinate system.

[0043] The formula for calculating the angle of the wind turbine nacelle in the world coordinate system is as follows:

[0044] ;

[0045] In the formula, This refers to the yaw angle of the drone.

[0046] According to another aspect of the present invention, a wind turbine nacelle orientation recognition and calculation system based on UAV vision is provided, comprising:

[0047] The position adjustment module pre-configures the positional relationship between the UAV and the wind turbine nacelle; the positional relationship includes the yaw angle of the UAV gimbal being the same as the yaw angle of the UAV body and the pitch angle of the UAV gimbal pointing downwards at 90°.

[0048] The image processing module acquires image data of the wind turbine nacelle based on the world coordinate system, performs size normalization on the image data to obtain the first image, and uses the YOLO-OBB detection model to filter the first image to obtain the second image.

[0049] The nacelle position calculation module divides the wind turbine nacelle into rotating rectangular boxes based on the second image, and uses the rotating box dataset to calculate the position of the wind turbine nacelle in the world coordinate system.

[0050] The nacelle pixel angle calculation module, based on the second image, divides the rotating rectangle of the wind turbine nacelle, the rotating rectangle of the hub, and the rotating rectangle of the three blades, and uses the rotating rectangle of the wind turbine nacelle, the rotating rectangle of the hub, or the rotating rectangle of the three blades to calculate the orientation angle of the wind turbine nacelle in the pixel coordinate system.

[0051] The nacelle angle calculation module calculates the angle of the wind turbine nacelle in the world coordinate system based on the orientation angle of the wind turbine nacelle in the pixel coordinate system.

[0052] The beneficial effects of this invention are as follows:

[0053] 1. This invention can handle a wider range of wind turbine scenarios through the YOLO-OBB deep learning model. By detecting the nacelle and hub, the orientation of the wind turbine nacelle can be calculated. The orientation of the wind turbine nacelle can also be calculated by detecting the nacelle and blades.

[0054] 2. In the case of detecting only the nacelle and blades, the present invention generates candidate vectors from the center point of the nacelle and the midpoints of the two short sides, and uses the vertex of the blade root as a reference point, thereby reducing the error caused by changes in blade attitude.

[0055] 3. This invention ensures that the orientation calculation results fall directly under the NED world coordinate system by transforming the pixel coordinate vector with the rigid body coordinate system of the UAV. This facilitates seamless integration with modules such as trajectory planning and orientation calibration. The algorithm implementation relies only on target detection and geometric calculation, with low computational load. It is easy to deploy in embedded flight control systems and meets the real-time and stability requirements of UAV inspection. As long as the UAV flies within the range above the wind turbine and the field of view of the gimbal camera includes the wind turbine, the orientation of the wind turbine nacelle can be calculated. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly described below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:

[0057] Figure 1 This is a flowchart of a wind turbine nacelle orientation recognition and calculation method based on UAV vision according to an embodiment of the present invention;

[0058] Figure 2 This is a schematic diagram of a wind turbine nacelle orientation recognition and calculation system based on UAV vision according to an embodiment of the present invention:

[0059] Figure 3 This is a flowchart illustrating the practical application of a wind turbine nacelle orientation recognition and calculation method and system based on UAV vision according to an embodiment of the present invention.

[0060] Figure 4 This is the second image obtained in a wind turbine nacelle orientation recognition calculation method based on UAV vision according to an embodiment of the present invention;

[0061] Figure 5 This is one of the schematic diagrams for calculating the included angle of the wind turbine nacelle in the pixel coordinate system in a wind turbine nacelle orientation recognition calculation method based on UAV vision according to an embodiment of the present invention;

[0062] Figure 6 This is the second schematic diagram of calculating the included angle of the wind turbine nacelle in the pixel coordinate system in a wind turbine nacelle orientation recognition calculation method based on UAV vision according to an embodiment of the present invention;

[0063] Figure 7 This is a schematic diagram of the angle transformation of the wind turbine nacelle in the world coordinate system in a wind turbine nacelle orientation recognition and calculation method based on UAV vision according to an embodiment of the present invention.

[0064] In the picture:

[0065] 1. Position adjustment module; 2. Image processing module; 3. Cabin position calculation module; 4. Cabin pixel angle calculation module; 5. Cabin angle calculation module. Detailed Implementation

[0066] To enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this application.

[0067] According to an embodiment of the present invention, a method and system for wind turbine nacelle orientation recognition calculation based on UAV vision are provided.

[0068] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments, such as... Figures 1-7 As shown, the wind turbine nacelle orientation recognition and calculation method based on UAV vision according to an embodiment of the present invention includes:

[0069] S1. Pre-configure the positional relationship between the UAV and the wind turbine nacelle; the positional relationship includes the yaw angle of the UAV gimbal being the same as the yaw angle of the UAV body and the pitch angle of the UAV gimbal pointing downwards at 90°.

[0070] Specifically, control the drone to fly to any position within a preset altitude range above the cabin. At this position, adjust the yaw angle of the gimbal camera to match the yaw angle of the drone body, where the yaw angle is defined in the world coordinate system (NED). Then, control the gimbal's pitch angle to point downwards at 90°, making the camera's optical axis perpendicular to the ground. In this state, fine-tune the drone's horizontal position to bring the wind turbine fully within the gimbal camera's field of view.

[0071] It should be noted that the yaw angle of the drone and the yaw angle of the gimbal are the same. When the gimbal pitch angle is 90° vertically downwards, the pixel coordinates of the image frame of the gimbal camera are... The y-axis is aligned with the R-axis of the UAV's rigid body coordinate system FRD, the y-axis is opposite to the F-axis of the UAV's rigid body coordinate system, and the z-axis (optical axis) is aligned with the D-axis. Here, F represents the direction the UAV's head is facing, R represents the positive direction to the right of the UAV, and D represents the positive direction vertically downwards for the UAV.

[0072] S2. Based on the world coordinate system, acquire image data of the wind turbine nacelle, and perform size normalization processing on the image data to obtain the first image; and use the YOLO-OBB detection model to filter the first image to obtain the second image;

[0073] In a preferred embodiment, the steps of acquiring image data of the wind turbine nacelle based on the world coordinate system, performing size normalization on the image data to obtain a first image, and then using the YOLO-OBB detection model to filter the first image to obtain a second image include the following steps:

[0074] S21. Modify the height and width of the data image to the preset data, and normalize the pixel values ​​to the pixel range to determine the first image;

[0075] Specifically, the video stream acquired by the gimbal camera is extracted frame by frame, then the height and width of the image are modified to 640, and the pixel values ​​are normalized to the [0,1] range.

[0076] S22. Use a convolutional neural network algorithm to extract the feature map of the first image, and use a detection head to predict the feature map to obtain the candidate target result;

[0077] Specifically, the low-level convolutions are used to obtain low-level features such as edges and textures, the middle-level convolutions extract local structures and contours, the high-level convolutions aggregate global semantic information, the detection head processes the feature maps to obtain candidate target data, and the detection head outputs the position information, orientation angle, category and confidence of the candidate targets.

[0078] S23. Use the non-maximum suppression algorithm to filter the subsequent target results to obtain the second image;

[0079] Specifically, a non-maximum suppression method is used to filter candidate results. This method selects the detection boxes with the highest confidence value in the candidate boxes and removes redundant target boxes in turn according to the degree of overlap to obtain the second image.

[0080] It should be noted that the YOLO-OBB detection model introduces a rotated bounding box prediction mechanism, which outputs the target's position, size, and category information, as well as its orientation angle information. This allows for accurate description of wind turbine components with directional characteristics (such as blades, nacelles, and hubs). During the training phase, the model's parameters are optimized using supervised learning, enabling it to learn feature representations of different structural components at different angles and scales.

[0081] S3. Based on the second image, divide the wind turbine nacelle into a rotating rectangle and use the rotating rectangle to calculate the position of the wind turbine nacelle in the world coordinate system.

[0082] In a preferred embodiment, the step of dividing the wind turbine nacelle into a rotating rectangular frame based on the second image, and calculating the position of the wind turbine nacelle in the world coordinate system using the rotating rectangular frame, includes the following steps:

[0083] S31. Using the drone's flight altitude and wheel hub height, calculate the mapping distance of the wheel hub on the optical axis in the camera coordinate system;

[0084] Specifically, the difference between the drone's flight altitude and the wheel hub height is used to obtain the mapped distance of the wheel hub on the optical axis in the camera coordinate system.

[0085] S32. Extract the center point of the rotating rectangle of the wind turbine nacelle, calculate the position of the center point of the wind turbine nacelle in the camera coordinate system using the camera intrinsic parameter matrix and optical axis mapping distance, and transform the center point of the wind turbine nacelle in the camera coordinate system to the UAV rigid body coordinate system.

[0086] Specifically, the formula for expressing the camera intrinsic parameter matrix is:

[0087] ;

[0088] In the formula, K For the camera intrinsic parameter matrix, Let be the equivalent focal length of the camera in the horizontal direction. It is the camera's equivalent focal length in the vertical direction. and These are the x and y coordinates of the center point in the image pixel coordinate system.

[0089] The formula for calculating the position of the wind turbine nacelle center point in the camera coordinate system is:

[0090] ;

[0091] In the formula, Center point of the cabin The coordinates of the center point in the pixel coordinate system. d The value is the difference between the drone's altitude above the ground and the wind turbine nacelle's height. It is a matrix The reverse, It is calculated Coordinates in the camera coordinate system.

[0092] The expression for transforming the center point of the wind turbine nacelle into the rigid body coordinate system of the UAV is:

[0093] ;

[0094] In the formula, For the conversion The coordinates of the point in the rigid body coordinate system of the UAV.

[0095] S33. Based on the yaw angle of the UAV, and taking the UAV as the origin, calculate the coordinates of the wind turbine nacelle center point in the world coordinate system:

[0096] Specifically, the formula for calculating the coordinates of the center point of the wind turbine nacelle in the world coordinate system is as follows:

[0097] ;

[0098] In the formula, Yaw angle It is The coordinates of a point in the rigid body coordinate system of the UAV are transformed to obtain the coordinates in the world coordinate system.

[0099] S4. Based on the second image, divide the wind turbine nacelle into rotating rectangular frames, hub rotating rectangular frames, and three blade rotating rectangular frames. Using the rotating rectangular frames of the wind turbine nacelle, hub, and / or three blades, calculate the orientation angle of the wind turbine nacelle in the pixel coordinate system.

[0100] In a preferred embodiment, the step of dividing the wind turbine nacelle, hub, and three blades into rotating rectangular frames based on the second image, and calculating the orientation angle of the wind turbine nacelle in the pixel coordinate system using the rotating rectangular frames of the nacelle, hub, and / or three blades, includes the following steps:

[0101] S41. Extract the center points of the rotating rectangle frames of the wind turbine nacelle and the hub, respectively.

[0102] S42. Construct a first vector pointing from the center point of the wind turbine nacelle to the center point of the hub in the pixel coordinate system, and use the first vector and... X The axis calculates the orientation angle of the wind turbine nacelle in the pixel coordinate system.

[0103] Specifically, such as Figure 5 As shown, the center point of the rotating rectangular frame of the wind turbine nacelle (denoted as frame 1) is... The center point of the rotating rectangular frame of the wheel hub (denoted as frame 2) is Constructing a system in pixel coordinates point to The first vector , where the first vector The angle between the wind turbine nacelle and the X-axis represents the orientation of the nacelle in the pixel coordinate system. The formula for calculating the orientation angle of the wind turbine nacelle in the pixel coordinate system is:

[0104] ;

[0105] In the formula, The first vector and X The included angle of the axis, for coordinates for The coordinates.

[0106] S41' Extract the coordinates of the midpoints of the two short sides of the rotating rectangle of the wind turbine nacelle and the coordinates of the four vertices of the rotating rectangle of the three blades;

[0107] S42' Based on the midpoints of the two short sides, construct the second and third vectors of the wind turbine nacelle in the pixel coordinate system. According to the coordinates of the four vertices, take the coordinates of the vertex closest to the center point of the wind turbine nacelle to construct the nearest vector. Calculate the angles between the second vector and the nearest vector and between the third vector and the nearest vector respectively to obtain a vector group. Use the second or third vector in the vector group to calculate the orientation angle of the wind turbine nacelle in the pixel coordinate system.

[0108] Specifically, such as Figure 6 As shown, determine the midpoints of the two short sides of the rotating rectangular frame of the wind turbine nacelle (denoted as frame 1). and ,according to and Constructing the second vector and the third vector These two vectors are candidate vectors for the wind turbine nacelle's orientation in the pixel coordinate system, with one end pointing towards the front of the nacelle (such as the second vector). The other end points towards the tail of the cabin (such as the third vector). ).

[0109] Next, calculate the coordinates of the four vertices of the rotating rectangles of the three blades (denoted as rectangles 2, 3, and 4), and select a distance within each rotating rectangle. The nearest vertex is denoted as , and Since the blade root is connected to the nacelle and its position in the image is closest to the hub, the vertex closest to the hub is used for judgment to ensure accurate direction calculation. , , In the middle, take the distance For the nearest point, construct the nearest vector, as shown in the figure. The nearest vector is .

[0110] Finally, calculate the second vector. With the nearest vector The angle between The third vector and near vector The angle between This yields a vector group, and vectors with included angles less than 90° are retained, such as... Figure 6 As shown and A vector group, where the second vector and X The included angle of the axes is the orientation angle of the wind turbine nacelle in the pixel coordinate system.

[0111] The formula is:

[0112] ;

[0113] In the formula, For the second vector and X The included angle of the axis.

[0114] S5. Based on the orientation angle of the wind turbine nacelle in the pixel coordinate system, the angle of the wind turbine nacelle in the world coordinate system is calculated.

[0115] In a preferred embodiment, calculating the angle of the wind turbine nacelle in the world coordinate system based on the orientation angle of the wind turbine nacelle in the pixel coordinate system further includes the following steps:

[0116] S51. Based on the orientation angle of the wind turbine nacelle in the pixel coordinate system, calculate the angle of the F-axis in the rigid body coordinate system of the UAV.

[0117] S52. Calculate the angle between the yaw angle of the UAV body and the F-axis in the UAV rigid body coordinate system to obtain the angle of the wind turbine nacelle in the world coordinate system.

[0118] Specifically, based on the vector obtained in S4 (such as the first vector) (and based on the orientation angle of the wind turbine nacelle in the pixel coordinate system) Calculate the first vector The angle between the vector and the UAV's F-axis is calculated using the following formula:

[0119] ;

[0120] In the formula, mod represents the modulo operation, indicating The remainder when divided by 360.

[0121] Next, the angle of the wind turbine nacelle in the world coordinate system is calculated using the angle between the yaw angle and the F-axis. The formula is:

[0122] ;

[0123] In the formula, This refers to the yaw angle of the drone.

[0124] It should be noted that the yaw angle of the drone body It is based on the known angle data of the UAV under NED, which is the angle between the N-axis of the UAV's NED coordinate system and the F-axis of the UAV's rigid body coordinate system.

[0125] like Figure 2 As shown, according to another embodiment of the present invention, a wind turbine nacelle orientation recognition and calculation system based on UAV vision is provided, comprising:

[0126] Position adjustment module 1 pre-configures the positional relationship between the UAV and the wind turbine nacelle; the positional relationship includes the yaw angle of the UAV gimbal being the same as the yaw angle of the UAV body and the pitch angle of the UAV gimbal pointing downwards at 90°.

[0127] Image processing module 2 acquires image data of the wind turbine nacelle based on the world coordinate system, performs size normalization on the image data to obtain the first image, and uses the YOLO-OBB detection model to filter the first image to obtain the second image.

[0128] The nacelle position calculation module 3, based on the second image, divides the wind turbine nacelle into a rotating rectangular frame, and uses the rotating frame dataset to calculate the position of the wind turbine nacelle in the world coordinate system;

[0129] The nacelle pixel angle calculation module 4, based on the second image, divides the rotating rectangle of the wind turbine nacelle, the rotating rectangle of the hub, and the rotating rectangle of the three blades, and uses the rotating rectangle of the wind turbine nacelle, the rotating rectangle of the hub, or the rotating rectangle of the three blades to calculate the orientation angle of the wind turbine nacelle in the pixel coordinate system.

[0130] The nacelle angle calculation module 5 calculates the angle of the wind turbine nacelle in the world coordinate system based on the orientation angle of the wind turbine nacelle in the pixel coordinate system.

[0131] In summary, by utilizing the technical solutions described above, the YOLO-OBB deep learning model can handle a wider range of wind turbine scenarios. By detecting the nacelle and hub, the orientation of the wind turbine nacelle can be calculated; similarly, the orientation can be calculated using the detected nacelle and blades. When only the nacelle and blades are detected, candidate vectors are created from the nacelle center point and the midpoints of the two short sides, and the vertex at the blade root is used as a reference point, reducing errors caused by blade attitude changes. Through the transformation between pixel coordinate vectors and the UAV rigid body coordinate system, the orientation calculation results are ensured to fall directly in the NED world coordinate system, facilitating seamless integration with modules such as trajectory planning and orientation calibration. The algorithm relies solely on target detection and geometric calculations, resulting in low computational load and easy deployment in embedded flight control systems. It meets the real-time and stability requirements of UAV inspections. As long as the UAV flies within the range above the wind turbine and the gimbal camera's field of view includes the wind turbine, the orientation of the wind turbine nacelle can be calculated.

[0132] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0133] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for wind turbine nacelle orientation recognition and calculation based on UAV vision, characterized in that, include: S1. Pre-configure the positional relationship between the drone and the wind turbine nacelle; The positional relationship includes the yaw angle of the UAV gimbal being the same as the yaw angle of the UAV body, and the pitch angle of the UAV gimbal pointing downwards at 90°. S2. Based on the world coordinate system, acquire image data of the wind turbine nacelle, and perform size normalization processing on the image data to obtain the first image; and use the YOLO-OBB detection model to filter the first image to obtain the second image; S3. Based on the second image, divide the wind turbine nacelle into a rotating rectangle and use the rotating rectangle to calculate the position of the wind turbine nacelle in the world coordinate system. S4. Based on the second image, divide the wind turbine nacelle into rotating rectangular frames, hub rotating rectangular frames, and three-blade rotating rectangular frames. Using the rotating rectangular frames of the nacelle, hub, or three blades, calculate the orientation angle of the wind turbine nacelle in the pixel coordinate system. Specifically, this includes: S41' Extract the coordinates of the midpoints of the two short sides of the rotating rectangle of the wind turbine nacelle and the coordinates of the four vertices of the rotating rectangle of the three blades; S42' Based on the midpoints of the two short sides, construct the second and third vectors of the wind turbine nacelle in the pixel coordinate system. According to the coordinates of the four vertices, take the coordinates of the vertex closest to the center point of the wind turbine nacelle to construct the nearest vector. Calculate the angles between the second vector and the nearest vector and between the third vector and the nearest vector to obtain a vector group. Use the second or third vector in the vector group to calculate the orientation angle of the wind turbine nacelle in the pixel coordinate system. S5. Based on the orientation angle of the wind turbine nacelle in the pixel coordinate system, the angle of the wind turbine nacelle in the world coordinate system is calculated.

2. The method for wind turbine nacelle orientation recognition and calculation based on UAV vision according to claim 1, characterized in that, The image data of the wind turbine nacelle is acquired based on the world coordinate system, and the image data is normalized to obtain the first image. The YOLO-OBB detection model was used to filter the first image, resulting in the second image, which includes: S21. Modify the height and width of the data image to the preset data, and normalize the pixel values ​​to the pixel range to determine the first image; S22. Use a convolutional neural network algorithm to extract the feature map of the first image, and use a detection head to predict the feature map to obtain the candidate target result; S23. The non-maximum suppression algorithm is used to filter the subsequent target results to obtain the second image.

3. The method for wind turbine nacelle orientation recognition and calculation based on UAV vision according to claim 2, characterized in that, The step of dividing the wind turbine nacelle into a rotating bounding box based on the second image and calculating the position of the wind turbine nacelle in the world coordinate system using the rotating bounding box dataset includes: S31. Using the drone's flight altitude and wheel hub height, calculate the mapping distance of the wheel hub on the optical axis in the camera coordinate system; S32. Extract the center point of the rotating rectangle of the wind turbine nacelle, calculate the position of the center point of the wind turbine nacelle in the camera coordinate system using the camera intrinsic parameter matrix and optical axis mapping distance, and transform the center point of the wind turbine nacelle in the camera coordinate system to the UAV rigid body coordinate system. S33. Based on the yaw angle of the UAV body, calculate the coordinates of the center point of the wind turbine nacelle in the world coordinate system with the UAV as the origin.

4. The wind turbine nacelle orientation recognition and calculation method and system based on UAV vision according to claim 3, characterized in that, The expression for calculating the position of the wind turbine nacelle center point in the camera coordinate system is as follows: ； In the formula, Center point of the cabin The coordinates of the center point in the pixel coordinate system. d The distance mapped to the optical axis. Camera intrinsic parameter matrix The reverse, It is calculated Coordinates in the camera coordinate system; The expression for the transformation of the wind turbine nacelle center point into the UAV rigid body coordinate system is as follows: ； In the formula, For the conversion The coordinates of the point in the rigid body coordinate system of the UAV.

5. The method for wind turbine nacelle orientation recognition and calculation based on UAV vision according to claim 4, characterized in that, The expression for calculating the coordinates of the center point of the wind turbine nacelle in the world coordinate system is as follows: ； In the formula, The yaw angle of the drone body. It is The coordinates of a point in the rigid body coordinate system of the UAV are transformed to obtain the coordinates in the world coordinate system.

6. The method for wind turbine nacelle orientation recognition and calculation based on UAV vision according to claim 1, characterized in that, Based on the second image, the rotating bounding boxes of the wind turbine nacelle and the hub are defined. Using these rotating bounding boxes, the orientation angle of the wind turbine nacelle in the pixel coordinate system is calculated, including: S41. Extract the center points of the rotating rectangle frames of the wind turbine nacelle and the hub, respectively. S42. Construct a first vector pointing from the center point of the wind turbine nacelle to the center point of the hub in the pixel coordinate system, and use the first vector and the X-axis to calculate the orientation angle of the wind turbine nacelle in the pixel coordinate system.

7. The method for wind turbine nacelle orientation recognition and calculation based on UAV vision according to claim 1, characterized in that, The angle of the wind turbine nacelle in the world coordinate system, calculated based on its orientation angle in the pixel coordinate system, includes: S51. Based on the orientation angle of the wind turbine nacelle in the pixel coordinate system, calculate the angle of the F-axis in the rigid body coordinate system of the UAV. S52. Calculate the angle between the yaw angle of the UAV body and the F-axis in the UAV rigid body coordinate system to obtain the angle of the wind turbine nacelle in the world coordinate system.

8. The method for wind turbine nacelle orientation recognition and calculation based on UAV vision according to claim 7, characterized in that, The formula for calculating the angle of the F-axis in the rigid body coordinate system of the UAV is: ； In the formula, mod represents the modulo operation, indicating The remainder when divided by 360; α The orientation angle of the wind turbine nacelle in the pixel coordinate system; The formula for calculating the angle of the wind turbine nacelle in the world coordinate system is as follows: ； In the formula, This refers to the yaw angle of the drone.

9. A wind turbine nacelle orientation recognition and calculation system based on UAV vision, used to implement the wind turbine nacelle orientation recognition and calculation method based on UAV vision as described in any one of claims 1-8, characterized in that, include: The position adjustment module pre-configures the positional relationship between the drone and the wind turbine nacelle; The positional relationship includes the yaw angle of the UAV gimbal being the same as the yaw angle of the UAV body, and the pitch angle of the UAV gimbal pointing downwards at 90°. The image processing module acquires image data of the wind turbine nacelle based on the world coordinate system, and performs size normalization processing on the image data to obtain the first image; The YOLO-OBB detection model is then used to filter the first image to obtain the second image; The nacelle position calculation module divides the wind turbine nacelle into rotating rectangular boxes based on the second image, and uses the rotating box dataset to calculate the position of the wind turbine nacelle in the world coordinate system. The nacelle pixel angle calculation module, based on the second image, divides the wind turbine nacelle into rotating rectangular frames, hub rotating rectangular frames, and three-blade rotating rectangular frames. Using these rotating rectangular frames, the hub rotating rectangular frames, or the three-blade rotating rectangular frames, the module calculates the orientation angle of the wind turbine nacelle in the pixel coordinate system. Specifically, this includes: Extract the coordinates of the midpoints of the two short sides of the rotating rectangle of the wind turbine nacelle and the coordinates of the four vertices of the rotating rectangle of the three blades; Based on the midpoints of the two short sides, construct the second and third vectors of the wind turbine nacelle in the pixel coordinate system. According to the coordinates of the four vertices, take the coordinates of the vertex closest to the center point of the wind turbine nacelle to construct the nearest vector. Calculate the angles between the second vector and the nearest vector and between the third vector and the nearest vector respectively to obtain a vector group. Then, use the second or third vector in the vector group to calculate the orientation angle of the wind turbine nacelle in the pixel coordinate system. The nacelle angle calculation module calculates the angle of the wind turbine nacelle in the world coordinate system based on the orientation angle of the wind turbine nacelle in the pixel coordinate system.

Images