Fan blade inspection path generation method and device based on unmanned aerial vehicle

Abstract

The invention relates to the technical field of unmanned aerial vehicle inspection, and discloses a fan blade inspection path generation method and device based on an unmanned aerial vehicle. Random parameters such as a fan yaw angle and a blade position angle of a fan as well as a known structure parameter set and a fixed parameter set are comprehensively considered; therefore, the planned and generated inspection path better fits the actual appearance of the fan, the front and rear edges, the windward and leeside and other parts of the fan blade can be covered, comprehensive inspection of the blade is realized, and the inspection quality is improved. Meanwhile, the draught fan does not need to be adjusted to be in a specific inverted-Y-shaped state, the position limitation of the traditional technology is broken through, the autonomy and intelligence degree of inspection work is improved, inspection path errors caused by manual adjustment of the shutdown posture are reduced, the efficiency and quality of autonomous inspection work of the unmanned aerial vehicle are improved, and the unmanned aerial vehicle is suitable for popularization and application. And the universality and the universality of the routing inspection fan path generation method are enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of drone inspection, and particularly to a method and device for generating an inspection path of a wind turbine blade based on a drone. Background Art

[0002] With the rapid development of the new energy industry, clean energy represented by wind power is gradually improving the original energy structure with its characteristics of low pollution and renewable, and strongly promoting the sustainable development of regional economy and society. However, with the continuous expansion of the scale of wind farms, the demand for equipment safety maintenance in the later stage has increased sharply, and the limited inspection manpower and complex inspection environment have greatly affected the operation efficiency and safety of equipment maintenance. Therefore, it is extremely urgent to explore an efficient and safe method for inspecting wind turbines.

[0003] Multi-rotor drones have the characteristics of high mobility and strong adaptability. They operate by carrying loads such as visible light cameras, infrared cameras or lidar, and with the help of automatic route planning algorithms, they can ensure that the drones autonomously collect data, effectively guaranteeing the rapidity and reliability of front-end data collection, and thus being able to help inspection personnel efficiently and accurately discover and handle abnormalities such as sand holes and gel coat damage on wind turbine blades.

[0004] In the existing related technologies, before starting the inspection flight mission, the wind turbine needs to be adjusted to the "inverted Y shape" state first, and then the position information of the key path points can be calculated according to the structural parameter information of the wind turbine. However, this process not only greatly increases the workload of inspection and operation and maintenance personnel, but also makes the operation and maintenance personnel need to spend a lot of time adjusting the position of the wind turbine rotor in advance. At the same time, it cannot ensure that the adjusted position is accurately located at the pre-conceived "inverted Y shape" position, resulting in a deviation between the inspection path and the actual position of the wind turbine blade, thereby reducing the degree of autonomy, accuracy and quality of drone inspection, and having great deficiencies in generality and universality, which restricts the application and promotion of the technology. Summary of the Invention

[0005] In view of this, the present invention provides a method and device for generating an inspection path of a wind turbine blade based on a drone, so as to solve the problem that the existing drone inspection method needs to first adjust the wind turbine to the "inverted Y shape" state, resulting in low degrees of autonomy, accuracy and quality.

[0006] In a first aspect, the present invention provides a method for generating an inspection path of a wind turbine blade based on a drone, and the method includes:

[0007] Obtain the structural parameter set of the fan to be detected, where the fan to be detected is in any shutdown state; based on the structural parameter set, through the processing of the fan shutdown attitude measurement algorithm, obtain the yaw angle of the fan nacelle and the blade position angle of the fan; establish the fan hub coordinate system using the preset northeast celestial coordinate system, and determine the fixed attitude parameter set of the fan to be detected according to the fan hub coordinate system and the structural parameter set; based on the yaw angle of the fan nacelle, the blade position angle, and the fixed attitude parameter set, calculate the longitude, latitude, and altitude coordinate values of multiple key path points in the preset northeast celestial coordinate system; generate the fan blade inspection path of the fan to be detected according to the multiple longitude, latitude, and altitude coordinate values.

[0008] The method for generating a fan blade inspection path based on an unmanned aerial vehicle provided by the present invention can comprehensively understand the external structure of the fan by obtaining the structural parameter set of the fan to be detected, enabling the subsequent inspection path planning to conform to the actual physical characteristics of the fan and ensuring that the inspection process can cover the key parts of the fan. Further, through the processing of the fan shutdown attitude measurement algorithm, the yaw angle of the fan nacelle and the blade position angle reflecting the actual parked attitude of the fan can be obtained, thereby enabling the subsequent inspection path planning to adapt to any shutdown attitude of the fan, breaking through the limitation in the traditional method that the fan needs to be adjusted to a specific "inverted Y shape" state. Further, the preset northeast celestial coordinate system provides a general reference standard for the entire space positioning. On this basis, establishing the fan hub coordinate system can connect the local structure of the fan with the general space coordinate system. Further, determining the fixed attitude parameter set through the fan hub coordinate system helps to clarify the relative position relationship of each part of the fan in a specific coordinate system. Further, by comprehensively considering the real-time attitude parameters of the fan, as well as known parameters such as the fixed attitude parameter set and the structural parameter set, the key positions that need to be passed through during the unmanned aerial vehicle inspection process can be accurately calculated, enabling the inspection path to closely conform to the actual appearance of the fan and ensuring the comprehensiveness of the blade inspection. At the same time, due to considering the actual attitude of the fan, the obtained longitude, latitude, and altitude coordinate values can reflect the real position of the fan in the real space, improving the matching degree of the inspection path with the actual position of the fan, and thus enhancing the inspection quality. Finally, generating the fan blade inspection path of the fan to be detected according to the multiple longitude, latitude, and altitude coordinate values improves the inspection efficiency, reduces the unnecessary flight path of the unmanned aerial vehicle, saves time and energy. At the same time, since the path conforms to the actual situation of the fan, the accuracy and reliability of the inspection are improved. Therefore, by implementing the present invention, the position limitation of the traditional technology is broken through, the autonomy and intelligence level of the inspection work are improved, the error caused by manual adjustment of the shutdown attitude is reduced, the efficiency and quality of the unmanned aerial vehicle autonomous inspection operation are improved, and the generality and universality of the method for generating the inspection path of the inspection fan are enhanced.

[0009] In an alternative embodiment, based on the structural parameter set and processed by the fan shutdown attitude measurement algorithm, the yaw angle of the fan nacelle and the blade position angle of the fan to be detected are obtained, including:

[0010] Based on the structural parameter set, use a drone to obtain a video image set of the fan to be detected; process the video image set through a blade image instance segmentation algorithm based on deep learning to obtain the yaw angle of the fan nacelle of the fan to be detected; based on the yaw angle of the fan nacelle, use a drone to obtain a fan image set of the fan to be detected; process the fan image set through a blade image instance segmentation algorithm based on deep learning to obtain the blade position angle of the fan to be detected.

[0011] The method for generating a fan blade inspection path based on a drone provided by the present invention can, by virtue of the high mobility and strong adaptability of the drone and in combination with the known structural parameter set of the fan, collect the omnidirectional and multi-angle video images of the fan to be detected in a targeted manner. Further, through processing by a blade image instance segmentation algorithm based on deep learning, the position and shape of the fan blade in the image can be accurately identified and segmented, and then the yaw angle of the fan nacelle can be accurately calculated using this information. Compared with traditional manual measurement or simple algorithm recognition, the accuracy and efficiency of angle measurement are greatly improved. Further, based on the known yaw angle of the fan nacelle, use a drone to obtain a fan image set again, which can more specifically capture images related to the measurement of the blade position angle. Finally, by applying the blade image instance segmentation algorithm based on deep learning to process the fan image set again, the detailed information in the newly obtained images can be fully exploited, the angle of the blade in the image can be accurately judged, and then a high-precision blade position angle can be obtained. Therefore, by implementing the present invention, the error of manually adjusting the fan attitude and measuring the angle is avoided, the accuracy of obtaining the attitude parameters is improved, and thus the accuracy of subsequent inspection path planning is improved.

[0012] In an alternative embodiment, based on the yaw angle of the fan nacelle, the blade position angle of the fan, and the fixed attitude parameter set, calculate the longitude, latitude, and altitude coordinate values of multiple key path points in a preset northeast celestial coordinate system, including:

[0013] Use the fixed attitude parameter set to calculate multiple first spatial coordinate values of multiple key path points in the fan hub coordinate system; based on the fixed attitude parameter set, the yaw angle of the fan nacelle, and the blade position angle, perform a transformation on the multiple first spatial coordinate values to obtain multiple second spatial coordinate values of multiple key path points in the preset northeast celestial coordinate system; convert the multiple second spatial coordinate values into multiple longitude, latitude, and altitude coordinate values.

[0014] The method for generating an inspection path of a wind turbine blade based on an unmanned aerial vehicle provided by the present invention can accurately convert the coordinates in the hub coordinate system established based on the structure of the wind turbine to the preset northeast celestial coordinate system related to the actual geographical space and represent them as geodetic coordinates through reasonable coordinate transformation, so that the subsequent inspection path planning can combine the actual spatial position of the wind turbine and geographical information, further optimize the inspection path, ensure that the unmanned aerial vehicle can comprehensively inspect the wind turbine blade according to the accurate path, improve the scientificity and accuracy of the inspection path planning, and enhance the inspection efficiency and quality.

[0015] In an alternative embodiment, multiple first spatial coordinate values of multiple key path points in the hub coordinate system of the wind turbine are calculated by using a fixed attitude parameter set, including:

[0016] Using the fixed attitude parameter set, calculate multiple third spatial coordinate values of multiple key path points on the windward side of the wind turbine to be detected in the hub coordinate system of the wind turbine, and multiple fourth spatial coordinate values when the wind turbine to be detected is in an inverted Y shape; use the rotation matrix formula to convert the multiple fourth spatial coordinate values into multiple fifth spatial coordinate values of multiple key path points on the leeward side of the wind turbine to be detected in the hub coordinate system of the wind turbine; determine multiple first spatial coordinate values according to the multiple third spatial coordinate values, multiple fourth spatial coordinate values, and multiple fifth spatial coordinate values.

[0017] The method for generating an inspection path of a wind turbine blade based on an unmanned aerial vehicle provided by the present invention fully considers the structural characteristics of different parts of the wind turbine. Through reasonable parameter application and mathematical transformation, it can accurately determine the positions of key parts (windward side, leeward side) of the wind turbine in the hub coordinate system, providing a basis for accurately planning the subsequent inspection path, ensuring that the inspection path can cover all key parts of the wind turbine blade, and improving the comprehensiveness and accuracy of the inspection.

[0018] In an alternative embodiment, based on a fixed attitude parameter set, the yaw angle of the wind turbine nacelle, and the blade position angle of the wind turbine, multiple first spatial coordinate values are converted to obtain multiple second spatial coordinate values of multiple key path points in the preset northeast celestial coordinate system, including:

[0019] Use the fixed attitude parameter set to determine the first spatial rotation matrix and the translation space matrix respectively; use the yaw angle of the wind turbine nacelle to determine the second spatial rotation matrix; use the blade position angle of the wind turbine to determine the third spatial rotation matrix; based on the first spatial rotation matrix, the translation space matrix, the second spatial rotation matrix, and the third spatial rotation matrix, determine the target rotation transformation matrix; use the target rotation transformation matrix to convert multiple first spatial coordinate values to obtain multiple second spatial coordinate values.

[0020] The method for generating an inspection path of a wind turbine blade based on an unmanned aerial vehicle provided by the present invention can fully consider the influence of the actual attitude of the wind turbine on coordinates by determining a rotation transformation matrix based on multiple parameters. Then, through precise matrix transformation, the coordinates in the wind turbine hub coordinate system can be accurately converted to the preset northeast celestial coordinate system, effectively compensating for the coordinate differences caused by the change in the wind turbine attitude, improving the accuracy of coordinate transformation, making the inspection path planning more conform to the actual attitude of the wind turbine, enhancing the accuracy and reliability of the inspection path, and further improving the efficiency and quality of the autonomous inspection of the unmanned aerial vehicle.

[0021] In an optional implementation manner, the method further includes: based on the inspection path of the wind turbine blade, using the unmanned aerial vehicle to inspect the wind turbine to be detected and obtain an inspection image set.

[0022] The method for generating an inspection path of a wind turbine blade based on an unmanned aerial vehicle provided by the present invention enables the unmanned aerial vehicle to efficiently and accurately inspect the wind turbine through the automatically generated precise inspection path, and then can obtain a high-quality inspection image set, providing strong data support for subsequent inspectors to discover and handle abnormalities of the wind turbine blade, and improving the efficiency and accuracy of the inspection work.

[0023] In a second aspect, the present invention provides a device for generating an inspection path of a wind turbine blade based on an unmanned aerial vehicle, and the device includes:

[0024] An acquisition module for acquiring a set of structural parameters of the wind turbine to be detected; a processing module for obtaining the yaw angle of the wind turbine nacelle and the blade position angle of the wind turbine to be detected through processing by a wind turbine shutdown attitude measurement algorithm based on the set of structural parameters; a determination module for establishing a wind turbine hub coordinate system using a preset northeast celestial coordinate system and determining a set of fixed attitude parameters of the wind turbine to be detected according to the wind turbine hub coordinate system and the set of structural parameters; a calculation module for calculating the longitude, latitude, and altitude coordinate values of multiple key path points in the preset northeast celestial coordinate system based on the yaw angle of the wind turbine nacelle, the blade position angle, and the set of fixed attitude parameters; a generation module for generating an inspection path of the wind turbine blade of the wind turbine to be detected according to the multiple longitude, latitude, and altitude coordinate values.

[0025] In a third aspect, the present invention provides a computer device, including: a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to execute the method for generating an inspection path of a wind turbine blade based on an unmanned aerial vehicle according to the first aspect or any corresponding implementation manner thereof.

[0026] In a fourth aspect, the present invention provides a computer-readable storage medium, on which computer instructions are stored, and the computer instructions are used to cause a computer to execute the method for generating an inspection path of a wind turbine blade based on an unmanned aerial vehicle according to the first aspect or any corresponding implementation manner thereof.

[0027] In a fifth aspect, the present invention provides a computer program product, including computer instructions for causing a computer to execute the method for generating an inspection path of a wind turbine blade based on a drone according to the first aspect or any corresponding embodiment thereof described above. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for use in the description of the specific embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0029] Figure 1 is a flowchart of a method for generating an inspection path of a wind turbine blade based on a drone according to an embodiment of the present invention;

[0030] Figure 2 is a schematic diagram of a wind turbine hub coordinate system according to an embodiment of the present invention;

[0031] Figure 3 is a flowchart of another method for generating an inspection path of a wind turbine blade based on a drone according to an embodiment of the present invention;

[0032] Figure 4 is a flowchart of yet another method for generating an inspection path of a wind turbine blade based on a drone according to an embodiment of the present invention;

[0033] Figure 5A is a schematic diagram of key path points in an inspection path of a wind turbine blade according to an embodiment of the present invention;

[0034] Figure 5B is a schematic diagram of key path points in another inspection path of a wind turbine blade according to an embodiment of the present invention;

[0035] Figure 5C is a schematic diagram of key path points in yet another inspection path of a wind turbine blade according to an embodiment of the present invention;

[0036] Figure 6 is a block diagram of a device for generating an inspection path of a wind turbine blade based on a drone according to an embodiment of the present invention;

[0037] Figure 7 is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0038] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0039] An embodiment of the present invention provides a method for generating an inspection path for a wind turbine blade based on a drone. By comprehensively considering random parameters such as the yaw angle of the wind turbine and the blade position angle, as well as known structural parameter sets and fixed parameter sets, automatic planning of the inspection path is carried out to improve the autonomy and intelligence of the inspection work, improve the efficiency and quality of the autonomous inspection operation of the drone, and enhance the versatility and universality of the method for generating the inspection path of the wind turbine.

[0040] According to an embodiment of the present invention, an embodiment of a method for generating an inspection path for a wind turbine blade based on a drone is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0041] In this embodiment, a method for generating an inspection path for a wind turbine blade based on a drone is provided, which can be used in electronic devices such as computers, mobile phones, and tablet computers. Figure 1 is a flowchart of a method for generating an inspection path for a wind turbine blade based on an embodiment of the present invention, as Figure 1 shown, the process includes the following steps:

[0042] Step S101, obtain the structural parameter set of the wind turbine to be detected.

[0043] Among them, the wind turbine to be detected is in any shutdown state; the structural parameter set is used to reflect the appearance structure of the wind turbine to be detected, and may include known fixed information such as the longitude and latitude geographical location information of the wind turbine tower, the hub height, the blade length, the forward movement distance of the wind wheel relative to the tower, the nacelle inclination angle, and the blade cone angle.

[0044] Specifically, shutting down the wind turbine to be detected to any state can ensure that the method provided in this embodiment is applicable to various natural shutdown situations of the wind turbine, is not restricted by a specific shutdown posture, and improves the versatility and universality of the method.

[0045] Furthermore, start the drone and automatically run the program on the on-board computer.

[0046] Furthermore, the airborne computer can read the appearance structure parameter file of the wind turbine to be detected, i.e., the structure parameter set, retained on the local storage space.

[0047] Step S102: Based on the structure parameter set, through the processing of the wind turbine shutdown attitude measurement algorithm, the yaw angle of the wind turbine nacelle and the blade position angle of the wind turbine to be detected are obtained.

[0048] Among them, the wind turbine shutdown attitude measurement algorithm represents an algorithm for accurately measuring and calculating attitude parameters such as the yaw angle of the nacelle and the blade position angle of a wind turbine in a shutdown state. It can comprehensively analyze relevant data by combining sensor data, image recognition technology, coordinate calculation, etc. to obtain the accurate attitude information of the wind turbine when it is shutdown; the yaw angle of the wind turbine nacelle and the blade position angle of the wind turbine can reflect the actual parked attitude of the wind turbine.

[0049] Specifically, by processing the structure parameter set through the wind turbine shutdown attitude measurement algorithm, the yaw angle of the wind turbine nacelle and the blade position angle of the wind turbine, which can reflect the actual parked attitude of the wind turbine, can be obtained. Furthermore, the subsequent inspection path planning can be adapted to any shutdown attitude of the wind turbine, breaking through the limitation in the traditional method that the wind turbine needs to be adjusted to a specific "inverted Y shape" state.

[0050] Step S103: Establish a wind turbine hub coordinate system using the preset northeast - sky coordinate system, and determine the fixed attitude parameter set of the wind turbine to be detected according to the wind turbine hub coordinate system and the structure parameter set.

[0051] Among them, the preset northeast - sky coordinate system (NEU coordinate system) represents a commonly used geospatial coordinate system. In this embodiment, its origin is set at the base point position of the tower column, and the GPS coordinates of this position are known. Further, in the preset northeast - sky coordinate system, the positive direction of the X - axis points east, the positive direction of the Y - axis points north, and the positive direction of the Z - axis is vertically upward, which can provide a macroscopic reference framework related to geographical orientation for the spatial positioning of the entire wind turbine and subsequent coordinate calculations.

[0052] The wind turbine hub coordinate system represents a coordinate system established at the center point of the wind turbine hub and used to describe the relative position relationship between the various components of the wind turbine itself. In this embodiment, the wind turbine hub coordinate system has the center of the wind turbine hub as the origin, and the coordinate axis directions are parallel to the preset northeast - sky coordinate system, as Figure 2 shown.

[0053] Furthermore, the fixed attitude parameter set is used to describe the inherent eigenvalue of the wind turbine in a specific state, which can be determined by combining the wind turbine hub coordinate system and the structural parameter set. Specifically, it can include: the blade inspection length L (approximately equal to 1.1 times the actual blade length l, i.e., L≈1.1, which is used to plan the inspection path of the UAV on the blade to ensure avoiding hitting the blade when changing direction from the blade root to the blade tip), the safe inspection distance w from the blade (ensuring a safe interval between the UAV and the blade during the inspection process), the blade cone angle v (reflecting the inclination angle of the blade in space), the nacelle inclination angle (reflecting the inclination degree of the nacelle relative to the horizontal direction), the hub radius r (the radius parameter of the wind turbine hub, which has a certain effect on analyzing the internal structure of the wind turbine and path planning), the hub height H (the vertical height of the wind turbine hub from the ground), the safety redundancy multiple t (used to avoid the nacelle at the rear end of the wind turbine to ensure inspection safety), and the forward movement distance s of the rotor relative to the tower column (clarifying the relative position relationship between the rotor and the tower column).

[0054] Step S104: Based on the yaw angle of the wind turbine nacelle, the blade position angle of the wind turbine, and the fixed attitude parameter set, calculate the longitude-latitude-altitude coordinate values of multiple key path points in the preset northeast celestial coordinate system.

[0055] Specifically, by comprehensively considering the real-time attitude parameters of the wind turbine, as well as known parameters such as the fixed attitude parameter set and the structural parameter set, the key positions that the UAV needs to pass through during the inspection process can be accurately calculated, enabling the inspection path to closely fit the actual appearance of the wind turbine, ensuring the comprehensiveness of blade inspection. At the same time, due to considering the actual attitude of the wind turbine, the obtained longitude-latitude-altitude coordinate values can reflect the real position of the wind turbine in the real space, improving the matching degree between the inspection path and the actual position of the wind turbine, and thus enhancing the inspection quality.

[0056] Step S105: Generate the inspection path of the wind turbine blades of the wind turbine to be detected according to the multiple longitude-latitude-altitude coordinate values.

[0057] Specifically, connecting the multiple longitude-latitude-altitude coordinate values can form the inspection path of the wind turbine blades of the wind turbine to be detected.

[0058] The method for generating an inspection path of a wind turbine blade based on a drone provided in this embodiment can comprehensively understand the appearance structure of the wind turbine by obtaining the structural parameter set of the wind turbine to be inspected, enabling the subsequent inspection path planning to conform to the actual physical characteristics of the wind turbine and ensuring that the inspection process can cover the key parts of the wind turbine. Further, through the processing of the wind turbine shutdown attitude measurement algorithm, the yaw angle of the wind turbine nacelle and the blade position angle of the wind turbine, which reflect the actual parked attitude of the wind turbine, can be obtained, and then the subsequent inspection path planning can adapt to any shutdown attitude of the wind turbine, breaking through the limitation in the traditional method that the wind turbine needs to be adjusted to a specific "inverted Y shape" state. Further, presetting the northeast celestial coordinate system provides a general reference standard for the entire spatial positioning. On this basis, establishing a wind turbine hub coordinate system can connect the local structure of the wind turbine with the general spatial coordinate system. Further, determining the fixed attitude parameter set through the wind turbine hub coordinate system helps to clarify the relative position relationship of each part of the wind turbine in a specific coordinate system. Further, by comprehensively considering the real-time attitude parameters of the wind turbine, as well as the known parameters such as the fixed attitude parameter set and the structural parameter set, the key positions that the drone needs to pass through during the inspection process can be accurately calculated, enabling the inspection path to closely conform to the actual appearance of the wind turbine and ensuring the comprehensiveness of the blade inspection. At the same time, due to considering the actual attitude of the wind turbine, the obtained longitude, latitude, and altitude coordinate values can reflect the real position of the wind turbine in the real space, improving the matching degree between the inspection path and the actual position of the wind turbine, and then enhancing the inspection quality. Finally, generating an inspection path of the wind turbine blade of the wind turbine to be inspected according to multiple longitude, latitude, and altitude coordinate values improves the inspection efficiency, reduces the unnecessary flight path of the drone, saves time and energy. At the same time, since the path conforms to the actual situation of the wind turbine, the accuracy and reliability of the inspection are improved. Therefore, by implementing the present invention, the position limitation of the traditional technology is broken through, the autonomy and intelligence level of the inspection work are improved, the error caused by manual adjustment of the shutdown attitude is reduced, the efficiency and quality of the autonomous inspection operation of the drone are improved, and the generality and universality of the method for generating the inspection path of the wind turbine are enhanced.

[0059] In this embodiment, a method for generating an inspection path of a wind turbine blade based on a drone is provided, which can be used in electronic devices such as computers, mobile phones, tablet computers, etc. Figure 3 It is a flowchart of the method for generating an inspection path of a wind turbine blade based on a drone according to an embodiment of the present invention, as Figure 3 shown, and this process includes the following steps:

[0060] Step S301, obtaining the structural parameter set of the wind turbine to be inspected. For details, please refer to Figure 1 step S101 of the embodiment shown, which will not be elaborated here.

[0061] Step S302, based on the structural parameter set, through the processing of the wind turbine shutdown attitude measurement algorithm, obtaining the yaw angle of the wind turbine nacelle and the blade position angle of the wind turbine to be inspected.

[0062] Specifically, the above step S302 includes:

[0063] Step S3021: Based on the structural parameter set, use a drone to obtain a video image set of the wind turbine to be detected.

[0064] Specifically, the flight altitude of the drone can be determined according to the hub height in the structural parameter set, and taking the GPS coordinate position of the wind turbine tower as the origin, considering factors such as the size of the wind turbine, the safety distance of the drone, and the clarity of image acquisition, a suitable flight radius distance is selected.

[0065] Secondly, the drone conducts a patrol flight around the wind turbine tower at the determined flight altitude and flight radius for one week. Further, during the flight, the visible light camera carried by the drone can continuously collect video and image data related to the wind turbine and form a corresponding video image set.

[0066] Step S3022: Process the video image set through a leaf image instance segmentation algorithm based on deep learning to obtain the yaw angle of the nacelle of the wind turbine to be detected.

[0067] Among them, the leaf image instance segmentation algorithm based on deep learning represents a technology that uses a deep neural network model to accurately identify and segment the wind turbine blades in the image. It can learn from a large number of labeled wind turbine blade image data, so as to accurately distinguish each blade instance from the input image and segment its contour.

[0068] Specifically, input the obtained video image set into the leaf image instance segmentation algorithm program based on deep learning on the on-board computer. Among them, this leaf image instance segmentation algorithm program uses the Yolact-Edge network model and the pre-trained model parameters.

[0069] Further, this algorithm program can process the input image, identify and segment the wind turbine blades.

[0070] Further, after this algorithm program outputs the identified and segmented wind turbine blades, record the GPS coordinates at the flight positions where two blades are stably detected.

[0071] Further, combining the GPS coordinate position of the wind turbine tower, using the principle that three points in space determine a plane, the included angle between the positive direction of the rotor plane and the due north direction, that is, the yaw angle of the nacelle of the wind turbine, can be determined through calculation.

[0072] Step S3023: Based on the yaw angle of the nacelle of the wind turbine, use a drone to obtain a wind turbine image set of the wind turbine to be detected.

[0073] Among them, the fan image set can reflect the positional relationship of the fan blades relative to a specific direction (determined by the nacelle yaw angle).

[0074] Specifically, according to the measured nacelle yaw angle of the fan, the flight path of the UAV can be adjusted so that the UAV flies to the position directly in front of the fan hub.

[0075] Furthermore, at the position directly in front of the fan hub, the camera carried by the UAV takes a front view image of the fan to be detected and forms a fan image set.

[0076] Step S3024: Process the fan image set through a blade image instance segmentation algorithm based on deep learning to obtain the fan blade position angle of the fan to be detected.

[0077] Specifically, by processing the obtained fan image set through a blade image instance segmentation algorithm based on deep learning, the fan blades in the image can be further segmented into instances. Among them, the specific process can refer to the description of step S2022 above and will not be elaborated here.

[0078] Furthermore, by analyzing the image after instance segmentation, the angle between the line connecting the upper left vertex and the lower right vertex of the blade rectangular boundary recognition box and the vertical direction, that is, the fan blade position angle, can be further calculated. In this embodiment, the analysis and calculation can be performed through relevant running programs on the on-board computer carried by the UAV.

[0079] Step S303: Establish a fan hub coordinate system using the preset northeast celestial coordinate system, and determine the fixed attitude parameter set of the fan to be detected according to the fan hub coordinate system and the structural parameter set. For details, please refer to Figure 1 Step S103 of the illustrated embodiment, which will not be elaborated here.

[0080] Step S304: Based on the nacelle yaw angle, the fan blade position angle, and the fixed attitude parameter set, calculate the longitude, latitude, and altitude coordinate values of multiple key path points in the preset northeast celestial coordinate system. For details, please refer to Figure 1 Step S104 of the illustrated embodiment, which will not be elaborated here.

[0081] Step S305: Generate a fan blade inspection path for the fan to be detected according to the multiple longitude, latitude, and altitude coordinate values. For details, please refer to Figure 1 Step S105 of the illustrated embodiment, which will not be elaborated here.

[0082] Step S306: Based on the fan blade inspection path, use the UAV to inspect the fan to be detected and obtain an inspection image set.

[0083] Specifically, load the inspection path of the fan blade into the flight control system of the UAV. Further, the flight control system can plan the flight trajectory of the UAV based on the information of multiple path points on the inspection path of the fan blade to ensure that the UAV can fly along the preset inspection path and cover all key parts of the fan blade.

[0084] Further, the parameters of the UAV's gimbal can be initialized according to the structural characteristics of the fan and the inspection requirements. Specifically, considering the limitation of the maximum movement angle of the gimbal, set the movement range of the gimbal during the inspection to ensure that the gimbal can flexibly adjust the shooting angle within the operable range.

[0085] Further, the UAV starts flying according to the loaded inspection path. During the flight, the pitch angle of the gimbal can be adjusted in real time based on the relative position angle of the inspected blade. When inspecting a certain blade, the gimbal can also be controlled so that the shooting pitch angle and the inspected blade are perpendicular to each other by 90°, thereby ensuring that the camera lens can be perpendicular to the blade surface for shooting and obtaining clear and complete blades.

[0086] Further, to ensure the image quality and prevent the high-speed rotating blades of the UAV from being captured (even when the fan is stopped, the possible slight swing needs to be considered), the maximum elevation angle during gimbal shooting is generally not more than 10°. In some cases, the gimbal can be directly adjusted to the horizontal angle for shooting. By optimizing the shooting angle, it is possible to further avoid image blurring or the inclusion of unnecessary interference elements caused by angle problems.

[0087] Further, during the flight of the UAV along the inspection path, the gimbal can, at the adjusted angle, use the equipped image acquisition device (such as a high-definition camera) and take pictures of the fan blade at preset time intervals or distance intervals to obtain the corresponding inspection image set.

[0088] The method for generating an inspection path for a wind turbine blade based on a drone provided in this embodiment can, by leveraging the characteristics of high mobility and strong adaptability of the drone and combining with a known set of wind turbine structure parameters, collect all-round and multi-angle video images of the wind turbine to be inspected in a targeted manner. Further, through processing by a leaf image instance segmentation algorithm based on deep learning, the position and shape of the wind turbine blade in the image can be accurately identified and segmented, and then the yaw angle of the wind turbine nacelle can be accurately calculated using this information. Compared with traditional manual measurement or simple algorithm recognition, the accuracy and efficiency of angle measurement are greatly improved. Further, based on the known yaw angle of the wind turbine nacelle, the drone is used again to obtain a set of wind turbine images, and images related to the measurement of the blade position angle can be taken more targeted. Finally, by applying the leaf image instance segmentation algorithm based on deep learning to process the set of wind turbine images again, the detailed information in the images can be fully mined from the newly obtained images, the angle of the blade in the image can be accurately judged, and then a high-precision blade position angle can be obtained. Therefore, by implementing the present invention, the error of manually adjusting the attitude of the wind turbine and measuring the angle is avoided, the accuracy of obtaining attitude parameters is improved, and the accuracy of subsequent inspection path planning is further improved.

[0089] In this embodiment, a method for generating an inspection path for a wind turbine blade based on a drone is provided, which can be used in electronic devices such as computers, mobile phones, and tablet computers. Figure 4 It is a flowchart of the method for generating an inspection path for a wind turbine blade based on the embodiment of the present invention, as Figure 4 shown, and this process includes the following steps:

[0090] Step S401, obtain a set of structure parameters of the wind turbine to be inspected. For details, please refer to Figure 1 step S101 of the embodiment shown here, which will not be elaborated here.

[0091] Step S402, based on the set of structure parameters, through processing by the wind turbine shutdown attitude measurement algorithm, obtain the yaw angle of the wind turbine nacelle and the blade position angle of the wind turbine to be inspected. For details, please refer to Figure 3 step S302 of the embodiment shown here, which will not be elaborated here.

[0092] Step S403, establish a wind turbine hub coordinate system using a preset northeast celestial coordinate system, and determine a set of fixed attitude parameters of the wind turbine to be inspected according to the wind turbine hub coordinate system and the set of structure parameters. For details, please refer to Figure 1 step S103 of the embodiment shown here, which will not be elaborated here.

[0093] Step S404, based on the yaw angle of the wind turbine nacelle, the blade position angle, and the set of fixed attitude parameters, calculate the longitude, latitude, and altitude coordinate values of multiple key path points in the preset northeast celestial coordinate system.

[0094] Specifically, the above step S404 includes:

[0095] Step S4041, calculating multiple first spatial coordinate values of multiple key path points in the wind turbine hub coordinate system by using a fixed attitude parameter set.

[0096] Specifically, multiple first spatial coordinate values of multiple key path points in the wind turbine hub coordinate system can be calculated through the fixed parameter information of the wind turbine.

[0097] In some optional embodiments, the above step S4041 includes:

[0098] Step a1, calculating multiple third spatial coordinate values of multiple windward key path points of the wind turbine to be detected in the wind turbine hub coordinate system, and multiple fourth spatial coordinate values when the wind turbine to be detected is in an inverted Y shape.

[0099] Step a2, using the rotation matrix formula to convert multiple fourth spatial coordinate values into multiple fifth spatial coordinate values of multiple leeward key path points of the wind turbine to be detected in the wind turbine hub coordinate system.

[0100] Step a3, determining multiple first spatial coordinate values according to multiple third spatial coordinate values, multiple fourth spatial coordinate values, and multiple fifth spatial coordinate values.

[0101] Specifically, multiple third spatial coordinate values of multiple windward key path points (p i , i = 1, 2, …, 12) of the wind turbine to be detected in the wind turbine hub coordinate system can be calculated by using a fixed attitude parameter set, as Figure 5A shown.

[0102] First, taking the front of the hub as the starting waypoint, the spatial coordinate values are respectively shown by the following relational expressions (1) to (8):

[0103] p 0 = (w, 0, 0) 1 (1)

[0104]

[0105] p 3 = (Lsinv, w, Lcosv) T (4)

[0106]

[0107] p 11 = (w, 0, 0) T (8)

[0108] Further, p 5 and p6 , p 8 and p 9 The positions of these four points can adopt the rotation matrix formula around the X-axis, that is, the spatial positions after rotating 120° and 240° around the X-axis can be calculated respectively. After the drone reaches the point position, it continues to fly along the blade to the position (coinciding with), closes, and then flies back to the position (i.e., directly in front of the hub).

[0109] Secondly, when calculating the spatial coordinate values of the key points of the inspection path on the leeward side, by considering factors such as blade cone angle, safety distance, and safe inspection length, it can better fit the actual appearance structure of the wind turbine.

[0110] Specifically, due to the inspection of the leading edge and trailing edge, which is also based on the symmetric structural characteristics of the wind wheel, the positions of the 4 key inspection points of the upper blade of the inverted Y-shaped wind turbine (i.e., the wind turbine to be detected in the inverted Y-shaped state), that is, multiple fourth spatial coordinate values, can be calculated first. As Figure 5B shown, they are respectively shown in the following relational expressions (9) to (12):

[0111] p 12 =(t×r×sin(v)+w×cos(v), 0, t×r×sin(v)-w×sin(v)) T (9)

[0112] p 13 =(L×sin(v)+w×cos(v), 0, L×cos(v)-w×sin(v)) T (10)

[0113] p 14 =(L×sin(v)-w×cos(v), 0, L×cos(v)+w×sin(v)) T (11)

[0114] p 15 =(t×r×sin(v)-w×cos(v), 0, t×r×sin(v)+w×sin(v)) T (12)

[0115] Among them, p 12 represents the inspection point on the upper blade close to the hub; p 13 represents the inspection point on the upper blade close to the tip; p 14 represents the inspection point on the other side of the upper blade close to the tip; p 15 represents the inspection point on the other side of the upper blade close to the hub.

[0116] Further, using the rotation matrix formula, the positions of the above-mentioned multiple fourth spatial coordinate values after rotating 120° and 240° around the X-axis can be calculated, which are the multiple fifth spatial coordinate values, respectively denoted as: p 16 The inspection point on the left blade near the hub (corresponding to p 12 ), p 18 The inspection point on the left blade near the tip (corresponding to p 13 ), p 17 The inspection point on the other side of the left blade near the tip (corresponding to p 14 ), p 19 The inspection point on the other side of the left blade near the hub (corresponding to p 15 ), as shown in Figure 5B .

[0117] p 20 The inspection point on the right blade near the hub (corresponding to p 12 ), p 21 The inspection point on the right blade near the tip (corresponding to p 13 ), p 22 The inspection point on the other side of the right blade near the tip (corresponding to p 14 ), p 23 The inspection point on the other side of the left blade near the hub (corresponding to p 15 ), as shown in Figure 5C .

[0118] Finally, by integrating the above-determined multiple third spatial coordinate values, multiple fourth spatial coordinate values, and multiple fifth spatial coordinate values, the multiple first spatial coordinate values of multiple key path points in the fan hub coordinate system can be determined (p o , i = 1, 2, …, 23).

[0119] Step S4042: Based on the fixed attitude parameter set, the fan nacelle yaw angle, and the fan blade position angle, transform the multiple first spatial coordinate values to obtain multiple second spatial coordinate values of multiple key path points in the preset northeast celestial coordinate system.

[0120] Specifically, by reasonably performing coordinate transformation on the multiple first spatial coordinate values in combination with the structural parameter set, the fan nacelle yaw angle, and the fan blade position angle, the influence of the actual attitude of the fan and the known fixed parameters on the coordinates can be fully considered. Furthermore, the coordinates in the fan hub coordinate system can be accurately transformed to the preset northeast celestial coordinate system, effectively compensating for the coordinate differences caused by the change in the fan attitude and improving the accuracy of coordinate transformation.

[0121] In some alternative embodiments, the above step S4042 includes:

[0122] Step b1: Use the fixed attitude parameter set to respectively determine the first spatial rotation matrix and the translation spatial matrix.

[0123] Step b2, determine the second spatial rotation matrix by using the yaw angle of the wind turbine nacelle.

[0124] Step b3, determine the third spatial rotation matrix by using the blade position angle of the wind turbine.

[0125] Step b4, determine the target rotation transformation matrix based on the first spatial rotation matrix, the translation spatial matrix, the second spatial rotation matrix, and the third spatial rotation matrix.

[0126] Step b5, transform multiple first spatial coordinate values by using the target rotation transformation matrix to obtain multiple second spatial coordinate values.

[0127] Specifically, the key path points (p o , i = 1, 2, …, 23) corresponding to multiple first spatial coordinate values can be homogenized to p i ′, i = 1, 2, …, 23, and then undergo spatial rotation and translation changes to obtain multiple second spatial coordinate values p i ″, i = 1, 2, …, 23, as shown in the following relational expression (13):

[0128] p i ″ = T * p i ‘ (13)

[0129] Among them, T represents the target rotation transformation matrix, which can include rotation and translation matrices around the X, Y, and Z axes.

[0130] First, the spatial rotation matrix around the X axis, i.e., the third spatial rotation matrix R X (θ), can be determined according to the blade position angle θ of the wind turbine, as shown in the following relational expression (14):

[0131]

[0132] Second, the spatial rotation matrix around the Y axis, i.e., the first spatial rotation matrix can be determined according to the nacelle inclination angle in the fixed attitude parameter set, as shown in the following relational expression (15):

[0133]

[0134] Furthermore, the translation spatial matrix X can be determined according to the hub height H and the forward displacement s of the wind wheel relative to the tower column in the fixed attitude parameter set, as shown in the following relational expression (16):

[0135] X = [s, 0, H, 1] T (16)

[0136] Then, the second spatial rotation matrix R Z (α) can be determined according to the yaw angle α of the fan nacelle, as shown in the following relational expression (17):

[0137]

[0138] Finally, by combining the first spatial rotation matrix, the translation spatial matrix, the second spatial rotation matrix, and the third spatial rotation matrix, the final target rotation transformation matrix can be determined, as shown in the following relational expression (18):

[0139]

[0140] Through the above method of determining the rotation transformation matrix based on multiple parameters, the influence of the actual attitude of the fan on the coordinates can be fully considered. Then, through precise matrix transformation, the coordinates in the fan hub coordinate system can be accurately converted to the preset northeast celestial coordinate system, effectively compensating for the coordinate differences caused by the fan attitude change, improving the accuracy of coordinate transformation, making the inspection path planning more conform to the actual attitude of the fan, enhancing the accuracy and reliability of the inspection path, and further improving the efficiency and quality of the UAV autonomous inspection.

[0141] Step S4043: Convert multiple second spatial coordinate values into multiple geodetic coordinate values.

[0142] Converting the obtained multiple second spatial coordinate values into corresponding geodetic coordinate values can enable the subsequent inspection path planning to combine the actual spatial position of the fan and geographical information, further optimizing the inspection path, ensuring that the UAV can comprehensively inspect the fan blades along the precise path, improving the scientificity and accuracy of the inspection path planning, and enhancing the inspection efficiency and quality.

[0143] Specifically, let the geodetic coordinate corresponding to the origin of the NEU coordinate system be s 0 =(lat 0 , lon 0 , alt 0 ), and the NEU coordinate of the inspection path point p i ″, that is, the second spatial coordinate value, is p i ″=(x i ′, y i ′, z i ′), and the corresponding geodetic coordinate is (lat i , lon i , alt i ). The corresponding conversion relational expression is as shown in the following relational expression (19):

[0144]

[0145] Therefore, through the above relation (19), multiple second space coordinate values can be converted into multiple latitude-longitude-height coordinate values.

[0146] Step S405: Generate an inspection path for the fan blades of the fan to be detected based on multiple latitude-longitude-height coordinate values. For details, please refer to Figure 1 Step S105 of the illustrated embodiment, which will not be elaborated here.

[0147] The method for generating an inspection path for fan blades based on an unmanned aerial vehicle (UAV) provided in this embodiment fully considers the structural characteristics of different parts of the fan. Through reasonable parameter application and mathematical transformation, it can accurately determine the positions of key parts (windward side, leeward side) of the fan in the hub coordinate system. Further, by determining the rotation transformation matrix based on multiple parameters, it can fully consider the influence of the actual attitude of the fan on the coordinates. Then, through accurate matrix transformation, the coordinates in the fan hub coordinate system are accurately transformed to the preset northeast celestial coordinate system and represented as latitude-longitude-height coordinates, effectively compensating for the coordinate differences caused by the attitude change of the fan, improving the accuracy of coordinate transformation, making the inspection path planning more conform to the actual attitude of the fan, and then enabling the subsequent inspection path planning to combine the actual spatial position and geographical information of the fan, further optimizing the inspection path, ensuring that the UAV can comprehensively inspect the fan blades according to the accurate path, improving the scientificity and accuracy of the inspection path planning, and enhancing the inspection efficiency and quality.

[0148] In one example, in order to automatically generate and plan the inspection path of the fan blades at any shutdown position and effectively increase the automation and intelligence level of UAV inspection, a method for determining the inspection flight path of the fan blades at any shutdown position by the UAV is provided. Among them, the UAV platform used is the DJI M300 RTK model, the gimbal camera carried is the H20T, and the on-board computer used is the Manifold 2C.

[0149] Specifically, the above method includes:

[0150] Step 1: Shut down the fan to be detected to any state, start the UAV and automatically run the program on the on-board computer. First, read a number of known appearance structure parameter files of the fan to be detected stored in the local storage space, including fixed information such as the latitude-longitude geographical location information of the fan tower, hub height, blade length, forward movement distance of the wind wheel relative to the tower, nacelle inclination angle, blade cone angle, etc. Then, use the fan shutdown attitude measurement algorithm to obtain two key shutdown attitude parameter information, namely the yaw angle and blade position angle of the fan to be detected.

[0151] Among them, a method for obtaining the yaw angle and blade position angle of the fan to be detected is as follows:

[0152] Step S11: Set the flight altitude as the hub height, take the GPS coordinate position of the wind turbine tower column as the origin, and select an appropriate flight radius distance. The UAV orbits around the wind turbine tower column for one inspection round;

[0153] Step S12: The H20T visible light camera carried captures video and image data, and inputs the video image data, etc. into the blade image instance segmentation algorithm program based on deep learning on the on-board computer. This algorithm program uses the Yolact-Edge network model, and also saves the pre-trained model parameters in the storage part of the on-board computer. When running, the algorithm program outputs the recognized and segmented wind turbine blades. Record the GPS coordinates at the flight position where two blades are stably detected. Then, combined with the GPS coordinate position of the tower column, a plane can be determined by three points in space, and the angle between the positive direction of the wind wheel plane and the due north direction calculated is the yaw angle of the wind turbine nacelle;

[0154] Step S13: According to the measured yaw angle, fly to the front of the wind turbine hub, take a front view image, and perform instance segmentation on the blades. Then, on the on-board computer carried by the UAV, run the program to calculate the angle between the line connecting the upper left vertex and the lower right vertex of the rectangular boundary recognition frame of the blade and the vertical direction, which is the blade position angle.

[0155] Step2: After obtaining the yaw angle and blade position angle of the wind turbine, set the origin of the NEU coordinate system at the base point position of the tower column, and the GPS coordinates of this position are known. Use the wind wheel based on an inverted Y shape for initial position analysis. First, establish a coordinate system at the center point position of the wind turbine hub As Figure 2 shown; the blade inspection length L (to avoid the UAV hitting the blade when changing direction from the blade root to the blade tip, set the blade inspection length L to be approximately equal to 1.1 times the actual length l of the real blade, i.e., L≈1.1), the safe inspection distance from the blade is w, the blade cone angle v, the nacelle inclination angle the hub radius r, the hub height v, the safety redundancy multiple size is H (used to avoid the nacelle at the rear end of the wind turbine), the forward movement distance t of the wind wheel relative to the tower column. Based on the measured random attitude parameter information of the wind turbine when it is shut down, combined with these fixed parameter information of the wind turbine, calculate the spatial coordinates of the key path points on the inspection path. The specific steps are as follows:

[0156] Step S21: Calculate the spatial coordinates of the key path points in the coordinate system. Take the front of the hub as the starting waypoint, and the coordinates are shown in the above relational expressions (1) to (8) respectively.

[0157] Furthermore, t 5 and p 6 p 8 and p 9The positions of these 4 points can be calculated using the rotation matrix formula about the X-axis, that is, the spatial positions after rotating 120° and 240° about the X-axis can be calculated respectively. After the UAV reaches the point position, it continues to fly along the blade to the position (coinciding with) and closes, and then flies back to the position (i.e., directly in front of the hub).

[0158] Step S22: The above are the calculation formulas for the key points of the inspection paths on the windward and leeward sides of the wind turbine, which consider factors such as blade cone angle, safety distance, and safe inspection length, and can fit the actual appearance structure of the wind turbine. Regarding the inspection of the leading edge and trailing edge, based on the symmetric structural characteristics of the wind wheel, the positions of 4 key inspection points on the upper blade of the inverted Y-shaped wind turbine can be calculated first, as shown in the above relationships (9) to (12).

[0159] Then, the positions of these 4 points are calculated using the rotation matrix formula for the positions after rotating 120° and 240° about the X-axis. They are respectively denoted as: p 16 The inspection point on the left blade near the hub (corresponding to p 12 ), p 18 The inspection point on the left blade near the tip (corresponding to p 13 ), p 17 The inspection point on the other side of the left blade near the tip (corresponding to p 14 ), p 19 The inspection point on the other side of the left blade near the hub (corresponding to p 15 ).

[0160] p 20 The inspection point on the right blade near the hub (corresponding to p 12 ), p 21 The inspection point on the right blade near the tip (corresponding to p 13 ), p 22 The inspection point on the other side of the right blade near the tip (corresponding to p 14 ), p 23 The inspection point on the other side of the left blade near the hub (corresponding to p 15 ).

[0161] Step3: Calculate the positions of each inspection point in the NEU coordinate system under the actual wind turbine. Homogenize the path point p i to p i ′, and then perform the rotation and translation transformation in space p i ″ = T * p i ‘.

[0162] Among them, the rotation transformation matrix T includes the rotation and translation about the X, Y, and Z axes, and can be determined by the above relationships (14) to (18).

[0163] Step 4: Convert the position of the point in the NEU coordinate system at this time into geodetic latitude, longitude and altitude. Let the geodetic latitude, longitude and altitude coordinates corresponding to the origin of the NEU coordinate system be s 0 =(lat 0 , lon 0 , alt 0 ), and the NEU coordinate of the inspection path point p i ″ is p i ″=(x i ′, y i ′, z i ′), and the corresponding geodetic latitude, longitude and altitude coordinates are (lat i , lon i , alt i ), and the corresponding conversion relationship is as shown in the above relationship (19).

[0164] Finally, input it into the flight control system of the UAV, and start the inspection flight of the blades according to the positions of the key position points.

[0165] Step 5: During the inspection flight of the UAV, on the premise of the maximum movement angle limit of the pan-tilt, make the pan-tilt face the fan as much as possible for shooting. Based on the relative position angle of the inspected blade, when inspecting this blade, make the pitch angle of the pan-tilt shooting form a 90° angle with the inspected blade. Among them, the maximum elevation angle during the pan-tilt shooting generally does not exceed 10° or directly adjust the pan-tilt to the horizontal angle for shooting, so as to prevent the high-speed rotating blades of the UAV from being photographed, ensuring the image quality.

[0166] The determination method provided in this embodiment for automatically generating and planning the inspection flight path of the fan blades at any shutdown position by the UAV has the following effects:

[0167] 1. When modeling the appearance structure of the fan and calculating the positions of the key path points, random parameters such as the yaw angle of the fan and the blade position angle when the fan is shut down, as well as known parameters and artificially set parameters such as the hub height, blade length, nacelle inclination angle, wind turbine cone angle, and safe shooting distance are comprehensively considered. Therefore, the planned inspection path fits the appearance of the fan better, including the leading and trailing edges, windward and leeward surfaces of the three fan blades, etc., ensuring full coverage of the blade inspection and improving the inspection quality at the same time.

[0168] 2. The versatility and universality of the inspection fan path generation method are improved. According to the fan yaw angle and blade stop position angle measured by the attitude recognition algorithm, the external inspection path of the fan in any stop attitude can be automatically generated and planned. Thus, it is neither restricted by the position limit of the traditional technical solution that makes the fan stop at the inverted Y-shaped position in advance, greatly improving the autonomy and intelligence of the inspection work, reducing the time of the whole inspection process, and reducing the error impact on the accuracy of the automatically generated inspection path caused by artificial operations such as adjusting the stop attitude, improving the efficiency and quality of the UAV autonomous inspection operation.

[0169] Therefore, through this example, considering the key stop attitude parameters of the fan to be inspected and the known external structure parameters related to the fan, the coordinates of the key path points on the inspection path are calculated and the flight order of the path points is set, and the inspection path of the fan blades at any stop position can be automatically planned for the UAV. Compared with the traditional manual inspection and the technology of the flyer controlling the UAV flight, the calculation method is simple and convenient, which not only effectively reduces the labor cost, but also greatly improves the efficiency of the UAV automatic inspection and operation and maintenance, and speeds up the construction pace of the new energy power station with "unattended" and "few people on duty".

[0170] In this embodiment, a device for generating a fan blade inspection path based on a UAV is also provided. This device is used to implement the above embodiments and preferred implementation manners, and those that have been described will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that can achieve a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware is also possible and contemplated.

[0171] This embodiment provides a device for generating a fan blade inspection path based on a UAV, as Figure 6 shown, the device includes:

[0172] An acquisition module 601, configured to acquire a set of structure parameters of the fan to be detected, and the fan to be detected is in any stop state.

[0173] A processing module 602, configured to obtain the fan nacelle yaw angle and the fan blade position angle of the fan to be detected through processing by a fan stop attitude measurement algorithm based on the set of structure parameters.

[0174] A determination module 603, configured to establish a fan hub coordinate system by using a preset northeast celestial coordinate system, and determine a set of fixed attitude parameters of the fan to be detected according to the fan hub coordinate system and the set of structure parameters.

[0175] A calculation module 604, configured to calculate the longitude, latitude and altitude coordinate values of multiple key path points in the preset northeast celestial coordinate system based on the fan nacelle yaw angle, the fan blade position angle and the set of fixed attitude parameters.

[0176] A generating module 605, configured to generate an inspection path for the fan blades of the fan to be detected according to a plurality of latitude-longitude-altitude coordinate values.

[0177] In some alternative embodiments, the processing module 602 includes:

[0178] A first acquisition sub-module, configured to acquire a video image set of the fan to be detected by using a drone based on a set of structural parameters.

[0179] A first processing sub-module, configured to process the video image set through a leaf image instance segmentation algorithm based on deep learning to obtain the yaw angle of the fan nacelle of the fan to be detected.

[0180] A second acquisition sub-module, configured to acquire a fan image set of the fan to be detected by using a drone based on the yaw angle of the fan nacelle.

[0181] A second processing sub-module, configured to process the fan image set through a leaf image instance segmentation algorithm based on deep learning to obtain the blade position angle of the fan to be detected.

[0182] In some alternative embodiments, the calculation module 604 includes:

[0183] A calculation sub-module, configured to calculate a plurality of first spatial coordinate values of a plurality of key path points in the fan hub coordinate system by using a set of fixed attitude parameters.

[0184] A first conversion sub-module, configured to convert the plurality of first spatial coordinate values based on the set of fixed attitude parameters, the yaw angle of the fan nacelle, and the blade position angle of the fan to obtain a plurality of second spatial coordinate values of the plurality of key path points in a preset northeast-down coordinate system.

[0185] A second conversion sub-module, configured to convert the plurality of second spatial coordinate values into a plurality of latitude-longitude-altitude coordinate values.

[0186] In some alternative embodiments, the calculation sub-module includes:

[0187] A calculation unit, configured to calculate a plurality of third spatial coordinate values of a plurality of key path points on the windward side of the fan to be detected in the fan hub coordinate system, and a plurality of fourth spatial coordinate values when the fan to be detected is in an inverted Y shape by using a set of fixed attitude parameters.

[0188] A first conversion unit, configured to convert the plurality of fourth spatial coordinate values into a plurality of fifth spatial coordinate values of a plurality of key path points on the leeward side of the fan to be detected in the fan hub coordinate system by using a rotation matrix formula.

[0189] The first determination unit is configured to determine a plurality of first space coordinate values according to a plurality of third space coordinate values, a plurality of fourth space coordinate values, and a plurality of fifth space coordinate values.

[0190] In some alternative embodiments, the first conversion sub-module includes:

[0191] The second determination unit is configured to respectively determine a first space rotation matrix and a translation space matrix by using a fixed attitude parameter set.

[0192] The third determination unit is configured to determine a second space rotation matrix by using the yaw angle of the fan nacelle.

[0193] The fourth determination unit is configured to determine a third space rotation matrix by using the blade position angle of the fan.

[0194] The fifth determination unit is configured to determine a target rotation change matrix based on the first space rotation matrix, the translation space matrix, the second space rotation matrix, and the third space rotation matrix.

[0195] The second conversion unit is configured to convert a plurality of first space coordinate values by using the target rotation change matrix to obtain a plurality of second space coordinate values.

[0196] In some alternative embodiments, the device further includes:

[0197] An inspection module is configured to inspect a fan to be detected by using a drone based on a fan blade inspection path and obtain an inspection image set.

[0198] The further function descriptions of the above-mentioned various modules and units are the same as those in the corresponding embodiments above, and will not be repeated here.

[0199] The device for generating a fan blade inspection path based on a drone in this embodiment is presented in the form of functional units. Here, the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and a memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0200] This embodiment of the present invention further provides a computer device having the above-mentioned Figure 6 shown device for generating a fan blade inspection path based on a drone.

[0201] Please refer to Figure 7 , Figure 7 which is a schematic structural diagram of a computer device provided by an alternative embodiment of the present invention. As shown in Figure 7As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Each component communicates with each other using different buses and can be installed on a common motherboard or installed in other ways as needed. The processor can process instructions executed within the computer device, including instructions stored in the memory or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some alternative embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (such as a server array, a set of blade servers, or a multi-processor system). Figure 7 Taking one processor 10 as an example in

[0202] The processor 10 can be a central processing unit, a network processor, or a combination thereof. Among them, the processor 10 can further include a hardware chip. The above hardware chip can be an application-specific integrated circuit, a programmable logic device, or a combination thereof. The above programmable logic device can be a complex programmable logic device, a field programmable gate array, a generic array logic, or any combination thereof.

[0203] Among them, the memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiments.

[0204] The memory 20 can include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data created according to the use of the computer device, etc. In addition, the memory 20 can include high-speed random access memory, and can also include non-transitory memory, such as at least one disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some alternative embodiments, the memory 20 can optionally include a memory remotely set relative to the processor 10, and these remote memories can be connected to the computer device through a network. Examples of the above network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0205] The memory 20 can include volatile memory, such as random access memory; the memory can also include non-volatile memory, such as flash memory, a hard disk, or a solid-state drive; the memory 20 can also include a combination of the above types of memory.

[0206] The computer device further includes a communication interface 30 for the computer device to communicate with other devices or communication networks.

[0207] Embodiments of the present invention also provide a computer-readable storage medium. The method according to the embodiments of the present invention can be implemented in hardware, firmware, or be implemented as computer code that can be recorded on a storage medium, or be implemented as computer code that is originally stored in a remote storage medium or a non-transitory machine-readable storage medium and downloaded through a network and will be stored in a local storage medium, so that the method described herein can be stored as such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only memory, a random access memory, a flash memory, a hard disk, or a solid-state drive, etc.; further, the storage medium can also include a combination of the above-mentioned types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by the computer, the processor, or the hardware, the method shown in the above embodiments is implemented.

[0208] A part of the present invention can be applied as a computer program product, such as computer program instructions. When executed by a computer, through the operation of the computer, the methods and / or technical solutions according to the present invention can be called or provided. Those skilled in the art should understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executes the instructions, or the computer compiles the instructions and then executes the corresponding compiled program, or the computer reads and executes the instructions, or the computer reads and installs the instructions and then executes the corresponding installed program. Herein, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to the computer.

[0209] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for generating a wind turbine blade inspection path based on an unmanned aerial vehicle, characterized in that: The method comprises: Acquire a structural parameter set of a fan to be detected, where the fan to be detected is in any shutdown state; Based on the structural parameter set, the wind turbine nacelle yaw angle and the wind turbine blade position angle of the wind turbine to be tested are obtained through processing by a wind turbine shutdown posture measurement algorithm; Establishing a wind turbine hub coordinate system by using a preset northeast celestial coordinate system, and determining a fixed attitude parameter set of the wind turbine to be detected according to the wind turbine hub coordinate system and the structural parameter set; Based on the yaw angle of the wind turbine nacelle, the position angle of the wind turbine blades and the fixed attitude parameter set, calculating multiple longitude and latitude high coordinate values ​​of multiple key path points in the preset northeast sky coordinate system; A fan blade inspection path of the wind turbine to be inspected is generated according to the multiple longitude and latitude coordinate values.

2. The method according to claim 1, characterized in that Based on the structural parameter set, after being processed by the wind turbine shutdown posture measurement algorithm, the wind turbine nacelle yaw angle and the wind turbine blade position angle of the wind turbine to be detected are obtained, including: Based on the structural parameter set, using a drone to obtain a video image set of the wind turbine to be inspected; Processing the video image set with a blade image instance segmentation algorithm based on deep learning to obtain the yaw angle of the wind turbine nacelle of the wind turbine to be detected; Based on the yaw angle of the wind turbine nacelle, using the drone to obtain a wind turbine image set of the wind turbine to be inspected; The fan image set is processed by the deep learning-based blade image instance segmentation algorithm to obtain the fan blade position angle of the fan to be detected.

3. The method according to claim 1, characterized in that: Based on the yaw angle of the wind turbine nacelle, the position angle of the wind turbine blades and the fixed attitude parameter set, multiple longitude and latitude high coordinate values ​​of multiple key path points in the preset northeast sky coordinate system are calculated, including: Calculating a plurality of first spatial coordinate values ​​of a plurality of key path points in the wind turbine hub coordinate system using the fixed attitude parameter set; Based on the fixed attitude parameter set, the yaw angle of the wind turbine nacelle and the position angle of the wind turbine blades, the multiple first space coordinate values ​​are converted to obtain multiple second space coordinate values ​​of multiple key path points in the preset northeast celestial coordinate system; The plurality of second spatial coordinate values ​​are converted into the plurality of latitude and longitude high coordinate values.

4. The method according to claim 3, characterized in that: Calculating a plurality of first spatial coordinate values ​​of a plurality of key path points in the wind turbine hub coordinate system using the fixed attitude parameter set includes: Using the fixed attitude parameter set, calculating a plurality of third spatial coordinate values ​​of a plurality of key path points on the windward surface of the wind turbine to be detected in the wind turbine hub coordinate system, and a plurality of fourth spatial coordinate values ​​when the wind turbine to be detected is in an inverted Y-shaped state; Using a rotation matrix formula, the plurality of fourth spatial coordinate values ​​are converted into a plurality of fifth spatial coordinate values ​​of a plurality of leeward key path points of the wind turbine to be detected in the wind turbine hub coordinate system; The plurality of first space coordinate values ​​are determined according to the plurality of third space coordinate values, the plurality of fourth space coordinate values, and the plurality of fifth space coordinate values.

5. The method according to claim 3, characterized in that: Based on the fixed attitude parameter set, the yaw angle of the wind turbine nacelle and the position angle of the wind turbine blades, the multiple first space coordinate values ​​are converted to obtain multiple second space coordinate values ​​of multiple key path points in the preset northeast sky coordinate system, including: Determine a first space rotation matrix and a translation space matrix respectively using the fixed posture parameter set; Determining a second spatial rotation matrix using the wind turbine nacelle yaw angle; Determining a third spatial rotation matrix using the fan blade position angle; Determine a target rotation change matrix based on the first spatial rotation matrix, the translation spatial matrix, the second spatial rotation matrix and the third spatial rotation matrix; The plurality of first space coordinate values ​​are transformed using the target rotation change matrix to obtain the plurality of second space coordinate values.

6. The method according to claim 1, characterized in that The method further comprises: Based on the wind turbine blade inspection path, a drone is used to inspect the wind turbine to be inspected and obtain an inspection image set.

7. A wind turbine blade inspection path generation device based on an unmanned aerial vehicle, characterized in that: The device comprises: An acquisition module, used for acquiring a set of structural parameters of a fan to be detected, wherein the fan to be detected is in any shutdown state; A processing module, configured to obtain a yaw angle of a wind turbine nacelle and a position angle of a wind turbine blade of the wind turbine to be detected based on the structural parameter set and processed by a wind turbine shutdown posture measurement algorithm; A determination module, used to establish a wind turbine hub coordinate system using a preset northeast celestial coordinate system, and determine a fixed attitude parameter set of the wind turbine to be detected according to the wind turbine hub coordinate system and the structural parameter set; A calculation module, used for calculating a plurality of latitude and longitude high coordinate values ​​of a plurality of key path points in the preset northeast sky coordinate system based on the yaw angle of the wind turbine nacelle, the position angle of the wind turbine blades and the fixed attitude parameter set; A generating module is used to generate a fan blade inspection path of the fan to be inspected according to the multiple longitude and latitude high coordinate values.

8. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the wind turbine blade inspection path generation method based on a drone as described in any one of claims 1 to 6 by executing the computer instructions.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the wind turbine blade inspection path generation method based on a drone according to any one of claims 1 to 6.

10. A computer program product, characterized in that It includes computer instructions, which are used to enable a computer to execute the wind turbine blade inspection path generation method based on a drone as described in any one of claims 1 to 6.

Images