Fan blade angle detection method and unmanned aerial vehicle fan blade inspection method

Abstract

The invention relates to the technical field of fan detection, and discloses a fan blade angle detection method and an unmanned aerial vehicle fan blade inspection method.The method comprises the steps that a fan image shot from the side face of a blade is obtained; determining a first line segment representing the tower drum and a second line segment representing the blade in the image through a line segment detection model; and calculating the included angle between the blade and the tower drum based on the position information of the first line segment and the position information of the second line segment, the fan blade and the straight line segment corresponding to the tower drum are determined in an image recognition mode, and the situation of the included angle between the tower drum and the blade of the fan in the actual state can be determined more conveniently and accurately.

Description

Technical Field

[0001] The present invention relates to the technical field of fan detection, and in particular to a fan blade angle detection method and a fan blade inspection method using an unmanned aerial vehicle. Background Art

[0002] When using drones to inspect the condition of wind turbines (referred to as wind turbines), it is often necessary to plan the flight trajectory of the drone in advance to avoid collisions between the drone and the blades of the wind turbine during the inspection process, and to ensure the safety of the drone during the inspection process. In order to ensure the accuracy of the drone's flight trajectory planning, it is necessary to plan the drone's flight trajectory based on specific information such as the specific length of the wind turbine blades, the actual height of the wind turbine, and the angle between the blades and the tower.

[0003] In the prior art, when determining the angle between the wind turbine blades and the tower, a fixed angle value is often determined based on the wind turbine specifications for drone path planning. However, during the actual operation of the wind turbine, the angle will change. Therefore, if a fixed angle value is used for drone path planning, the planned drone path may not conform to the current actual situation, resulting in risks in the drone detection process. Summary of the invention

[0004] In view of this, the present invention provides a fan blade angle detection method and a UAV fan blade inspection method to solve the problem that the fan angle under actual conditions cannot be determined, resulting in risks in the UAV detection process.

[0005] In a first aspect, the present invention provides a method for detecting the angle of a wind turbine blade, wherein the wind turbine comprises a tower and blades, and the method comprises:

[0006] Acquire an image of the wind turbine taken from the side of the blade;

[0007] Determine, by a line segment detection model, a first line segment representing the tower and a second line segment representing the blade in the image;

[0008] Based on the position information of the first line segment and the position information of the second line segment, the angle between the blade and the tower is calculated.

[0009] The method provided by the present invention identifies an image taken from the side of a wind turbine blade to obtain a first line segment and a second line segment in the image for characterizing a tower and a blade, thereby determining the angle between the blade and the tower of the wind turbine based on specific position information of the first line segment and the second line segment. This method can quickly and accurately determine the current angle between the blade and the tower of the wind turbine, thereby facilitating planning of a drone inspection route based on the actual angle to inspect the wind turbine.

[0010] In an optional implementation, the determining, by a line segment detection model, a first line segment representing the tower and a second line segment representing the blade in the image includes:

[0011] Identify the line segment to be determined in the image by using the line segment detection model;

[0012] Traversing the relative lengths and positional relationships of any two line segments among all the line segments to be determined, and determining the line segments representing the two edges of the tower;

[0013] Determine the center line of the line segments at the two edges of the tower as the first line segment;

[0014] Traverse the positions of the two endpoints of any line segment among all the line segments to be determined, calculate the distance between the two endpoints of the line segment to obtain the length of the line segment, and among the line segments to be determined whose lengths meet the preset length interval and whose slopes in the image meet the preset slope interval, determine the line segment corresponding to the endpoint with the highest endpoint position as the second line segment.

[0015] In this embodiment, after the line segment detection model detects multiple line segments in the image, the corresponding line segments are determined by the matching rules corresponding to the tower and the blades to achieve accurate and efficient matching and improve the accuracy of the angle finally calculated.

[0016] In an optional implementation, identifying the line segment to be determined in the image by using the line segment detection model includes:

[0017] All line segments in the image are identified by the line segment detection model, and the line segments to be determined among all the line segments are determined according to the length, inclination angle and credibility of each line segment.

[0018] In this embodiment, after all line segments in the image are determined by the line segment detection model, preliminary screening is performed based on the length, inclination and credibility of each line segment to improve the efficiency of subsequent line segment matching.

[0019] In an optional implementation, the determining, using a line segment detection model, a first line segment representing the tower and a second line segment representing the blade in the image further includes:

[0020] Determine a plurality of first candidate line segments representing the tower and a plurality of second candidate line segments representing the blades in the image;

[0021] Determining confidences of the plurality of first candidate line segments and the plurality of second candidate line segments;

[0022] The first candidate line segment corresponding to the highest confidence is determined as the first line segment, and the second candidate line segment corresponding to the highest confidence is determined as the second line segment.

[0023] In this embodiment, the first candidate line segment corresponding to the tower and the second candidate line segment corresponding to the blade in the image are determined, and then according to the confidence of each candidate line segment, the first line segment corresponding to the tower and the second line segment corresponding to the blade are determined from the first candidate line segment and the second candidate line segment respectively, thereby ensuring efficiency in matching the corresponding line segments of the tower and the blade.

[0024] In an optional implementation, the confidence of the first candidate line segment is determined based on the inclination angle and / or length of the line segment;

[0025] The confidence level of the second candidate line segment is determined based on the inclination angle and / or the length of the line segment.

[0026] In this implementation, the confidences corresponding to different line segments are determined by the specific inclination angles and lengths of the line segments, thereby achieving the determination of the first line segment and the second line segment.

[0027] In an optional implementation, the image of the wind turbine taken from the side of the blade includes multiple frames of images at consecutive moments;

[0028] The calculating the angle between the blade and the tower based on the position information of the first line segment and the position information of the second line segment includes:

[0029] Based on the position information of the first line segment and the position information of the second line segment in each frame of the image, calculating the angle between the blade and the tower in each frame of the image;

[0030] If the number of images corresponding to the angle being 0 is greater than the number of images corresponding to the angle not being 0, the angle between the blade and the tower is determined to be 0.

[0031] In this embodiment, the angles corresponding to multiple frames of images are determined, and when the number of images corresponding to zero angles is too large, the angle between the tower and the blade is determined to be zero.

[0032] In an optional embodiment, the method further includes:

[0033] If the number of images corresponding to the angle of 0 is less than or equal to the number of images corresponding to the angle of non-0, determining the angle weight corresponding to the image with the angle of non-0;

[0034] The angle between the blade and the tower is determined by performing weighted averaging based on the angle corresponding to the image corresponding to the angle not being 0 and the angle weight.

[0035] In this embodiment, when there are more images with non-zero angles, the weights corresponding to these images are determined, so that the angle between the blade and the tower is calculated according to the specific weights, so that the finally calculated angle can be more consistent with the actual situation.

[0036] In an optional implementation, the earlier the time point when the image is captured, the smaller the weight of the angle.

[0037] In this method, the weight corresponding to the angle is determined by the time when the image is taken, so that the angle weight corresponding to the image taken later is higher, thereby ensuring that the angle finally calculated is more in line with the current actual situation.

[0038] In an optional implementation, the line segment detection model includes: an M-LSD straight line detection model.

[0039] In this implementation, the M-LSD straight line detection model is used to perform line segment detection on the image, which can achieve a better line segment detection effect.

[0040] In a second aspect, the present invention provides a method for inspecting wind turbine blades using a drone, the method comprising:

[0041] Based on the wind turbine blade angle detection method described in the first aspect or any corresponding embodiment thereof, calculating the angle between the blade and the tower;

[0042] Planning an inspection route for the drone to inspect the wind turbine based on the angle;

[0043] The drone is controlled to inspect the wind turbine along the inspection route.

[0044] The method provided in this aspect detects the angle between the current wind turbine blades and the tower, determines the inspection path of the drone when inspecting the wind turbine, and thus inspects the wind turbine, which can effectively avoid danger during the drone inspection process.

[0045] In a third aspect, the present invention provides a device for detecting the angle of a wind turbine blade, wherein the wind turbine comprises a tower and blades, and the device comprises:

[0046] An image acquisition module, used to acquire an image of the wind turbine taken from the side of the blade;

[0047] A line segment determination module, used to determine a first line segment representing the tower and a second line segment representing the blade in the image through a line segment detection model;

[0048] The angle calculation module is used to calculate the angle between the blade and the tower based on the position information of the first line segment and the position information of the second line segment.

[0049] In a fourth aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, computer instructions being stored in the memory, and the processor executing the fan blade angle detection method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0050] In a fifth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the fan blade angle detection method of the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0052] Figure 1 is a schematic flow chart of a method for detecting a fan blade angle according to an embodiment of the present invention;

[0053] Figure 2 is an example diagram of a side view of a wind turbine according to an embodiment of the present invention;

[0054] Figure 3 is a schematic flow chart of another fan blade angle detection method according to an embodiment of the present invention;

[0055] Figure 4 is an example diagram of a straight line segment after image recognition according to an embodiment of the present invention;

[0056] Figure 5 is an example diagram of a wind turbine tower corresponding to a straight line segment according to an embodiment of the present invention;

[0057] Figure 6 is an example diagram of an angle detection process according to an embodiment of the present invention;

[0058] Figure 7 is an example diagram of a process of performing straight line recognition by using OpenCV in a traditional manner according to an embodiment of the present invention;

[0059] Figure 8 is a structural block diagram of a fan blade angle detection device according to an embodiment of the present invention;

[0060] Fig. 9 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0062] When using drones to inspect the condition of wind turbines, it is often necessary to plan the flight trajectory of the drone in advance to avoid collisions between the drone and the blades of the wind turbine during inspection and to ensure the safety of the drone during the inspection process. In order to ensure the accuracy of the drone's flight trajectory planning, it is necessary to plan the drone's flight trajectory based on specific information such as the specific length of the wind turbine blades, the actual height of the wind turbine, and the angle between the blades and the tower.

[0063] In the prior art, when determining the angle between the wind turbine blades and the tower, a fixed angle value is often determined based on the wind turbine specifications for drone path planning. However, during the actual operation of the wind turbine, the angle will change. Therefore, if a fixed angle value is used for drone path planning, the planned drone path may not conform to the current actual situation, resulting in risks in the drone detection process.

[0064] To this end, an embodiment of the present invention provides a method for detecting the angle of a wind turbine blade. By identifying an image taken from the side of the wind turbine blade, a first line segment and a second line segment in the image for characterizing the tower and the blade are obtained, and then the angle between the blade and the tower of the wind turbine is determined based on the specific position information of the first line segment and the second line segment. The current angle between the blade and the tower of the wind turbine can be quickly and accurately determined, thereby facilitating planning of a drone inspection path based on the actual angle to detect the wind turbine.

[0065] According to an embodiment of the present invention, an embodiment of a method for detecting the angle of a wind turbine blade is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown can be executed in an order different from that shown here and the final result can be obtained.

[0066] In this embodiment, a method for detecting the angle of a wind turbine blade is provided, which can be used for detecting the angle between the blade and the tower of the wind turbine. Figure 1 is a flow chart of a method for detecting a fan blade angle according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0067] Step S101, acquiring an image of the wind turbine taken from the side of the blade.

[0068] A wind turbine is also called a wind turbine. It is mainly composed of blades that are responsible for generating electricity by rotating under wind force and a tower that supports the blades. The windward side of the blade is defined as the front side of the blade. In the image taken from the side of the blade, the multiple blades of the wind turbine are in a straight line. Specifically, the image of the wind turbine taken from the side of the blade can be an image taken by a drone at a preset position. For example, Figure 2 Shown is an example diagram of a side view of a wind turbine according to an embodiment of the present invention.

[0069] Step S102: determining a first line segment representing the tower and a second line segment representing the blade in the image by using a line segment detection model.

[0070] The line segment detection model can identify straight lines in an image. In an image containing a wind turbine, in addition to the wind turbine, there will be other elements in a straight line state, such as the boundaries of houses, skylines, roads on the ground, etc. After identifying the straight line segments corresponding to these objects, the line segment detection model can determine the specific image parts of the wind turbine blades and wind turbine tower according to the relevant determination rules of the wind turbine in the image. For example, corresponding judgment rules are set for the inclination, length, position and other information of the line segment, so as to identify the first line segment corresponding to the tower and the second line segment corresponding to the blade.

[0071] Step S103, calculating the angle between the blade and the tower based on the position information of the first line segment and the position information of the second line segment.

[0072] After determining the first line segment corresponding to the tower and the second line segment corresponding to the blade, the angle between the two line segments can be determined according to their specific positions in the image. This angle can be regarded as the angle between the blade and the tower of the wind turbine.

[0073] The wind turbine blade angle detection method provided in this embodiment identifies an image taken from the side of the wind turbine blade to obtain the first line segment and the second line segment in the image used to characterize the tower and the blade, thereby determining the angle between the wind turbine blade and the tower according to the specific position information of the first line segment and the second line segment, and can quickly and accurately determine the current angle between the wind turbine blade and the tower, thereby facilitating planning of the drone inspection path according to the actual angle to detect the wind turbine. The angle between the wind turbine blade and the tower will change with the size and direction of the wind force, and the angle between the blade and the tower can be calculated using the image, so that the change of the angle between the blade and the tower can be monitored at any time to obtain a more accurate angle value.

[0074] In one or more embodiments, Figure 3 As shown, Figure 3FIG. 4 is a flow chart of another fan blade angle detection method according to an embodiment of the present invention. Figure 3 As shown, the process includes the following steps:

[0075] Step S201, acquiring an image of the wind turbine taken from the side of the blade.

[0076] Specifically, the image of the wind turbine taken from the side of the blade includes multiple frame images at consecutive moments.

[0077] It can be understood that multiple frames of wind turbine images can be taken on the side of the blades, and the angle between the wind turbine blades and the tower in each frame can be determined respectively, so as to determine the angle of the wind turbine in the current period according to the angle in each image, so as to improve the accuracy of the final determined wind turbine blade angle.

[0078] Step S202: determining a first line segment representing the tower and a second line segment representing the blade in the image through a line segment detection model.

[0079] Specifically, in step S202, it includes:

[0080] Step S2021, identifying the line segments to be determined in the image through the line segment detection model.

[0081] The line segment detection model can identify straight line elements in the image and mark the corresponding line segments, for example, wind turbine blades, towers, skylines in the background, roads on the ground and other elements in a straight line state in the image. After determining the line segments corresponding to these elements, their corresponding credibility, inclination in the image, length, endpoint coordinates and other information can be determined. These line segments are preliminarily screened according to preset rules, for example, line segments with low credibility and short length are eliminated to improve the efficiency of determining the first and second line segments corresponding to the towers and blades respectively.

[0082] Specifically, in step S2021, identifying the line segment to be determined in the image by using the line segment detection model includes:

[0083] All line segments in the image are identified through the line segment detection model, and the line segments to be determined among all the line segments are determined according to the length, inclination angle and credibility of each line segment.

[0084] It can be understood that after all the identified line segments in the image are determined by the line segment detection model, the line segments can be preliminarily screened according to specific screening rules. Specifically, the line segments with low credibility can be eliminated. At the same time, since the wind turbine blades and towers are both vertical in the image and have a long length, the line segments with a large inclination, that is, the line segments with a larger angle than the vertical axis of the image and a shorter length can be eliminated.

[0085] Furthermore, the line segment detection model includes: M-LSD line segment detection model. This model is a line segment detection model based on a neural network, which can have a better recognition effect when identifying straight line segments in an image, thereby more accurately and efficiently identifying multiple straight line segments in each frame of the wind turbine image.

[0086] For example, please refer to Figure 4 FIG. 1 is an example diagram of a straight line segment after image recognition according to an embodiment of the present invention. Figure 4 The white line segments in the image are the straight line segments corresponding to different elements in the image identified by the image recognition model, which can be obtained from Figure 4 As can be seen from the figure, a straight line segment corresponding to the wind turbine blade and two straight line segments corresponding to the edges on both sides of the wind turbine tower are identified.

[0087] By using a neural network-based line segment detection model to identify straight line segments in an image, all straight line segments in the image can be determined more accurately and quickly. At the same time, its use of neural networks for straight line detection has better robustness in the face of complex environments than conventional machine vision methods.

[0088] In actual application scenarios, the conventional process of OpenCV for line segmentation can also be used to identify straight line segments in images. However, when OpenCV is used for line segmentation, the light, contrast, and grayscale in the image will affect the threshold during grayscale or binarization, thereby affecting the detection accuracy, and the parameters need to be repeatedly adjusted in different scenarios, and the stability and robustness are poor. Preferably, a line segment detection model based on a neural network can be used for image recognition.

[0089] Step S2022, traverse the relative lengths and positional relationships of any two line segments among all the line segments to be determined, determine the line segments representing the two edges of the tower; and determine the center lines of the line segments at the two edges of the tower as the first line segment.

[0090] When the line segment detection model recognizes straight lines in an image, it will identify the straight lines corresponding to the two sides of the tower. Due to the physical properties of the tower, the straight lines corresponding to the two sides are nearly parallel and have similar lengths. Therefore, two line segments can be traversed as a group, that is, any two line segments are traversed. In each traversal, the relative length and position relationship of the two line segments are determined to determine whether the length difference of the two line segments is less than the preset value, and whether the position relationship is close to the parallel state, that is, the relative angle difference is less than the corresponding preset value. In the actual implementation process, there may be multiple groups of line segments whose relative length and relative angle differences are less than the corresponding preset values. The group of line segments that are closest to the parallel state or have the smallest relative length difference is determined as the line segments at the two edges of the tower.

[0091] After determining the line segments corresponding to the two sides of the tower, the center lines of the two sides are determined as the first line segment corresponding to the tower. For example, Figure 5 As shown, an example diagram of a straight line segment corresponding to a wind turbine tower according to an embodiment of the present invention is shown. In some cases, the two sides of the tower are completely parallel, and the line segment corresponding to any side can be determined as the line segment corresponding to the tower to calculate the subsequent angle value.

[0092] Step S2023, traverse the positions of the two endpoints of any line segment among all the line segments to be determined, calculate the distance between the two endpoints of the line segment to obtain the length of the line segment, and among the line segments to be determined whose length meets the preset length range and whose slope in the image meets the preset slope range, determine the line segment corresponding to the endpoint with the highest endpoint position as the second line segment.

[0093] When determining the line segment corresponding to the fan blade, since the line segment corresponding to the fan blade in the image is relatively long and has a relatively large slope, that is, it is in a relatively vertical state in the image, and one end of the fan blade is at the highest point in the image. Therefore, the positions of the two endpoints of any line segment can be traversed to determine the length of the traversed line segment, and the line segments that meet the preset length range and the preset slope range are determined in the line segments to be determined, and the highest endpoint is determined in these qualified line segments, and the line segment corresponding to the endpoint is determined as the second line segment representing the blade.

[0094] After the line segment detection model detects multiple line segments in the image, the corresponding line segments are determined by the matching rules corresponding to the tower and the blades to achieve accurate and efficient matching and improve the accuracy of the angle finally calculated.

[0095] Exemplarily, in one example, using a line segment detection model to determine a first line segment representing a tower and a second line segment representing a blade in an image further includes:

[0096] Determine a plurality of first candidate line segments representing the tower and a plurality of second candidate line segments representing the blades in the image;

[0097] Determining confidences of the plurality of first candidate line segments and the plurality of second candidate line segments;

[0098] The first candidate line segment corresponding to the highest confidence is determined as the first line segment, and the second candidate line segment corresponding to the highest confidence is determined as the second line segment.

[0099] It can be understood that after the line segment detection model determines the line segments in the image, these line segments are classified to determine the first candidate line segment corresponding to the tower and the second candidate line segment corresponding to the blade. Since the tower and the blade appear vertical in the image and are relatively long, due to some shooting errors and the actual inclination of the blade, the tower and the blade may not be completely vertical, that is, they are in a relatively vertical state in the image, but the angle relative to the longitudinal axis of the image is also relatively small. Therefore, the inclination of each line segment can be determined based on the angle of each line segment in the image relative to the longitudinal axis, and it is determined that the inclination corresponding to the tower is in a certain range, and the blade corresponds to another range, as well as the range limit corresponding to the line segment length, and the first candidate line segment corresponding to the tower and the second candidate line segment corresponding to the blade are screened out.

[0100] Exemplarily, the inclination interval corresponding to the tower is A to B degrees, and the inclination interval corresponding to the blade is C to D degrees, so that all line segments are screened according to the degree range to determine the first candidate line segment corresponding to the tower and the second candidate line segment corresponding to the blade.

[0101] At the same time, the line segment detection model can also calculate the specific confidence level based on the inclination and length corresponding to each candidate line segment. The confidence level can measure the probability that each candidate line segment is a tower or a blade.

[0102] Specifically, the confidence of the first candidate line segment is determined based on the inclination angle and / or length of the line segment; the confidence of the second candidate line segment is determined based on the inclination angle and / or length of the line segment.

[0103] It can be understood that after determining the first candidate line segment and the second candidate line segment, the corresponding confidence can be further calculated according to the inclination and length of each candidate line segment. For example, the longer the candidate line segment, the greater its corresponding confidence; the smaller the inclination, the greater its corresponding confidence; or the tower and the blade are divided into more detailed length intervals and inclination intervals, and the corresponding confidence is determined according to the intervals where the length and inclination of each candidate line segment are located. Specifically, the specific confidence calculation rules can be set according to the actual situation, and there is no limitation here.

[0104] Step S203: Calculate the angle between the blade and the tower based on the position information of the first line segment and the position information of the second line segment.

[0105] In the description of step S202, it is explained how to determine the line segments corresponding to the blades and the tower in a frame of image. The angle between the blades and the tower can be calculated according to the positions of the determined line segments.

[0106] In some cases, the image of the wind turbine taken from the side of the blade includes multiple frames of images at consecutive moments. In this case, in the above step S203, the angle between the blade and the tower is calculated based on the position information of the first line segment and the position information of the second line segment, including:

[0107] Based on the position information of the first line segment and the position information of the second line segment in each frame of the image, calculating the angle between the blade and the tower in each frame of the image;

[0108] If the number of images corresponding to an angle of 0 is greater than the number of images corresponding to an angle not being 0, the angle between the blade and the tower is determined to be 0.

[0109] It can be understood that the angle between the blade and the tower corresponding to each frame of image can be calculated through the position information corresponding to the first line segment representing the tower and the second line segment representing the blade in each frame of image. The number of images with an angle of 0 and the number of images with an angle not of 0 are determined in these multiple frames of images and compared. If the number of images with an angle of 0 is greater, the angle between the blade and the tower is determined to be 0.

[0110] In some cases, if the number of images corresponding to an angle of 0 is less than or equal to the number of images corresponding to an angle not being 0, then the angle weights corresponding to the images having an angle not being 0 are determined;

[0111] The angle between the blade and the tower is determined by taking a weighted average based on the angle corresponding to the image corresponding to the angle not being 0 and the angle weight.

[0112] When determining the angle weight corresponding to an image with an angle not equal to 0, the corresponding weight can be determined according to the time when the image was taken. Specifically, the earlier the image was taken, the smaller the angle weight. It can be understood that the later the image was taken, the more the angle it was taken in line with the current actual situation, so a larger weight can be given. The angle between the wind turbine blade and the tower calculated based on the weight can be more in line with the actual situation.

[0113] The method for detecting the angle of wind turbine blades provided in the present embodiment obtains the first line segment and the second line segment in each image for characterizing the tower and the blades by identifying multiple frames of images taken from the side of the wind turbine blades, thereby determining the angle between the blades and the tower of the wind turbine corresponding to each frame of the image based on the specific position information of the first line segment and the second line segment, thereby determining the wind turbine angle that is more in line with the actual state by combining the number of images with zero angles and the weights corresponding to each image with non-zero angles, thereby facilitating planning of the drone inspection path based on the actual angle conditions for detecting the wind turbine.

[0114] According to an embodiment of the present invention, a method for inspecting wind turbine blades using a drone is also provided, which is as follows:

[0115] Step 1: According to any of the above-mentioned wind turbine blade angle detection method embodiments and implementation methods thereof, calculate the angle between the blade and the tower.

[0116] The above method embodiment provides a specific method for determining the angle between the blade and the tower, which will not be repeated here.

[0117] Step 2: Plan the inspection route of the wind turbine using the drone based on the angle.

[0118] When planning the inspection route of a drone, specific wind turbine parameters are often input, such as wind turbine height, blade length, tower diameter, single blade width, blade angle range, and other specific information, so as to plan the inspection route of the drone according to the preset rules. Therefore, after determining the actual angle between the blade and the tower, the conventional inspection route planning method can be combined to plan an accurate inspection route to improve the safety and detection scanning coverage of the drone during wind turbine inspection.

[0119] Step 3: Control the drone to inspect the wind turbine along the inspection route.

[0120] After determining the inspection route of the drone, the drone can be controlled through the corresponding program to inspect the wind turbine according to the inspection route.

[0121] The UAV wind turbine blade inspection method provided in accordance with the embodiment of the present invention can effectively improve the safety and detection accuracy of UAV wind turbine inspections.

[0122] In order to facilitate understanding of the above-mentioned fan blade angle detection method embodiment, the embodiment of the present invention also provides an angle detection process example diagram, such as Figure 6 shown.

[0123] First, the drone autonomously flies to the side of the wind turbine through the preset target position point and maintains a suitable distance so that the entire wind turbine outline falls into the camera's field of view and occupies a sufficiently large frame ratio. In this way, it can be ensured that the target wind turbine whose angle is to be calculated occupies the main frame in the image and avoids interference from other surrounding wind turbines. When the drone arrives at the pre-specified shooting position, it starts taking pictures. At the same time, the M-LSD line segmentation model starts running and begins to extract all straight line segments according to the incoming image frame sequence. The output of the model is finally parsed into a list containing the coordinates of both ends of all line segments and the confidence of the line. The M-LSD line segmentation model can specify the confidence score and the expected straight line pixel length during detection to preliminarily filter out interfering straight line segments, which can avoid most of the straight line detection interference caused by the complex background in the graphics, and finally retain the straight line segments with appropriate length and high confidence.

[0124] After the straight line segments are initially filtered, the two red and blue straight lines representing the plane where the blades are located and the center of the column required for calculating the angle can be matched from the remaining straight lines. It can be understood that in actual application scenarios, the plane where the blades are located and the straight line segments corresponding to the center of the column can be distinguished by the color of the line segments. The red line corresponds to the plane where the fan blades are located, and the blue line corresponds to the fan column, that is, the fan tower mentioned in the above embodiment.

[0125] The matching process of the plane where the fan blades are located corresponding to the red line is as follows: Figure 2 The state shown is an example. When the blade rotates to the position shown in the figure, it is easiest to determine the straight line segment corresponding to the fan blade, because at this time one end point of the straight line segment corresponding to the fan blade is at the highest height, that is, the pixel corresponding to this end point in the image has the smallest y value. In addition, due to the influence of the mechanical structure of the fan, the angle of the fan can usually only change within a small range, so the red line should remain nearly perpendicular to the horizontal line, that is, the slope of the red line is nearly infinite. Adding this constraint can filter out interference with red line detection at the intersection of the skyline and the ground at some shooting angles, such as Figure 4 The identified white straight line segment corresponds to the junction of the skyline and the ground.

[0126] For the fan cylinder, the matching method of the corresponding straight line segment is that the two sides of the cylinder are two highly recognizable straight lines, which are almost perpendicular to the image plane, and the two straight lines are almost parallel and of equal height. Using this feature, the two straight lines on the side can be clearly detected using the traversal method, and the center lines of the two straight lines can be calculated to obtain the blue line representing the center of the cylinder. Finally, after determining the straight line segments corresponding to the fan blades and the fan tower, the angle between the straight line segments can be calculated and determined as the fan angle.

[0127] When calculating the fan angle, affected by the complex lighting environment or preset model parameters, the M-LSD model may not be able to accurately identify the red and blue straight lines for all captured images. At this time, the model will output the uncalculated default angle, that is, zero degrees. When the actual fan angle is not zero, the output results of this part will interfere with the accurate results. Therefore, these erroneous angle values ​​should be filtered out. Compared with the 0-degree value output by the model algorithm when the actual fan angle is 0 degrees, the erroneous output result of 0-degree value when the actual angle is not 0 degrees is sporadic (only a few frames in the continuous image frames fail to correctly detect the red and blue straight lines). In this regard, the mode of the output result sequence can be taken to determine whether the 0-degree value of this part is the correct fan angle value.

[0128] After filtering out the erroneous results that are output by default due to inaccurate detection, the results of the remaining continuous frame sequences are filtered by weighted mean to stabilize the output results and reduce the interference caused by pixel deviation. The weight of each output result should take into account the influence of the time dimension. The closer the image frame is to the current time frame, the greater the weight, and the further back in time, the smaller the weight. Finally, the angle value of the fan is calculated based on the weight and angle corresponding to each frame of the fan image.

[0129] Specifically, in addition to the M-LSD model to identify straight line segments, OpenCV can also be used to perform straight line segmentation to identify straight line segments. For example, Figure 7 As shown, it is an example diagram of the process of straight line recognition through OpenCV in a traditional way. First, an image of the straight line segment to be detected is obtained, that is, the wind turbine image collected by the drone in the above embodiment, and then the image is grayed and binarized; the straight line segment is extracted by Canny edge detection and Hough transform. For specific content, please refer to relevant specific technical means, which will not be repeated here.

[0130] The method provided by the embodiment of the present invention can respectively identify the side view straight line of the plane where the blades are located and the center straight line of the cylinder by taking advantage of the fact that the blades and cylinder of the wind turbine appear as long straight lines in the image at a specific viewing angle, and calculate the angle between the two straight lines. Finally, based on the weights corresponding to the wind turbine images in different frames, the angle between the blades and the tower of the wind turbine can be obtained quickly and accurately.

[0131] In this embodiment, a fan blade angle detection device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and will not be repeated here. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0132] This embodiment provides a fan blade angle detection device, such as Figure 8 As shown, including:

[0133] An image acquisition module 401 is used to acquire an image of the wind turbine taken from the side of the blade;

[0134] A line segment determination module 402, configured to determine a first line segment representing a tower and a second line segment representing a blade in an image by using a line segment detection model;

[0135] The angle calculation module 403 is used to calculate the angle between the blade and the tower based on the position information of the first line segment and the position information of the second line segment.

[0136] In some optional implementations, the line segment determination module 402, when determining the first line segment representing the tower and the second line segment representing the blade in the image through the line segment detection model, includes:

[0137] Identify the line segments to be determined in the image by using a line segment detection model;

[0138] Traverse the relative lengths and positional relationships of any two line segments among all the line segments to be determined, and determine the line segments representing the two edges of the tower;

[0139] Determine the center line of the line segments at the two edges of the tower as the first line segment;

[0140] Traverse the positions of the two endpoints of any line segment among all the line segments to be determined, calculate the distance between the two endpoints of the line segment to obtain the length of the line segment, and among the line segments to be determined whose length meets the preset length interval and whose slope in the image meets the preset slope interval, determine the line segment corresponding to the endpoint with the highest endpoint position as the second line segment.

[0141] In some optional implementations, the line segment determination module 402, when identifying the line segment to be determined in the image through the line segment detection model, includes:

[0142] All line segments in the image are identified through the line segment detection model, and the line segments to be determined among all the line segments are determined according to the length, inclination angle and credibility of each line segment.

[0143] In some optional implementations, the line segment determination module 402, when determining the first line segment representing the tower and the second line segment representing the blade in the image using the line segment detection model, may further include:

[0144] Determine a plurality of first candidate line segments representing the tower and a plurality of second candidate line segments representing the blades in the image;

[0145] Determining confidences of the plurality of first candidate line segments and the plurality of second candidate line segments;

[0146] The first candidate line segment corresponding to the highest confidence is determined as the first line segment, and the second candidate line segment corresponding to the highest confidence is determined as the second line segment.

[0147] In some optional implementations, the line segment determination module 402, when determining the confidences of the plurality of first candidate line segments and the plurality of second candidate line segments, determines the confidences of the first candidate line segments based on the inclination angle and / or length of the line segments;

[0148] The confidence of the second candidate line segment is determined based on the inclination angle and / or the length of the line segment.

[0149] In some optional implementations, the image acquisition module 401, the image of the wind turbine taken from the side of the blade includes multiple frames of images at consecutive moments;

[0150] The angle calculation module 403, when calculating the angle between the blade and the tower based on the position information of the first line segment and the position information of the second line segment, includes:

[0151] Based on the position information of the first line segment and the position information of the second line segment in each frame of the image, calculating the angle between the blade and the tower in each frame of the image;

[0152] If the number of images corresponding to an angle of 0 is greater than the number of images corresponding to an angle not being 0, the angle between the blade and the tower is determined to be 0.

[0153] In some optional implementations, if the number of images corresponding to an angle of 0 is less than or equal to the number of images corresponding to an angle not being 0, the angle calculation module 403 determines the angle weights corresponding to the images having an angle not being 0;

[0154] The angle between the blade and the tower is determined by taking a weighted average based on the angle corresponding to the image corresponding to the angle not being 0 and the angle weight.

[0155] In some optional implementations, the angle calculation module 403 determines the angle weight corresponding to the image with a non-zero angle in the following manner: the earlier the image is captured, the smaller the angle weight is.

[0156] In some optional implementations, when the line segment determination module 402 determines the first line segment representing the tower and the second line segment representing the blade in the image through the line segment detection model, the line segment detection model includes: an M-LSD straight line detection model.

[0157] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0158] The fan blade angle detection device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0159] The embodiment of the present invention also provides a computer device having the above Figure 8 The fan blade angle detection device shown.

[0160] See also Fig. 9 , Fig. 9 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Fig. 9As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Fig. 9 A processor 10 is taken as an example.

[0161] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0162] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.

[0163] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0164] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0165] The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Fig. 9 The example of connecting through bus is taken in the following.

[0166] The input device 30 can receive input digital or character information, and generate key signal input related to the user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0167] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0168] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for detecting the angle of a fan blade, characterized in that: The wind turbine comprises a tower and blades, and the method comprises: Acquire an image of the wind turbine taken from the side of the blade; Determine, by a line segment detection model, a first line segment representing the tower and a second line segment representing the blade in the image; Based on the position information of the first line segment and the position information of the second line segment, the angle between the blade and the tower is calculated.

2. The method according to claim 1, characterized in that The determining, by using a line segment detection model, a first line segment representing the tower and a second line segment representing the blade in the image includes: Identify the line segment to be determined in the image by using the line segment detection model; Traversing the relative lengths and positional relationships of any two line segments among all the line segments to be determined, and determining the line segments representing the two edges of the tower; Determine the center line of the line segments at the two edges of the tower as the first line segment; Traverse the positions of the two endpoints of any line segment among all the line segments to be determined, calculate the distance between the two endpoints of the line segment to obtain the length of the line segment, and among the line segments to be determined whose length meets the preset length range and whose slope in the image meets the preset slope range, determine the line segment corresponding to the endpoint with the highest endpoint position as the second line segment.

3. The method according to claim 2, characterized in that The identifying the line segment to be determined in the image by using the line segment detection model includes: All line segments in the image are identified by the line segment detection model, and the line segments to be determined among all the line segments are determined according to the length, inclination angle and credibility of each line segment.

4. The method according to claim 1, characterized in that: The determining, by using a line segment detection model, a first line segment representing the tower and a second line segment representing the blade in the image further includes: Determine a plurality of first candidate line segments representing the tower and a plurality of second candidate line segments representing the blades in the image; Determining confidences of the plurality of first candidate line segments and the plurality of second candidate line segments; The first candidate line segment corresponding to the highest confidence is determined as the first line segment, and the second candidate line segment corresponding to the highest confidence is determined as the second line segment.

5. The method according to claim 4, characterized in that The confidence of the first candidate line segment is determined based on the inclination angle and / or length of the line segment; The confidence level of the second candidate line segment is determined based on the inclination angle and / or the length of the line segment.

6. The method according to any one of claims 1 to 5, characterized in that: The image of the wind turbine taken from the side of the blade includes multiple frames of images at consecutive moments; The calculating the angle between the blade and the tower based on the position information of the first line segment and the position information of the second line segment includes: Based on the position information of the first line segment and the position information of the second line segment in each frame of the image, calculating the angle between the blade and the tower in each frame of the image; If the number of images corresponding to the angle being 0 is greater than the number of images corresponding to the angle not being 0, the angle between the blade and the tower is determined to be 0.

7. The method according to claim 6, characterized in that The method further comprises: If the number of images corresponding to the angle of 0 is less than or equal to the number of images corresponding to the angle of non-0, determining the angle weight corresponding to the image with the angle of non-0; The angle between the blade and the tower is determined by performing weighted averaging based on the angle corresponding to the image corresponding to the angle not being 0 and the angle weight.

8. The method according to claim 7, characterized in that The earlier the time point when the image is captured, the smaller the weight of the angle.

9. The method according to any one of claims 1 to 5, characterized in that: The line segment detection model includes: an M-LSD straight line detection model.

10. A method for inspecting wind turbine blades using a drone, characterized in that: The method comprises: Utilizing the wind turbine blade angle detection method according to any one of claims 1 to 9, calculating the angle between the blade and the tower; Planning an inspection route for the drone to inspect the wind turbine based on the angle; The drone is controlled to inspect the wind turbine along the inspection route.

11. A fan blade angle detection device, characterized in that: The wind turbine comprises a tower and blades, and the device comprises: An image acquisition module, used to acquire an image of the wind turbine taken from the side of the blade; A line segment determination module, used to determine a first line segment representing the tower and a second line segment representing the blade in the image through a line segment detection model; The angle calculation module is used to calculate the angle between the blade and the tower based on the position information of the first line segment and the position information of the second line segment.

12. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the fan blade angle detection method according to any one of claims 1 to 9 by executing the computer instructions.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the wind turbine blade angle detection method according to any one of claims 1 to 9.

Images