Unmanned aerial vehicle-based blade detection method and device, electronic equipment and storage medium

Abstract

This application discloses a method, apparatus, electronic device, and storage medium for blade inspection based on unmanned aerial vehicles (UAVs), relating to the field of image detection technology. The method includes: acquiring images of individual blades of a wind turbine using a UAV; for any given individual blade image, performing detection using a preset detection model to obtain a detection result; and outputting an alarm message if corrosion is detected, wherein the alarm message includes the detection result, the individual blade image, and the wind turbine to which the individual blade image belongs. This application achieves unmanned automated inspection, replacing manual inspection, which can improve inspection efficiency, reduce inspection risks, and thus achieve ideal inspection results that are difficult to obtain in complex environments, meeting the needs of current inspection work.

Description

Technical Field

[0001] This application relates to the field of image detection technology, and in particular to a method, apparatus, electronic device and storage medium for leaf detection based on unmanned aerial vehicles (UAVs). Background Technology

[0002] Currently, with the increase in China's cumulative installed wind power capacity, the single-unit capacity and blade length of wind turbines are also increasing, which brings challenges to wind turbine inspection work. Regarding wind turbine blades, the blade length has increased from 20-30 meters to over 100 meters, and the tower height has also increased to over 200 meters, significantly increasing the difficulty and workload of inspection work. Current traditional manual inspection methods mainly rely on high-powered telescopes, high-altitude descent for visual inspection (such as the "spider-man" method), and blade maintenance platform inspections. These methods are inefficient, risky, and difficult to achieve ideal inspection results in complex environments, making them even less suitable for the current inspection requirements. The above content is only for assisting in understanding the technical solution of this application and does not represent an admission that the above content is prior art. Traditional inspection methods are insufficient to meet the current requirements of wind turbine inspection work. Summary of the Invention

[0003] The main purpose of this application is to provide a blade inspection method, device, electronic equipment and storage medium based on UAV, which aims to solve the technical problem that traditional inspection methods cannot meet the task requirements of current wind turbine inspection work.

[0004] To achieve the above objectives, this application proposes a blade detection method based on unmanned aerial vehicles (UAVs), which includes the following steps:

[0005] Images of individual blades of a wind turbine were obtained using drones.

[0006] For any one of the local leaf images, the local leaf image is detected by a preset detection model to obtain the detection result of the local leaf image;

[0007] If the detection result indicates the presence of corrosion, an alarm message is output, which includes the detection result, the partial blade image, and the wind turbine to which the partial blade image belongs.

[0008] Optionally, the step of detecting the local leaf image using a preset detection model to obtain the detection result of the local leaf image includes:

[0009] The time of the capture of the partial blade image, the geographical location when the partial blade image was captured, the weather conditions when the partial blade image was captured, and the wind turbine speed when the partial blade image was captured are determined.

[0010] The shooting time, geographical location, weather conditions, wind turbine speed, and local blade image are input into the preset detection model to obtain the detection result of the local blade image.

[0011] Optionally, before the step of detecting the local leaf image using a preset detection model to obtain the detection result of the local leaf image, the method further includes:

[0012] The original image set is obtained by acquiring the original images of the wind turbine blades;

[0013] After the original images in the original image set are labeled, a sample image set is obtained;

[0014] The sample images in the sample image set are subjected to derivative processing to expand the sample image set, resulting in an expanded sample image set;

[0015] The initial classification model is iteratively trained using the expanded set of sample images, and the initial classification model that meets the preset training conditions is used as the preset detection model.

[0016] Optionally, the step of performing derivation processing on the sample images in the sample image set includes at least one of the following:

[0017] Add a dynamic blur effect to the sample images in the sample image set;

[0018] Add background noise to the sample images in the sample image set;

[0019] Add weather noise to the sample images in the sample image set.

[0020] Optionally, the step of acquiring images of individual blades of the wind turbine using a drone includes:

[0021] The drone was used to capture an overall image of the wind turbine.

[0022] The hub component of the wind turbine is identified from the overall image based on an object detection algorithm;

[0023] Control the drone to move to an initial position set at a preset distance directly in front of the wheel hub component;

[0024] The movement step length of the drone is calculated based on the preset distance and the field of view of the drone's camera;

[0025] The drone is controlled to move in a preset direction by a certain step size, and the blades of the wind turbine are photographed to obtain a subset of local blade images, wherein the preset direction is parallel to the blade plane, and the blade plane is the plane in which each blade of the wind turbine is located;

[0026] Return to the step of controlling the UAV to move in a direction parallel to the wind turbine blades by the specified movement step size until a preset movement condition is reached, wherein the obtained subset of local blade images is combined to form the respective local blade images.

[0027] Optionally, the step of capturing images of the wind turbine blades to obtain a subset of local blade images includes:

[0028] The shooting interval is calculated based on the wind turbine rotation speed, and the number of shots is determined based on the number of blades.

[0029] Determine the initial time point for photographing the leaves;

[0030] Starting from the initial time point, the drone is controlled to photograph the leaves according to the shooting interval until the shooting number is reached, thereby obtaining the subset of local leaf images.

[0031] Optionally, the step of determining the initial time point for photographing the leaf includes:

[0032] The drone is controlled to continuously photograph the blades to obtain candidate images.

[0033] For any candidate image, identify the leaf region in the candidate image and determine the leaf center of the leaf region;

[0034] The distribution position of the leaf center in the candidate image is determined, and the leaf deviation of the candidate image is calculated based on the positional distance between the distribution position and the center position of the candidate image, wherein the leaf deviation increases with the increase of the positional distance;

[0035] The initial time point is calculated based on the shooting time point of the candidate image with the smallest leaf deviation among all candidate images and the shooting interval.

[0036] Furthermore, to achieve the above objectives, this application also proposes a blade inspection device based on unmanned aerial vehicles (UAVs), the UAV-based blade inspection device comprising:

[0037] The acquisition module is used to acquire images of various parts of the wind turbine blades using a drone;

[0038] The detection module is used to detect any one of the local leaf images using a preset detection model to obtain the detection result of the local leaf image.

[0039] An alarm module is used to output an alarm prompt when the detection result indicates the presence of corrosion. The alarm prompt includes the detection result, the local blade image, and the wind turbine to which the local blade image belongs.

[0040] In addition, to achieve the above objectives, this application also proposes an electronic device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the UAV-based blade detection method described above.

[0041] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the UAV-based blade detection method described above.

[0042] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the UAV-based blade detection method described above.

[0043] One or more technical solutions proposed in this application have at least the following technical effects:

[0044] In this embodiment, images of individual blades of a wind turbine are acquired using a drone. For any given individual blade image, a preset detection model is used to detect corrosion, resulting in a detection result. If corrosion is detected, an alarm is output, including the detection result, the individual blade image, and the wind turbine to which it belongs. This embodiment uses a drone to replace manual labor in capturing blade images, improving efficiency and avoiding safety risks associated with manual image capture. Furthermore, a pre-trained model replaces manual visual identification of individual blade images, reducing labor costs and improving accuracy. This application achieves automated, unmanned inspection, improving efficiency and reducing risks, thus addressing the challenges of achieving ideal inspection results in complex environments and meeting current inspection requirements. Attached Figure Description

[0045] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0046] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 This is a flowchart illustrating the first embodiment of the blade detection method based on unmanned aerial vehicles (UAVs) in this application.

[0048] Figure 2 This is a flowchart illustrating the second embodiment of the blade detection method based on unmanned aerial vehicles (UAVs) in this application.

[0049] Figure 3 This is a flowchart illustrating the third embodiment of the blade detection method based on unmanned aerial vehicles (UAVs) in this application.

[0050] Figure 4 This is a schematic diagram of the blade detection device based on an unmanned aerial vehicle (UAV) according to this application;

[0051] Figure 5 This is a schematic diagram of the device structure of the hardware operating environment involved in the blade detection method based on UAV in the embodiments of this application.

[0052] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0053] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0054] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0055] Currently, with the increase in China's cumulative installed wind power capacity, the single-unit capacity and blade length of wind turbines are also increasing, which brings challenges to wind turbine inspection work. Regarding wind turbine blades, the blade length has increased from 20-30 meters to over 100 meters, and the tower height has also increased to over 200 meters, significantly increasing the difficulty and workload of inspection work. Traditional manual inspection methods mainly rely on high-powered telescopes, high-altitude descent for visual inspection (such as the "spider-man" method), and blade maintenance platform inspections. These methods are inefficient, risky, and difficult to achieve ideal inspection results in complex environments, making them even less suitable for the current inspection requirements.

[0056] The main solution of this application embodiment is: to acquire images of each local blade of the wind turbine through a drone;

[0057] For any one of the local leaf images, the local leaf image is detected by a preset detection model to obtain the detection result of the local leaf image;

[0058] If the detection result indicates the presence of corrosion, an alarm message is output, which includes the detection result, the partial blade image, and the wind turbine to which the partial blade image belongs.

[0059] In this embodiment, drones are used to replace manual labor in capturing leaf images, thereby improving work efficiency and avoiding the safety risks associated with manual leaf image capture. Simultaneously, a pre-trained model is used to replace manual visual identification of specific leaf images, reducing labor costs and improving identification accuracy. It is understood that this application achieves unmanned automated inspection, replacing manual labor, which can improve inspection efficiency, reduce inspection risks, and thus achieve ideal inspection results in complex environments, meeting the current inspection task requirements.

[0060] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program execution functions, such as a drone, cloud platform, electronic device, main controller in electronic device, computer, mobile phone, etc., or an electronic device capable of realizing the above functions.

[0061] Based on this, embodiments of this application provide a blade detection method based on an unmanned aerial vehicle (UAV), referring to... Figure 1 This is a flowchart illustrating the first embodiment of the blade detection method based on unmanned aerial vehicles (UAVs) in this application.

[0062] In this embodiment, the UAV-based blade detection method includes steps S10 to S30:

[0063] Step S10: Obtain images of various local blades of the wind turbine using a drone;

[0064] It should be noted that in this embodiment, the main focus is on inspecting the blades of a wind turbine generator. The aforementioned UAV-based blade inspection method can be applied to UAVs or to control devices that communicate with the UAV, such as computers or cloud servers.

[0065] For example, in this embodiment, a drone can be controlled to approach a wind turbine. The wind turbine can be located at sea or on land. Then, the camera on the drone is used to capture partial images of the wind turbine blades (typically, the blades are long enough that it is difficult to capture the entire blade in one shot, so partial images of the blades are captured), thereby obtaining images of each partial blade of the wind turbine.

[0066] Understandably, drones can replace manual labor in taking images of leaf blades, thereby improving work efficiency and avoiding the safety risks associated with manual leaf blade photography.

[0067] Step S20: For any one of the local leaf images, the local leaf image is detected by a preset detection model to obtain the detection result of the local leaf image;

[0068] It should be noted that in this embodiment, a pre-trained model will be used to replace manual visual identification of local leaf images, thereby reducing labor costs and improving identification accuracy. Since the identification process for each local leaf image is basically the same, this embodiment will use one as an example for explanation.

[0069] For example, for any one local blade image in the various local blade images, a preset detection model can be used to detect that local blade image. If the local blade image is input into the preset detection model, the model outputs a detection result. The result can be whether corrosion exists in the local image or not. If corrosion exists, the result can further include the type of corrosion, which can include: environmental corrosion (e.g., wind turbines are usually installed outdoors, exposed to wind, sun, and rain, especially in marine or coastal areas where salt spray accelerates corrosion of metal components, and in high-humidity environments, water films easily form on metal surfaces, leading to electrochemical corrosion); stress corrosion (some components of wind turbines may experience stress corrosion cracking under continuous stress in specific corrosive environments); fatigue corrosion (due to alternating loads during wind turbine operation, fatigue corrosion may occur); and other types of corrosion, which will not be elaborated here. Furthermore, the aforementioned preset detection model is pre-trained. For example, training samples can be obtained by manually annotating images, and then the model can be trained using these training samples. Correspondingly, the annotations can indicate whether corrosion exists or the type of corrosion.

[0070] Step S30: If the detection result indicates the presence of corrosion, an alarm message is output, wherein the alarm message includes the detection result, the local blade image, and the wind turbine to which the local blade image belongs.

[0071] For example, if the detection result indicates the presence of corrosion, a corresponding alarm can be output. This alarm can be issued via a terminal device connected to inspection or maintenance personnel, or via the control platform for centralized wind turbine management. Furthermore, the alarm can include the detection result obtained from the aforementioned model, an image of the corroded section of the blade, and the wind turbine to which that blade belongs. This facilitates maintenance of the corroded blades by the appropriate personnel.

[0072] In this embodiment, images of individual blades of a wind turbine are acquired using a drone. For any given individual blade image, a preset detection model is used to detect corrosion, resulting in a detection result. If corrosion is detected, an alarm is output, including the detection result, the individual blade image, and the wind turbine to which it belongs. This embodiment uses a drone to replace manual labor for capturing blade images, improving efficiency and avoiding the safety risks associated with manual blade image capture. Furthermore, a pre-trained model replaces manual visual identification of individual blade images, reducing labor costs and improving accuracy. This application achieves unmanned automated inspection, improving efficiency and reducing risks, thus addressing the challenges of achieving ideal inspection results in complex environments and meeting current inspection requirements.

[0073] Reference Figure 2 This is a second embodiment of the UAV-based blade detection method proposed in the first embodiment above. Contents in this embodiment that are the same as or similar to those in the above embodiments can be referred to the above description and will not be repeated hereafter. The step of detecting the local blade image using a preset detection model to obtain the detection result of the local blade image includes steps S21 to S22:

[0074] Step S21: Determine the shooting time of the partial blade image, the geographical location when the partial blade image was taken, the weather conditions when the partial blade image was taken, and the wind turbine speed when the partial blade image was taken.

[0075] Step S22: Input the shooting time, geographical location, weather conditions, wind turbine speed and the local blade image into the preset detection model to obtain the detection result of the local blade image.

[0076] It should be noted that current image recognition solutions typically use a single image as input to the model. However, in the application scenario of this embodiment, the captured image is affected by the shooting environment, which may affect the model's detection results. Therefore, to improve the accuracy of the detection results, a local leaf image obtained by combining the shooting conditions at the time of taking the photo is also input into the model, allowing the model to obtain more dimensional information and ensuring the accuracy of the model's detection results.

[0077] For example, the shooting time of the partial blade image, the geographical location at which the partial blade image was taken, the weather conditions at which the partial blade image was taken, the wind turbine speed at which the partial blade image was taken, and the partial blade image itself are input into a preset detection model, which then outputs the detection result of the partial blade image. It is understood that the most important factors affecting the quality of a photograph or image are the illumination of the blades and their movement. Therefore, in this embodiment, shooting time, geographical location, and weather conditions are added to characterize illumination conditions. Weather conditions can include the illumination intensity collected by the drone through sensors and the type of weather (e.g., fog, rain). It is worth noting that fog may cause obstruction, and rain will form small dewdrops on the blade surface. Additionally, in this embodiment, photos can be taken while the wind turbine is running. However, photos taken while the turbine is running may have ghosting, so wind turbine speed is added as a feature dimension. When the drone takes a partial blade image, it can obtain the wind turbine speed at the time of shooting through the wind turbine operation and maintenance platform, and associate and save the wind turbine speed with the partial blade image. Correspondingly, the associated and saved information also includes the shooting time, geographical location at the time of shooting, and weather conditions at the time of shooting. This allows for the direct acquisition of various conditions of a local leaf image through correlation during local leaf image detection. In a feasible implementation, before the step of detecting the local leaf image using a preset detection model to obtain the detection result of the local leaf image, the method further includes steps S210 to S240:

[0078] Step S210: Acquire the original images of the wind turbine blades to obtain the original image set;

[0079] Step S220: After the original images in the original image set are labeled, a sample image set is obtained;

[0080] Step S230: Perform derivative processing on the sample images in the sample image set to expand the sample image set and obtain an expanded sample image set;

[0081] Step S240: Iteratively train the initial classification model using the expanded set of sample images, and use the initial classification model that meets the preset training conditions as the preset detection model.

[0082] It should be noted that in the detection of the local leaf image using the preset detection model, this embodiment will train the initial model to obtain a preset detection model with image detection capability.

[0083] For example, a raw image set can be obtained by collecting raw images of wind turbine blades using a drone. The process of collecting raw images of wind turbine blades is similar to that described above for acquiring images of individual blades of a wind turbine using a drone, so it will not be repeated here. The collected raw image set can be manually labeled, such as indicating whether corrosion exists and the type of corrosion. The labeled raw images are the sample images, so after labeling the raw image set, a sample image set can be obtained. In practical applications, the sample image set can also be directly used for model training. However, in this embodiment, the sample images in the sample image set will be further processed to obtain new sample images, thereby enriching the sample types and expanding the number of samples, resulting in an expanded sample image set. The expanded sample image set is then used to iteratively train the initial classification model. The initial classification model can be a neural network classification model. During the iterative training process, for any sample image input to the initial classification model, the initial classification model outputs a classification result. Then, based on the difference between the classification result and the label of the sample image, the model parameters in the initial classification model are updated until the initial classification model reaches the preset training conditions. The initial classification model that has reached the preset training conditions is then used as the preset detection model. The preset training conditions can be the number of times the model is trained, or the model convergence, etc., and no specific limitations are made here.

[0084] It is understandable that, since the sample images in the augmented set are more abundant and diverse, training the model with the augmented set can improve the model's detection performance and avoid overfitting.

[0085] In one feasible implementation, the step of performing derivation processing on the sample images in the sample image set includes at least one of the following:

[0086] Step S231: Add a dynamic blur effect to the sample images in the sample image set;

[0087] Step S232: Add background noise to the sample images in the sample image set;

[0088] Step S233: Add weather noise to the sample images in the sample image set.

[0089] For example, in this embodiment, various derivative processing methods can be applied to the sample images in the sample image set. In practical applications, for any sample image, one or more derivative methods can be selected to process it. The derivative methods may include: adding a motion blur effect to the sample images in the sample image set, which mainly addresses the ghosting problem that may occur when a drone photographs moving leaves; adding background noise to the sample images in the sample image set, which mainly addresses the problem of different backgrounds that may appear behind the leaves when photographing them, such as birds, trees, sky, clouds, etc., which will not be elaborated here; and adding weather noise to the sample images in the sample image set, which can be fog, rain, etc.

[0090] Understandably, different derivative processing can greatly enrich the sample types, enabling the preset detection model to cope with different natural interferences in the shooting environment and improve the robustness of the preset detection model.

[0091] Reference Figure 3 This is a third embodiment of the UAV-based blade detection method in this application, based on the first and second embodiments described above. Contents in this embodiment that are the same as or similar to those in the above embodiments can be referred to the above description and will not be repeated hereafter. The steps of acquiring images of various local blades of the wind turbine using a UAV include steps S11 to S16:

[0092] Step S11: Acquire an overall image of the wind turbine using the drone;

[0093] Step S12: Identify the hub component of the wind turbine from the overall image based on the target detection algorithm;

[0094] Step S13: Control the drone to move to an initial position set at a preset distance directly in front of the wheel hub component;

[0095] Step S14: Calculate the movement step length of the drone based on the preset distance and the field of view of the drone's camera;

[0096] Step S15: Control the UAV to move in a preset direction by the movement step length, and take pictures of the blades of the wind turbine to obtain a subset of local blade images, wherein the preset direction is parallel to the blade plane, and the blade plane is the plane where each blade of the wind turbine is located.

[0097] Step S16: Return to the step of controlling the UAV to move in a direction parallel to the wind turbine blades by the specified movement step length until a preset movement condition is reached, wherein the obtained subset of local blade images is combined into the respective local blade images.

[0098] It should be noted that in this embodiment of the application, a drone will be used to take photos of the wind turbine in motion, so a drone with strong wind resistance can be selected.

[0099] For example, a drone can first be used to acquire an overall image of the wind turbine. Then, a target detection algorithm is used to identify the hub component of the wind turbine from the overall image. The outline component can include the front and back of the outline component, or the windward or non-windward side of the hub component. The processing procedures for both sides are largely the same, so they will not be described separately in this embodiment. The drone is controlled to move to an initial position set at a preset distance directly in front of the hub component. The preset distance can be set by a technician according to actual needs. The drone's movement step length is then calculated based on the preset distance and the drone's camera field of view. The movement step length can be calculated using mathematical geometric formulas. For example, the ratio of the preset distance to half the step length (i.e., a trigonometric function relationship) can be calculated using the field of view. Then, the half step length is calculated using the ratio and the known preset distance. Multiplying the half step length by two yields the movement step length.

[0100] Accordingly, after determining the movement step size, the drone is controlled to move in the preset direction by that step size and take pictures of the wind turbine blades, obtaining a subset of local blade images, such as taking one picture each time a blade passes by. It is also important to note that the preset direction must be parallel to the blade plane, which refers to the plane containing each blade of the wind turbine. After completing one shot, the process returns to controlling the drone to move in the direction parallel to the wind turbine blade by the same step size until a preset movement condition is met. At this point, the shooting is considered complete, and the resulting subsets of local blade images are grouped into the aforementioned local blade images. The preset condition can be the number of moves, which can be the result of dividing the blade length by the movement step size; another preset condition is that the blade does not pass through the local blade image, meaning the blade tip is captured.

[0101] In one feasible implementation, the step of capturing images of the wind turbine blades to obtain a subset of local blade images includes steps S151 to S152:

[0102] Step S151: Calculate the shooting interval based on the wind turbine rotation speed and determine the number of shots based on the number of blades;

[0103] Step S152: Determine the initial time point for photographing the leaf;

[0104] For example, the rotational speed of the large wind turbine is obtained from the wind turbine operation and maintenance platform. The shooting interval is then calculated based on the rotational speed, and the number of shots is determined based on the number of blades. The shooting interval is obtained by dividing the angle between two adjacent blades by the rotational speed. The initial time point for shooting the blades is determined; this initial time point refers to the point at which the actual shooting begins and a partial blade image is obtained. For example, the drone can first take multiple images continuously, and then the shooting time of the image with the largest blade area can be selected as the initial time point. After determining the initial time point, the drone is controlled to continuously shoot the blades according to the shooting interval until the number of shots determined in the above steps is reached. The partial blade images captured in this process constitute a subset of partial blade images.

[0105] In one feasible implementation, the step of determining the initial time point for photographing the blade includes steps S1521 to S1524:

[0106] Step S1521: Control the UAV to continuously photograph the blades to obtain candidate images;

[0107] Step S1522: For any candidate image, identify the leaf region in the candidate image and determine the leaf center of the leaf region;

[0108] Step S1523: Determine the distribution position of the leaf center in the candidate image, and calculate the leaf deviation of the candidate image based on the positional distance between the distribution position and the center position of the candidate image, wherein the leaf deviation increases with the increase of the positional distance;

[0109] Step S1524: Calculate the initial time point based on the shooting time point of the candidate image with the smallest leaf deviation among the candidate images and the shooting interval.

[0110] For example, when determining the initial time point, a drone can be controlled to continuously photograph the blades, thereby obtaining candidate images. These candidate images are then filtered. For any given candidate image, an edge detection algorithm can be used to detect lines. Based on prior knowledge (such as the blade outline being relatively smooth; the approximate direction of the blade outline being known from the drone's shooting position; and the blade typically being a white area), the outline lines belonging to the wind turbine blade are filtered out. This identifies the blade region in the candidate image. Furthermore, the geometric center of this blade region, i.e., the blade center, is identified. The distribution position of the blade center in the candidate images is then identified, such as the position of the corresponding pixel relative to the candidate image. The blade deviation of the candidate image is calculated based on the distance between the distribution position and the center position of the candidate image. The blade deviation increases with increasing distance, i.e., the blade deviation is directly proportional to the distance. After traversing each candidate image, the candidate image with the smallest blade deviation is selected. The initial time point is then calculated using the shooting time point and shooting interval of the candidate image with the smallest blade deviation. Alternatively, the initial time point can be obtained by directly using the shooting time point as the initial time point, or by adding the shooting time point to n times the shooting interval. Here, n can be set by the technicians, which will not be elaborated here.

[0111] It is understood that, in this embodiment, the above control process can ensure that images of various positions of the blades are captured while the wind turbine is running.

[0112] Reference Figure 4 This is a schematic diagram of the UAV-based blade inspection device provided in this application. The UAV-based blade inspection device includes:

[0113] The acquisition module 10 acquires images of various parts of the wind turbine blades using a drone;

[0114] The detection module 20 detects any one of the local leaf images using a preset detection model to obtain the detection result of the local leaf image.

[0115] The output module 30 outputs an alarm message when the detection result indicates the presence of corrosion. The alarm message includes the detection result, the local blade image, and the wind turbine to which the local blade image belongs.

[0116] Optionally, the detection module 20 is further configured to:

[0117] The time of the capture of the partial blade image, the geographical location when the partial blade image was captured, the weather conditions when the partial blade image was captured, and the wind turbine speed when the partial blade image was captured are determined.

[0118] The shooting time, geographical location, weather conditions, wind turbine speed, and local blade image are input into the preset detection model to obtain the detection result of the local blade image.

[0119] Optionally, the detection module 20 is further configured to:

[0120] The original image set is obtained by acquiring the original images of the wind turbine blades;

[0121] After the original images in the original image set are labeled, a sample image set is obtained;

[0122] The sample images in the sample image set are subjected to derivative processing to expand the sample image set, resulting in an expanded sample image set;

[0123] The initial classification model is iteratively trained using the expanded set of sample images, and the initial classification model that meets the preset training conditions is used as the preset detection model.

[0124] Optionally, the detection module 20 is further configured to:

[0125] Add a dynamic blur effect to the sample images in the sample image set;

[0126] Add background noise to the sample images in the sample image set;

[0127] Add weather noise to the sample images in the sample image set.

[0128] Optionally, the acquisition module 10 is further configured to:

[0129] The drone was used to capture an overall image of the wind turbine.

[0130] The hub component of the wind turbine is identified from the overall image based on an object detection algorithm;

[0131] Control the drone to move to an initial position set at a preset distance directly in front of the wheel hub component;

[0132] The movement step length of the drone is calculated based on the preset distance and the field of view of the drone's camera;

[0133] The drone is controlled to move in a preset direction by a certain step size, and the blades of the wind turbine are photographed to obtain a subset of local blade images, wherein the preset direction is parallel to the blade plane, and the blade plane is the plane in which each blade of the wind turbine is located;

[0134] Return to the step of controlling the UAV to move in a direction parallel to the wind turbine blades by the specified movement step size until a preset movement condition is reached, wherein the obtained subset of local blade images is combined to form the respective local blade images.

[0135] Optionally, the acquisition module 10 is further configured to:

[0136] The shooting interval is calculated based on the wind turbine rotation speed, and the number of shots is determined based on the number of blades.

[0137] Determine the initial time point for photographing the leaves;

[0138] Starting from the initial time point, the drone is controlled to photograph the leaves according to the shooting interval until the shooting number is reached, thereby obtaining the subset of local leaf images.

[0139] Optionally, the acquisition module 10 is further configured to:

[0140] The drone is controlled to continuously photograph the blades to obtain candidate images.

[0141] For any candidate image, identify the leaf region in the candidate image and determine the leaf center of the leaf region;

[0142] The distribution position of the leaf center in the candidate image is determined, and the leaf deviation of the candidate image is calculated based on the positional distance between the distribution position and the center position of the candidate image, wherein the leaf deviation increases with the increase of the positional distance;

[0143] The initial time point is calculated based on the shooting time point of the candidate image with the smallest leaf deviation among all candidate images and the shooting interval.

[0144] The UAV-based blade inspection device provided in this application employs the UAV-based blade inspection method described in the above embodiments, aiming to solve the technical problem that traditional inspection methods are unable to meet the task requirements of current wind turbine inspection work. Compared with the prior art, the beneficial effects of the UAV-based blade inspection device provided in this application are the same as those of the UAV-based blade inspection method provided in Embodiment 1 above, and other technical features in this UAV-based blade inspection device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0145] This application provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the UAV-based blade detection method described in Embodiment 1 above.

[0146] The following is for reference. Figure 5 The diagram illustrates a structural schematic suitable for implementing the electronic device in the embodiments of this application. The electronic device in the embodiments of this application may include, but is not limited to, mobile terminals such as computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as computers. Figure 5 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0147] like Figure 5 As shown, the electronic device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the electronic device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. While electronic devices with various systems are shown in the figures, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.

[0148] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0149] The electronic device provided in this application employs the UAV-based blade inspection method described in the above embodiments, which can solve the technical problem that traditional inspection methods are unable to meet the task requirements of current wind turbine inspection work. Compared with the prior art, the beneficial effects of the electronic device provided in this application are the same as those of the UAV-based blade inspection method provided in the above embodiments, and other technical features of the electronic device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0150] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0151] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0152] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, which are used to execute the UAV-based blade detection method in the above embodiments.

[0153] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0154] The aforementioned computer-readable storage medium may be included in an electronic device or may exist independently without being assembled into an electronic device.

[0155] The aforementioned computer-readable storage medium carries one or more programs, which, when executed by an electronic device, cause the electronic device to:

[0156] Images of individual blades of a wind turbine were obtained using drones.

[0157] For any one of the local leaf images, the local leaf image is detected by a preset detection model to obtain the detection result of the local leaf image;

[0158] If the detection result indicates the presence of corrosion, an alarm message is output, which includes the detection result, the partial blade image, and the wind turbine to which the partial blade image belongs.

[0159] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0160] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0161] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0162] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., computer programs) for executing the aforementioned UAV-based blade inspection method. This solves the technical problem that traditional inspection methods cannot meet the task requirements of current wind turbine inspection work. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the UAV-based blade inspection method provided in the above embodiments, and will not be repeated here.

[0163] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the UAV-based blade detection method described above.

[0164] The computer program product provided in this application can solve the technical problem of blade detection based on unmanned aerial vehicles (UAVs). Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the UAV-based blade detection method provided in the above embodiments, and will not be repeated here.

[0165] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A blade detection method based on unmanned aerial vehicles (UAVs), characterized in that, The UAV-based blade detection method includes the following steps: Images of individual blades of a wind turbine are acquired using a drone, wherein the wind turbine is located at sea or on land; For any one of the local leaf images, the local leaf image is detected by a preset detection model to obtain the detection result of the local leaf image; If the detection result indicates the presence of corrosion, an alarm is output. The alarm includes the detection result, a partial image of the blade, and the wind turbine to which the partial blade image belongs. The detection result includes the type of corrosion, which may be environmental corrosion, stress corrosion, or fatigue corrosion. The steps of acquiring images of various local blades of a wind turbine using a drone include: The shooting interval is calculated based on the rotational speed of the wind turbine, and the number of shots is determined based on the number of blades; Determine the initial time point for photographing the leaves; Starting from the initial time point, the drone is controlled to photograph the leaves according to the shooting interval until the number of shooting times is reached, thereby obtaining a subset of local leaf images; The step of determining the initial time point for photographing the leaf includes: The drone is controlled to continuously photograph the blades to obtain candidate images. For any candidate image, identify the leaf region in the candidate image and determine the leaf center of the leaf region; The distribution position of the leaf center in the candidate image is determined, and the leaf deviation of the candidate image is calculated based on the positional distance between the distribution position and the center position of the candidate image, wherein the leaf deviation increases with the increase of the positional distance; The initial time point is calculated based on the shooting time point of the candidate image with the smallest leaf deviation among all candidate images and the shooting interval; The step of detecting the local leaf image using a preset detection model to obtain the detection result of the local leaf image includes: The time of the capture of the partial blade image, the geographical location when the partial blade image was captured, the weather conditions when the partial blade image was captured, and the wind turbine speed when the partial blade image was captured are determined. The shooting time, geographical location, weather conditions, wind turbine speed, and local blade image are input into a preset detection model to obtain the detection result of the local blade image.

2. The blade detection method based on unmanned aerial vehicles as described in claim 1, characterized in that, Before the step of detecting the local leaf image using a preset detection model to obtain the detection result of the local leaf image, the method further includes: The original image set is obtained by acquiring the original images of the wind turbine blades; After the original images in the original image set are labeled, a sample image set is obtained; The sample images in the sample image set are subjected to derivative processing to expand the sample image set, resulting in an expanded sample image set; The initial classification model is iteratively trained using the expanded set of sample images, and the initial classification model that meets the preset training conditions is used as the preset detection model.

3. The blade detection method based on unmanned aerial vehicles as described in claim 2, characterized in that, The step of performing derivation processing on the sample images in the sample image set includes at least one of the following: Add a dynamic blur effect to the sample images in the sample image set; Add background noise to the sample images in the sample image set; Add weather noise to the sample images in the sample image set.

4. The blade detection method based on unmanned aerial vehicles as described in claim 1, characterized in that, The step of acquiring images of various local blades of a wind turbine using a drone also includes: The drone was used to capture an overall image of the wind turbine. The hub component of the wind turbine is identified from the overall image based on an object detection algorithm; Control the drone to move to an initial position set at a preset distance directly in front of the wheel hub component; The movement step length of the drone is calculated based on the preset distance and the field of view of the drone's camera; The drone is controlled to move in a preset direction by a certain step size, and the blades of the wind turbine are photographed to obtain a subset of local blade images, wherein the preset direction is parallel to the blade plane, and the blade plane is the plane in which each blade of the wind turbine is located; Return to the step of controlling the UAV to move in the preset direction by the specified movement step size until the preset movement condition is reached, wherein the obtained subset of local blade images is combined into the local blade images.

5. A blade inspection device based on an unmanned aerial vehicle (UAV), characterized in that, The UAV-based blade inspection device includes: An acquisition module is used to acquire images of individual blades of a wind turbine using a drone. The steps of acquiring these images include: calculating the shooting interval based on the wind turbine's rotational speed and determining the number of shots based on the number of blades; determining the initial time point for shooting the blades; controlling the drone to shoot the blades at the shooting interval from the initial time point until the number of shots is reached, thus obtaining a subset of individual blade images. The step of determining the initial time point for shooting the blades includes: controlling the drone to continuously shoot the blades, obtaining candidate images; for any candidate image, identifying the blade region within the candidate image and determining the blade center of that region; determining the distribution position of the blade center in the candidate image; calculating the blade deviation of the candidate image based on the positional distance between the distribution position and the center position of the candidate image, wherein the blade deviation increases with the positional distance; and calculating the initial time point based on the shooting time point of the candidate image with the smallest blade deviation and the shooting interval. The detection module is used to detect any one of the partial blade images in the various partial blade images using a preset detection model to obtain the detection result of the partial blade image. The step of detecting the partial blade image using the preset detection model to obtain the detection result includes: determining the shooting time of the partial blade image, the geographical location when the partial blade image was taken, the weather conditions when the partial blade image was taken, and the wind turbine speed when the partial blade image was taken; inputting the shooting time, the geographical location, the weather conditions, the wind turbine speed, and the partial blade image into the preset detection model to obtain the detection result of the partial blade image. An alarm module is used to output an alarm prompt when the detection result indicates the presence of corrosion. The alarm prompt includes the detection result, the local blade image, and the wind turbine to which the local blade image belongs. The detection result includes the type of corrosion, which includes environmental corrosion, stress corrosion, and fatigue corrosion.

6. An electronic device, characterized in that, The electronic device includes: a processor, a memory, and a UAV-based blade detection program stored in the memory and executable on the processor, wherein the UAV-based blade detection program, when executed, implements the steps of the UAV-based blade detection method as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a UAV-based blade detection program, which, when executed, implements the steps of the UAV-based blade detection method as described in any one of claims 1 to 4.

Images