Blade detection method and device based on unmanned aerial vehicle, electronic equipment and storage medium

Abstract

The invention discloses a blade detection method and device based on an unmanned aerial vehicle, electronic equipment and a storage medium, and relates to the technical field of image detection, and the method comprises the steps: obtaining each local blade image of a wind driven generator through the unmanned aerial vehicle; for any one of the local blade images, detecting the local blade image through a preset detection model to obtain a detection result of the local blade image; when the detection result is that corrosion exists, an alarm prompt is output, and the alarm prompt comprises the detection result, the local blade image and the wind driven generator to which the local blade image belongs. According to the invention, unmanned automatic inspection is realized to replace manual inspection, the inspection efficiency can be improved, the inspection risk can be reduced, an ideal inspection effect is difficult to achieve in a complex environment, and the requirements of current inspection work are met.

Description

Technical Field

[0001] The present application relates to the field of image detection technology, and in particular to a blade detection method, device, electronic device and storage medium based on an unmanned aerial vehicle. Background Art

[0002] At present, with the increase in the cumulative installed capacity of wind power in China, the unit capacity and blade length of wind turbines are also increasing, which brings challenges to the inspection of wind turbines. As far as wind turbine blades are concerned, the length of the blades has increased from 20 to 30 meters to more than 100 meters now, and the tower height has also increased to more than 200 meters, which has greatly increased the difficulty and workload of the inspection work. The current traditional manual inspection methods mainly rely on high-power telescope inspections, high-altitude circumnavigation and descent visual inspections (such as the Spider-Man method), blade maintenance platform inspections, etc. These methods are inefficient and high-risk, and it is difficult to achieve ideal inspection results in complex environments, and it is even more difficult to meet the inspection task requirements of current inspection work. The above content is only used to assist in understanding the technical solution of this application, and does not mean that the above content is recognized as prior art. Traditional inspection methods are difficult to meet the current task requirements of wind turbine inspections. Summary of the invention

[0003] The main purpose of this application is to provide a blade detection method, device, electronic equipment and storage medium based on a drone, aiming to solve the technical problem that traditional inspection methods are difficult to meet the task requirements of current wind turbine inspection work.

[0004] To achieve the above objectives, the present application proposes a blade detection method based on a drone, and the blade detection method based on a drone comprises the following steps: Use drones to obtain images of each local blade of a wind turbine; For any one of the partial leaf images, detecting the partial leaf image by using a preset detection model to obtain a detection result of the partial leaf image; When the detection result indicates that corrosion exists, an alarm prompt is output, wherein the alarm prompt includes the detection result, the local blade image, and the wind turbine to which the local blade image belongs.

[0005] Optionally, the step of detecting the local leaf image by using a preset detection model to obtain a detection result of the local leaf image includes: Determine the shooting time of the partial blade image, the geographical location when the partial blade image is shot, the weather conditions when the partial blade image is shot, and the fan speed when the partial blade image is shot; The shooting time, the geographical location, the weather conditions, the fan speed and the local blade image are input into the preset detection model to obtain the detection result of the local blade image.

[0006] Optionally, before the step of detecting the local leaf image by using a preset detection model to obtain a detection result of the local leaf image, the method further includes: Collecting original images of fan blades to obtain an original image set; After the original images in the original image set are marked, a sample image set is obtained; Performing derivative processing on sample images in the sample image set to expand the sample image set to obtain an expanded sample image set; The initial classification model is iteratively trained through the sample image expansion set, and the initial classification model that meets the preset training conditions is used as the preset detection model.

[0007] Optionally, the step of performing derivative processing on the sample images in the sample image set includes at least one of the following: Adding a dynamic blur effect to the sample images in the sample image set; adding background noise to the sample images in the sample image set; Weather noise is added to the sample images in the sample image set.

[0008] Optionally, the step of acquiring the image of each local blade of the wind turbine by using a drone includes: Collecting an overall image of the wind turbine by using the drone; Identifying a hub component of the wind turbine from the overall image based on a target detection algorithm; Controlling the UAV to move to an initial position set at a preset distance directly in front of the hub component; Calculating the moving step length of the drone based on the preset distance and the camera field of view of the drone; Controlling the moving step length of the drone to move in a preset direction, photographing the blades of the wind turbine to obtain a local blade image subset, wherein the preset direction is parallel to a blade plane, and the blade plane is a plane where each blade of the wind turbine is located; Return to the step of controlling the UAV to move in a direction parallel to the wind turbine blades until a preset movement condition is reached, wherein the obtained subsets of local blade images are combined into the local blade images.

[0009] Optionally, the step of photographing the blades of the wind turbine to obtain a local blade image subset comprises: Calculate the shooting interval according to the rotation speed of the wind turbine, and determine the number of shots according to the number of blades; Determine the initial time point for photographing leaves; From the initial time point, the drone is controlled to photograph the leaves according to the photographing interval until the number of photographing times is reached to obtain the local leaf image subset.

[0010] Optionally, the step of determining an initial time point for photographing the leaf includes: Controlling the UAV to continuously photograph the blades to obtain candidate images; For any candidate image, identifying a leaf region in the candidate image, and determining a leaf center of the leaf region; Determine the distribution position of the center of the blade in the candidate image, and calculate the blade deviation of the candidate image based on the position distance between the distribution position and the center position of the candidate image, wherein the blade deviation increases with the increase of the position distance; The initial time point is calculated according to the shooting time point of the candidate image with the smallest blade deviation among the candidate images and the shooting interval.

[0011] In addition, to achieve the above purpose, the present application also proposes a blade detection device based on a drone, and the blade detection device based on a drone includes: An acquisition module, used for acquiring images of each local blade of a wind turbine through a drone; A detection module, configured to detect any one of the partial leaf images by using a preset detection model to obtain a detection result of the partial leaf image; An alarm module is used to output an alarm prompt when the detection result shows that corrosion exists, wherein the alarm prompt includes the detection result, the local blade image and the wind turbine to which the local blade image belongs.

[0012] In addition, to achieve the above-mentioned purpose, the present application also proposes an electronic device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the drone-based blade detection method as described above.

[0013] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the drone-based blade detection method as described above are implemented.

[0014] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the steps of the drone-based blade detection method as described above.

[0015] One or more technical solutions proposed in this application have at least the following technical effects: In an embodiment of the present application, each local blade image of a wind turbine is obtained by a drone; for any local blade image in each local blade image, the local blade image is detected by a preset detection model to obtain the detection result of the local blade image; when the detection result is that corrosion exists, an alarm prompt is output, wherein the alarm prompt includes the detection result, the local blade image, and the wind turbine to which the local blade image belongs. In an embodiment of the present application, a drone will be used to replace manual work to complete the shooting of blade images, thereby improving work efficiency and avoiding the problem of safety risks in manual shooting of blade images. At the same time, a pre-trained model is used to replace manual naked eye recognition of local blade images, thereby reducing labor costs and improving recognition accuracy. It can be understood that the present application realizes unmanned automatic inspection instead of manual work, which can improve inspection efficiency and reduce inspection risks, so that it is difficult to achieve an ideal inspection effect in a complex environment to meet the inspection task requirements of the current inspection work. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings herein are incorporated in and constitute a part of the specification, illustrate embodiments consistent with the present application, and together with the description, are used to explain the principles of the present application.

[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0018] Figure 1 This is a flow chart of the first embodiment of the blade detection method based on drone of the present application; Figure 2 This is a flow chart of the second embodiment of the blade detection method based on drone of the present application; Figure 3 This is a flow chart of the third embodiment of the blade detection method based on drone of the present application; Figure 4 This is a schematic diagram of the structure of the blade detection device based on a drone in this application; Figure 5 This is a schematic diagram of the device structure of the hardware operating environment involved in the drone-based blade detection method in the embodiment of the present application.

[0019] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0020] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0021] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0022] At present, with the increase in China's cumulative installed capacity of wind power, the unit capacity and blade length of wind turbines are also increasing, which brings challenges to the inspection of wind turbines. As for wind turbine blades, the length of blades has increased from 20 to 30 meters to more than 100 meters now, and the tower height has also increased to more than 200 meters, which has greatly increased the difficulty and workload of inspection work. The current traditional manual inspection methods mainly rely on high-power telescope inspections, high-altitude detours and visual inspections (such as the Spider-Man method), blade maintenance platform inspections, etc. These methods are inefficient and high-risk, and it is difficult to achieve ideal inspection results in complex environments, which makes it even more difficult to meet the inspection task requirements of current inspection work.

[0023] The main solution of the embodiment of the present application is: obtaining the images of each local blade of the wind turbine by using a drone; For any one of the partial leaf images, detecting the partial leaf image by using a preset detection model to obtain a detection result of the partial leaf image; When the detection result indicates that corrosion exists, an alarm prompt is output, wherein the alarm prompt includes the detection result, the local blade image, and the wind turbine to which the local blade image belongs.

[0024] That is, in this embodiment, drones will be used to replace manual work to complete the shooting of blade images, thereby improving work efficiency and avoiding the problem of safety risks in manual shooting of blade images. At the same time, a pre-trained model is used to replace manual naked eye recognition of local blade images, thereby reducing labor costs and improving recognition accuracy. It can be understood that the present application realizes unmanned automatic inspection instead of manual work, which can improve inspection efficiency and reduce inspection risks, so that it is difficult to achieve ideal inspection results in complex environments, in order to meet the inspection task requirements of current inspection work.

[0025] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a drone, a cloud platform, an electronic device, a main controller in an electronic device, a computer, a mobile phone, etc., or an electronic device that can realize the above functions.

[0026] Based on this, the present application embodiment provides a blade detection method based on a drone, referring to Figure 1 , which is a flow chart of the first embodiment of the blade detection method based on drone of the present application.

[0027] In this embodiment, the blade detection method based on a drone includes steps S10 to S30: Step S10, acquiring images of local blades of the wind turbine using a drone; It should be noted that in this embodiment, the blades of the wind turbine generator set are mainly inspected. The above-mentioned blade inspection method based on drones can be applied to drones, and can also be applied to control devices connected to drones, such as computers or cloud servers.

[0028] For example, in this embodiment, a drone can be controlled to approach a wind turbine, which can be located at sea or on land. A camera on the drone is then used to capture partial images of wind turbine blades (usually, blades are long and it is difficult to capture the entire blade at one time, so partial images of the blades are captured), thereby obtaining partial blade images of the wind turbine.

[0029] It is understandable that drones can replace manual labor in taking blade images, thereby improving work efficiency and avoiding the safety risks of manual photography of blade images.

[0030] Step S20, for any one of the partial leaf images, detecting the partial leaf image by using a preset detection model to obtain a detection result of the partial leaf image; It should be noted that, in this embodiment, a pre-trained model is used to replace the manual naked eye to identify the local leaf image, thereby reducing labor costs and improving the accuracy of identification. Since the identification process of each local leaf image is basically the same, this embodiment will take one of them as an example for explanation.

[0031] Exemplarily, for any one of the local blade images, a preset detection model can be used to detect the local blade image. If the local blade image is input into the preset detection model, the preset detection model then outputs the detection result, and the detection result can be that the local image has corrosion, or does not have corrosion. In the case of corrosion, the detection result can further include the type of corrosion, which can include: environmental corrosion, such as wind turbines are usually installed outdoors, exposed to wind, sun and rain, especially in the ocean or coastal areas, the salt spray environment will accelerate the corrosion of metal parts, and in an environment with high humidity, a water film is easily formed on the metal surface, leading to electrochemical corrosion; stress corrosion: some parts of the wind turbine may undergo stress corrosion cracking in a specific corrosive environment under continuous stress; fatigue corrosion: due to the alternating loads of the wind turbine during operation, fatigue corrosion may occur, and other types of corrosion may also be included, which will not be described one by one here. In addition, the above-mentioned preset detection model has been trained in advance, for example, the training sample can be obtained by manually annotating the image, and then the model is trained using the training sample, so as to obtain the above-mentioned preset detection model. Accordingly, the content of the annotation can be whether there is corrosion, or the type of corrosion, etc.

[0032] Step S30, when the detection result indicates that corrosion exists, outputting an alarm prompt, wherein the alarm prompt includes the detection result, the local blade image, and the wind turbine to which the local blade image belongs.

[0033] For example, if the detection result shows that corrosion exists, a corresponding alarm prompt can be output, for example, the alarm prompt can be output through a terminal device corresponding to the inspection or operation and maintenance personnel. The alarm prompt can also be output through a control platform for centralized management of the wind turbine. Furthermore, the alarm prompt can include the detection result obtained by the above model, the image of the local blade with corrosion, and the wind turbine to which the local blade image belongs. This makes it easier for the corresponding personnel to maintain the corroded blades.

[0034] In this embodiment, each local blade image of the wind turbine is obtained by a drone; for any local blade image in each local blade image, the local blade image is detected by a preset detection model to obtain the detection result of the local blade image; when the detection result is that corrosion exists, an alarm prompt is output, wherein the alarm prompt includes the detection result, the local blade image, and the wind turbine to which the local blade image belongs. In the embodiment of the present application, a drone will be used to replace manual work to complete the shooting of blade images, thereby improving work efficiency and avoiding the problem of safety risks in manual shooting of blade images. At the same time, a pre-trained model is used to replace manual naked eye recognition of local blade images, thereby reducing labor costs and improving recognition accuracy. It can be understood that the present application realizes unmanned automatic inspection instead of manual inspection, which can improve inspection efficiency and reduce inspection risks, so that it is difficult to achieve an ideal inspection effect in a complex environment to meet the inspection task requirements of the current inspection work.

[0035] Reference Figure 2 , which is a second embodiment of the leaf detection method based on drone in the present application, proposed based on the above first embodiment. In this embodiment, the same or similar contents as the above embodiment can be referred to the above introduction, and will not be repeated later. The step of detecting the local leaf image by using a preset detection model to obtain the detection result of the local leaf image includes steps S21 to S22: Step S21, determining the shooting time of the partial blade image, the geographical location when the partial blade image was shot, the weather conditions when the partial blade image was shot, and the fan speed when the partial blade image was shot; Step S22, inputting the shooting time, the geographical location, the weather conditions, the fan speed and the local blade image into the preset detection model to obtain the detection result of the local blade image.

[0036] It should be noted that the current image recognition scheme usually uses a single image data as the input of the model. In the application scenario of this embodiment, the image obtained by shooting will be affected by the shooting environment, which may cause the model to affect the detection result of the image. Therefore, in order to improve the accuracy of the detection result, the local leaf image obtained by combining the shooting conditions when taking the photo is also input into the model, so that the model can obtain more dimensional information to ensure the accuracy of the model detection result.

[0037] Exemplarily, the shooting time of the local blade image, the geographical location when the local blade image is shot, the weather conditions when the local blade image is shot, the fan speed when the local blade image is shot, and the local blade image are input into the preset detection model, and the preset detection model then outputs the detection result of the local blade image. It can be understood that the most important factors affecting the shooting effect of the photo or image are the illumination of the blade and the motion state of the blade. Therefore, in this embodiment, the shooting time, geographical location, and weather conditions are added to characterize the illumination conditions, wherein the weather conditions may include the light brightness and weather types (such as fog, rain) collected by the drone through the sensor. It is worth noting that fog may cause occlusion, and rain will form small dewdrops on the surface of the blade. In addition, in this embodiment, photos can be taken when the fan is running, and the photos taken under operation may have afterimages, so the feature dimension of fan speed is added. When the drone shoots the local blade image, the fan speed at the time of shooting can be obtained through the fan operation and maintenance platform, and the fan speed is associated with the local blade image and saved. Accordingly, the content associated and saved at the same time also includes the shooting time, the geographical location at the time of shooting, and the weather conditions at the time of shooting. In order to facilitate the detection of the local leaf image, the various conditions of the local leaf image can be directly obtained through the association relationship. In a feasible implementation, before the step of detecting the local leaf image by using a preset detection model to obtain the detection result of the local leaf image, the method further includes steps S210 to S240: Step S210, collecting original images of the fan blades to obtain an original image set; Step S220, obtaining a sample image set after the original images in the original image set are marked; Step S230, performing derivative processing on the sample images in the sample image set to expand the sample image set to obtain an expanded sample image set; Step S240, iteratively training the initial classification model through the sample image expansion set, and using the initial classification model that meets the preset training conditions as the preset detection model.

[0038] It should be noted that, in detecting the local leaf image by using a preset detection model, this embodiment will train the initial model to obtain a preset detection model with image detection capability.

[0039] Exemplarily, the original image set can be obtained by collecting the original image of the wind turbine blade by a drone, wherein the process of collecting the original image of the wind turbine blade is similar to the above-mentioned process of obtaining the local blade images of the wind turbine by a drone, so it will not be repeated here. The collected original image set can be manually annotated, such as annotating whether it is corroded, the type of corrosion, etc. The marked original image is the sample image, so after the original image set is annotated, the sample image set can be obtained. In practical applications, the sample image set can also be directly used for model training. However, in an embodiment, the sample images in the sample image set are further derived to obtain new sample images, thereby enriching the sample types, expanding the number of samples, and obtaining the sample image expansion set. The sample image expansion set is then used to iteratively train the initial classification model. The initial classification model can be a neural network classification model. During the iterative training process, any sample image is input into the initial classification model, and the initial classification model outputs a classification result. Then, according to the difference between the classification result and the label of the sample image, the model parameters in the initial classification model are updated until the initial classification model reaches the preset training conditions, and then the initial classification model that reaches the preset training conditions is used as the preset detection model. The preset training condition may be the number of model training times or model convergence, etc., and no specific limitation is given here.

[0040] It is understandable that since the samples in the sample image expansion set are more abundant and diverse, using the sample image expansion set to train the model can improve the detection performance of the model and avoid overfitting of the model.

[0041] In a feasible implementation manner, the step of performing derivative processing on the sample images in the sample image set includes at least one of the following: Step S231, adding a dynamic blur effect to the sample images in the sample image set; Step S232, adding background noise to the sample images in the sample image set; Step S233: adding weather noise to the sample images in the sample image set.

[0042] Exemplarily, in this embodiment, a variety of derivative processing can be performed on the sample images in the sample image set. In practical applications, for any sample image, one or more derivative methods can be selected to process it. Among them, the types of derivative methods may include adding dynamic blur effects to the sample images in the sample image set. The dynamic blur effect is mainly aimed at the afterimage problem that may be formed when the drone shoots the moving leaves; adding background noise to the sample images in the sample image set. The background noise is mainly aimed at the problem that different backgrounds may appear behind the leaves when shooting the leaves, such as flying birds, trees, sky, white clouds, etc., which will not be described one by one here; adding weather noise to the sample images in the sample image set. The weather noise can be fog and rain, etc.

[0043] It is understandable that different derivative processing can greatly enrich the sample types, so that the preset detection model can cope with different natural interferences in the shooting environment and improve the robustness of the preset detection model.

[0044] Reference Figure 3 , which is the third embodiment of the blade detection method based on drone in the present application, based on the first and second embodiments mentioned above. In this embodiment, the same or similar contents as the above embodiments can be referred to the above introduction, and will not be repeated hereafter. The step of obtaining the local blade images of the wind turbine by drone includes step S11 to step S16: Step S11, collecting an overall image of the wind turbine by the drone; Step S12, identifying the hub component of the wind turbine from the overall image based on a target detection algorithm; Step S13, controlling the drone to move to an initial position set at a preset distance directly in front of the hub component; Step S14, calculating the moving step length of the drone based on the preset distance and the camera field of view of the drone; Step S15, controlling the moving step length of the drone to move in a preset direction, photographing the blades of the wind turbine to obtain a local blade image subset, wherein the preset direction is parallel to a blade plane, and the blade plane is a plane where each blade of the wind turbine is located; Step S16, returning to the step of controlling the UAV to move in a direction parallel to the wind turbine blades until a preset movement condition is reached, wherein the obtained subsets of local blade images are combined into the local blade images.

[0045] It should be noted that in the embodiment of the present application, a drone will be used to take photos of the wind turbine in motion, so a drone with strong wind resistance can be selected.

[0046] Exemplarily, a drone can be used to collect an overall image of a wind turbine. Then, a target detection algorithm is used to identify the hub component of the wind turbine from the overall image. Among them, the contour component may include a front contour component and a back hub component, or a windward surface or a non-windward surface of the hub component, and the processing process of the two sides is roughly the same, so in the embodiment of the present application, it will not be described separately. The drone is controlled to move to an initial position set at a preset distance directly in front of the hub component, wherein the preset distance can be set by a technician according to actual needs. Then, the moving step length of the drone is calculated according to the preset distance and the field of view of the unmanned camera, and the moving step length can be calculated by a mathematical geometry formula. For example, the proportional relationship (i.e., trigonometric function relationship) between the preset distance and the half-step length is calculated by the field of view, and then the half-step length is calculated by the proportional relationship and the known preset distance, and the moving step length can be obtained by multiplying the half-step length by two.

[0047] Accordingly, after determining the moving step length, the drone is controlled to move in a preset direction by the moving step length, and the blades of the wind turbine are photographed to obtain a subset of local blade images, such as taking a photo each time the blade passes. In addition, it should be noted that the preset direction must be parallel to the blade plane, and the blade plane refers to the plane where the blades of the wind turbine are located. After completing one shot, continue to return to the step of controlling the drone to move in a direction parallel to the wind turbine blades until the preset moving condition is reached, and the shooting is considered to be completed, and the subsets of local blade images obtained by the shooting are grouped into the above-mentioned local blade images. Among them, the preset condition can be the number of movements, and the number of movements can be the result of dividing the blade length by the moving step length; the preset condition can also be that the blade does not pass through the local blade image, that is, the tip of the blade is photographed.

[0048] In a feasible implementation manner, the step of photographing the blades of the wind turbine to obtain a local blade image subset includes steps S151 and S152: Step S151, calculating the shooting interval according to the rotation speed of the wind turbine, and determining the number of shooting times according to the number of blades; Step S152, determining the initial time point for photographing the leaves; Exemplarily, the rotation speed of the large wind turbine is obtained from the wind turbine operation and maintenance platform, and then the shooting interval is calculated according to the rotation speed, and the number of shots is determined according to the number of blades. Among them, the shooting interval can be obtained by dividing the interval angle between two adjacent blades by the rotation speed. In determining the initial time point for shooting the blades, the initial time point refers to the time point when the shooting of the local blade image is officially started. For example, the drone can be allowed to take multiple images continuously, and then the shooting time of an image with the largest area of ​​the wind turbine blades can be selected from the multiple images as the above-mentioned initial time point. After determining the initial time point, based on the initial time point, the drone is controlled to continuously shoot the blades according to the shooting interval until the number of shots determined in the above steps is reached. The local blade images captured this time can constitute a subset of local blade images.

[0049] In a feasible implementation manner, the step of determining the initial time point for photographing the leaf includes steps S1521 to S1524: Step S1521, controlling the drone to continuously photograph the blade to obtain candidate images; Step S1522, for any candidate image, identifying a leaf region in the candidate image, and determining a leaf center of the leaf region; Step S1523, determining the distribution position of the center of the blade in the candidate image, and calculating the blade deviation of the candidate image based on the position distance between the distribution position and the center position of the candidate image, wherein the blade deviation increases with the increase of the position distance; Step S1524, calculating the initial time point according to the shooting time point of the candidate image with the smallest blade deviation among the candidate images and the shooting interval.

[0050] Exemplarily, when determining the initial time point, the drone can be controlled to continuously shoot the blades to obtain each candidate image. Then the candidate images are screened. For example, for any candidate image, lines can be detected from the candidate image by an edge detection algorithm, and then the contour lines belonging to the wind turbine blades are screened according to prior knowledge (such as the contour of the blade should be smooth; according to the position of the drone shooting, the general direction of the blade contour can also be known; the blade is usually a white area). In this way, the blade area in the candidate image can be identified, and on this basis, the geometric center of the blade area, that is, the blade center, is identified. The distribution position of the blade center in the candidate image is identified, such as the distribution position can be the position of the pixel point corresponding to the blade center relative to the candidate image. Then, according to the position distance between the distribution position and the center position of the candidate image, the blade deviation of the candidate image is calculated. The blade deviation increases with the increase of the position distance, such as the blade deviation is proportional to the position distance. After traversing each candidate image, the candidate image with the smallest blade deviation is screened out. And the initial time point is calculated by the shooting time point and shooting interval of the candidate image with the smallest blade deviation. For example, the shooting time point may be directly used as the initial time point, or the shooting time point may be added to n times the shooting interval to obtain the initial time point, wherein n may be set by a technician and will not be described in detail here.

[0051] It can be understood that, in this embodiment, through the above control process, it can be ensured that images of various positions of the blades are captured when the fan is running.

[0052] Reference Figure 4 , is a schematic diagram of the structure of a blade detection device based on a drone provided in the present application, and the blade detection device based on a drone includes: An acquisition module 10 acquires images of local blades of a wind turbine through a drone; A detection module 20 detects any one of the partial leaf images by using a preset detection model to obtain a detection result of the partial leaf image; The output module 30 outputs an alarm prompt when the detection result indicates that corrosion exists, wherein the alarm prompt includes the detection result, the local blade image, and the wind turbine to which the local blade image belongs.

[0053] Optionally, the detection module 20 is further used for: Determine the shooting time of the partial blade image, the geographical location when the partial blade image is shot, the weather conditions when the partial blade image is shot, and the fan speed when the partial blade image is shot; The shooting time, the geographical location, the weather conditions, the fan speed and the local blade image are input into the preset detection model to obtain the detection result of the local blade image.

[0054] Optionally, the detection module 20 is further used for: Collecting original images of fan blades to obtain an original image set; After the original images in the original image set are marked, a sample image set is obtained; Performing derivative processing on sample images in the sample image set to expand the sample image set to obtain an expanded sample image set; The initial classification model is iteratively trained through the sample image expansion set, and the initial classification model that meets the preset training conditions is used as the preset detection model.

[0055] Optionally, the detection module 20 is further used for: Adding a dynamic blur effect to the sample images in the sample image set; adding background noise to the sample images in the sample image set; Weather noise is added to the sample images in the sample image set.

[0056] Optionally, the acquisition module 10 is further used for: Collecting an overall image of the wind turbine by using the drone; Identifying a hub component of the wind turbine from the overall image based on a target detection algorithm; Controlling the UAV to move to an initial position set at a preset distance directly in front of the hub component; Calculating the moving step length of the drone based on the preset distance and the camera field of view of the drone; Controlling the moving step length of the drone to move in a preset direction, photographing the blades of the wind turbine to obtain a local blade image subset, wherein the preset direction is parallel to a blade plane, and the blade plane is a plane where each blade of the wind turbine is located; Return to the step of controlling the UAV to move in a direction parallel to the wind turbine blades until a preset movement condition is reached, wherein the obtained subsets of local blade images are combined into the local blade images.

[0057] Optionally, the acquisition module 10 is further used for: Calculate the shooting interval according to the rotation speed of the wind turbine, and determine the number of shots according to the number of blades; Determine the initial time point for photographing leaves; From the initial time point, the drone is controlled to photograph the leaves according to the photographing interval until the number of photographing times is reached to obtain the local leaf image subset.

[0058] Optionally, the acquisition module 10 is further used for: Controlling the UAV to continuously photograph the blades to obtain candidate images; For any candidate image, identifying a leaf region in the candidate image, and determining a leaf center of the leaf region; Determine the distribution position of the center of the blade in the candidate image, and calculate the blade deviation of the candidate image based on the position distance between the distribution position and the center position of the candidate image, wherein the blade deviation increases with the increase of the position distance; The initial time point is calculated according to the shooting time point of the candidate image with the smallest blade deviation among the candidate images and the shooting interval.

[0059] The drone-based blade detection device provided in this application adopts the drone-based blade detection method in the above embodiment, aiming to solve the technical problem that the traditional inspection method is difficult to meet the task requirements of the current wind turbine inspection work. Compared with the prior art, the beneficial effects of the drone-based blade detection device provided in the embodiment of this application are the same as the beneficial effects of the drone-based blade detection method provided in the above embodiment 1, and the other technical features of the drone-based blade detection device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0060] The present application provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the drone-based blade detection method in the above-mentioned embodiment one.

[0061] Reference below Figure 5 , which shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiment of the present application. The electronic device in the embodiment of the present application may include but is not limited to mobile terminals such as computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and fixed terminals such as computers. Figure 5The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0062] like Figure 5 As shown, the electronic device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the electronic device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows an electronic device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0063] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0064] The electronic device provided by the present application adopts the blade detection method based on drone in the above embodiment, which can solve the technical problem that the traditional inspection method is difficult to meet the task requirements of the current wind turbine inspection work. Compared with the prior art, the beneficial effects of the electronic device provided by the present application are the same as the beneficial effects of the blade detection method based on drone provided in the above embodiment, and the other technical features in the electronic device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0065] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0066] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0067] The present application provides a computer-readable storage medium having computer-readable program instructions (i.e., computer programs) stored thereon, and the computer-readable program instructions are used to execute the drone-based blade detection method in the above-mentioned embodiment.

[0068] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0069] The computer-readable storage medium may be included in the electronic device, or may exist independently without being installed in the electronic device.

[0070] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by an electronic device, the electronic device: Use drones to obtain images of each local blade of a wind turbine; For any one of the partial leaf images, detecting the partial leaf image by using a preset detection model to obtain a detection result of the partial leaf image; When the detection result indicates that corrosion exists, an alarm prompt is output, wherein the alarm prompt includes the detection result, the local blade image, and the wind turbine to which the local blade image belongs.

[0071] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0072] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0073] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0074] The readable storage medium provided in this application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned drone-based blade detection method, and can solve the technical problem that the traditional inspection method is difficult to meet the task requirements of the current wind turbine inspection work. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the drone-based blade detection method provided in the above-mentioned embodiment, and will not be repeated here.

[0075] The present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the above-mentioned drone-based blade detection method.

[0076] The computer program product provided in this application can solve the technical problem of blade detection based on drones. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as the beneficial effects of the blade detection method based on drones provided in the above embodiments, which will not be repeated here.

[0077] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A blade detection method based on drone, characterized in that: The blade detection method based on drone includes the following steps: Use drones to obtain images of each local blade of a wind turbine; For any one of the partial leaf images, detecting the partial leaf image by using a preset detection model to obtain a detection result of the partial leaf image; When the detection result indicates that corrosion exists, an alarm prompt is output, wherein the alarm prompt includes the detection result, the local blade image, and the wind turbine to which the local blade image belongs.

2. The blade detection method based on a drone as claimed in claim 1, characterized in that: The step of detecting the local leaf image by using a preset detection model to obtain the detection result of the local leaf image comprises: Determining the shooting time of the partial blade image, the geographical location when the partial blade image was shot, the weather conditions when the partial blade image was shot, and the fan speed when the partial blade image was shot; The shooting time, the geographical location, the weather conditions, the fan speed and the local blade image are input into the preset detection model to obtain the detection result of the local blade image.

3. The blade detection method based on a drone as claimed in claim 1, characterized in that: Before the step of detecting the local leaf image by using a preset detection model to obtain a detection result of the local leaf image, the method further includes: Collecting original images of fan blades to obtain an original image set; After the original images in the original image set are marked, a sample image set is obtained; Performing derivative processing on sample images in the sample image set to expand the sample image set to obtain an expanded sample image set; The initial classification model is iteratively trained through the sample image expansion set, and the initial classification model that meets the preset training conditions is used as the preset detection model.

4. The blade detection method based on a drone as claimed in claim 3, characterized in that: The step of performing derivative processing on the sample images in the sample image set comprises at least one of the following: Adding a dynamic blur effect to the sample images in the sample image set; adding background noise to the sample images in the sample image set; Weather noise is added to the sample images in the sample image set.

5. The blade detection method based on a drone as claimed in claim 1, characterized in that: The step of obtaining the images of each local blade of the wind turbine by using the drone comprises: Collecting an overall image of the wind turbine by using the drone; Identifying a hub component of the wind turbine from the overall image based on a target detection algorithm; Controlling the UAV to move to an initial position set at a preset distance directly in front of the hub component; Calculating the moving step length of the drone based on the preset distance and the camera field of view of the drone; Controlling the moving step length of the drone to move in a preset direction, photographing the blades of the wind turbine to obtain a local blade image subset, wherein the preset direction is parallel to a blade plane, and the blade plane is a plane where each blade of the wind turbine is located; Return to the step of controlling the UAV to move in a direction parallel to the wind turbine blades until a preset movement condition is reached, wherein the obtained subsets of local blade images are combined into the local blade images.

6. The blade detection method based on a drone as claimed in claim 5, characterized in that: The step of photographing the blades of the wind turbine to obtain a local blade image subset comprises: Calculate the shooting interval according to the rotation speed of the wind turbine, and determine the number of shots according to the number of blades; Determine the initial time point for photographing leaves; From the initial time point, the drone is controlled to photograph the leaves according to the photographing interval until the photographing times are reached to obtain the local leaf image subset.

7. The blade detection method based on a drone as claimed in claim 6, characterized in that: The step of determining the initial time point for photographing the leaf comprises: Controlling the UAV to continuously photograph the blades to obtain candidate images; For any candidate image, identifying a leaf region in the candidate image, and determining a leaf center of the leaf region; Determine the distribution position of the center of the blade in the candidate image, and calculate the blade deviation of the candidate image based on the position distance between the distribution position and the center position of the candidate image, wherein the blade deviation increases with the increase of the position distance; The initial time point is calculated according to the shooting time point of the candidate image with the smallest blade deviation among the candidate images and the shooting interval.

8. A blade detection device based on a drone, characterized in that: The blade detection device based on drone includes: An acquisition module, used for acquiring images of each local blade of a wind turbine through a drone; A detection module, configured to detect any one of the partial leaf images by using a preset detection model to obtain a detection result of the partial leaf image; An alarm module is used to output an alarm prompt when the detection result shows that corrosion exists, wherein the alarm prompt includes the detection result, the local blade image and the wind turbine to which the local blade image belongs.

9. An electronic device, characterized in that: The electronic device includes: a processor, a memory, and a drone-based blade detection program stored in the memory and executable on the processor. When the drone-based blade detection program is executed, the steps of the drone-based blade detection method as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a drone-based blade detection program, and when the drone-based blade detection program is executed, the steps of the drone-based blade detection method according to any one of claims 1 to 7 are implemented.

Images