Fan non-stop routing inspection path planning method and system

Abstract

The invention relates to unmanned aerial vehicle routing inspection, in particular to a fan non-stop routing inspection path planning method and system, and the method comprises the steps: controlling an unmanned aerial vehicle to fly to a preset height, and photographing a to-be-detected fan by using an image collection device to obtain an original image; performing fan structure feature recognition on the original image by using a pre-trained fan structure recognition model to obtain a first target image including a blade tip area and a second target image including a blade area; in the first target image, determining coordinates of a blade tip; in the second target image, calculating the direction range of the paddle blade; according to the photographing position of the unmanned aerial vehicle, the coordinates of the blade tip and the direction range of the blade, determining an inspection point corresponding to the to-be-detected fan; after inspection points corresponding to all the to-be-detected fans are obtained, planning an inspection path of the unmanned aerial vehicle; according to the technical scheme provided by the invention, the defect that unmanned aerial vehicle automatic routing inspection is difficult to carry out on the fan blades under the condition that the fan is not stopped can be effectively overcome.

Description

Technical Field

[0001] The present invention relates to unmanned aerial vehicle inspection, and in particular to a method and system for planning an inspection path without stopping a wind turbine. Background Art

[0002] The design life of the blades in a wind turbine (wind turbine for short) is generally 20 years. Once the blades enter the middle stage of operation, under the repeated fatigue load, the manufacturing defects remaining in the blade manufacturing will gradually appear. Many blades begin to have problems such as local cracking and delamination, and some serious ones will break. Every blade breakage will cause great losses to the wind farm. Most of the reasons for blade breakage are that the manufacturing defects have not been discovered and repaired in time. Under fatigue operation, the cracks continue to expand, and eventually cause the blade to become unstable and break.

[0003] With the development of drone technology, drone inspections have gradually become an effective means for energy companies to conduct daily inspections and data collection. Drone inspections have opened up new ways for wind turbine blade inspections, which can improve the quality and efficiency of inspections and greatly improve the level of intelligence in wind farm operation and management. In the existing drone inspection methods for wind turbine blades, the inspection path needs to be replanned for each wind turbine inspection, which affects the inspection efficiency, and drone inspections cannot be carried out without shutting down the wind turbines, which has a certain degree of impact on the normal operation of wind farms. Summary of the invention

[0004] 1. Technical issues to be resolved

[0005] In view of the above-mentioned shortcomings of the prior art, the present invention provides a method and system for planning an inspection path for a wind turbine without stopping the wind turbine, which can effectively overcome the defect of the prior art that it is difficult to perform automated drone inspection on wind turbine blades without stopping the wind turbine.

[0006] (II) Technical solution

[0007] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0008] A method for planning a wind turbine inspection path without shutting down the wind turbine comprises the following steps:

[0009] S1. Control the UAV to fly to a predetermined height, and use an image acquisition device to take a picture of the wind turbine to be tested to obtain an original image;

[0010] S2. Using a pre-trained fan structure recognition model to perform fan structure feature recognition on the original image, and obtaining a first target image including a blade tip area and a second target image including a blade blade area;

[0011] S3. Determine the coordinates of the blade tip in the first target image;

[0012] S4, in the second target image, calculating the direction range of the blade;

[0013] S5. Determine the inspection point corresponding to the wind turbine to be tested according to the position of the drone taking the photo, the coordinates of the blade tip, and the direction range of the blade;

[0014] S6. Repeat S1 to S5 until all inspection points corresponding to the wind turbines to be tested are obtained, and the inspection path of the UAV is planned according to these inspection points.

[0015] Preferably, in S2, a pre-trained fan structure recognition model is used to perform fan structure feature recognition on the original image to obtain a first target image including a blade tip area and a second target image including a blade blade area, including:

[0016] Training a fan structure recognition model, and inputting the preprocessed original image into the pre-trained fan structure recognition model;

[0017] The wind turbine structure recognition model recognizes the wind turbine structure features of the original image by extracting contour features, and obtains a first target image including the blade tip area and a second target image including the blade blade area.

[0018] Preferably, the training of the wind turbine structure identification model includes:

[0019] S21, dividing the historical data set into a training set, a validation set, and a test set according to a preset ratio;

[0020] S22, setting a loss function and an optimizer of a wind turbine structure identification model;

[0021] S23, inputting the training set into the fan structure identification model for model training;

[0022] S24, calculating the loss value based on the loss function, and the optimizer updates the model parameters according to the loss value and the network gradient information;

[0023] S25. If the loss value is less than the preset threshold, the model training ends, and the current wind turbine structure identification model is the trained wind turbine structure identification model. Otherwise, the process returns to S23 and continues to use the training set for model training.

[0024] S26. Input the validation set into the trained wind turbine structure recognition model, evaluate the generalization ability of the model by observing the performance on the validation set, and tune the hyperparameters and structure of the model;

[0025] S27, inputting the test set into the optimized wind turbine structure identification model to evaluate the model performance;

[0026] Among them, the fan structure recognition model is built based on the convolutional neural network CNN.

[0027] Preferably, in S21, the historical data set is divided into a training set, a validation set and a test set according to a preset ratio, including:

[0028] S211, collecting historical wind turbine images, distinguishing and marking the outlines of blade tips and blade blades in each historical wind turbine image, and setting blade tip labels and blade blade labels for each historical wind turbine image to form a historical data set;

[0029] S212. Divide the historical data set into a training set, a validation set, and a test set according to a preset ratio.

[0030] Preferably, in S3, determining the coordinates of the blade tip in the first target image includes:

[0031] S31, establishing a pixel coordinate system in the first target image, and obtaining the coordinates of the blade tip in the pixel coordinate system;

[0032] S32, performing coordinate conversion on the coordinates of the blade tip in the pixel coordinate system, acquiring the coordinates of the blade tip in the geographic coordinate system, and obtaining the coordinates of the blade tip.

[0033] Preferably, in S32, coordinate conversion is performed on the coordinates of the blade tip in the pixel coordinate system to obtain the coordinates of the blade tip in the geographic coordinate system, and the coordinates of the blade tip are obtained, including:

[0034] According to the attitude angle of the drone body and the attitude angle of the image acquisition device when the drone takes pictures, the coordinates of the blade tip in the pixel coordinate system are transformed to obtain the coordinates of the blade tip in the geographic coordinate system, thereby obtaining the coordinates of the blade tip.

[0035] Preferably, in S4, calculating the direction range of the blade in the second target image includes:

[0036] S41, determining a location region of the second target image in the original image, and finding two edge pixel points in the location region;

[0037] S42, calculating two deviation lengths in the space according to the distances between the two edge pixels and the central pixel of the original image respectively;

[0038] S43. Calculate the direction range of the blade according to the two deviation lengths.

[0039] Preferably, in S42, two deviation lengths in the space are calculated according to the distances between the two edge pixel points and the central pixel point of the original image, including:

[0040] The deviation length is calculated using the following formula:

[0041] d i '=d i ·sigmoid(ω·d i + b);

[0042] Among them, d i is the distance between the edge pixel i and the center pixel of the original image, ω is the spatial weight, b is the spatial bias, sigmoid represents the sigmoid function, d i ' is the deviation length between the edge pixel i and the center pixel of the original image.

[0043] Preferably, in S43, calculating the direction range of the blade according to the two deviation lengths includes:

[0044] The minimum directional β of the blade is calculated using the following formula: min :

[0045]

[0046] Among them, d c ' is the deviation length between the edge pixel closest to the center pixel of the original image and the center pixel of the original image, d n is the normalization parameter, ω1 is the first direction weight;

[0047] The maximum direction β of the blade is calculated using the following formula: max :

[0048]

[0049] Among them, d f ' is the deviation length between the edge pixel point farthest from the central pixel point of the original image and the central pixel point of the original image, and ω2 is the second direction weight.

[0050] A wind turbine non-stop inspection path planning system, comprising a drone control module, a target image acquisition module, a blade tip coordinate determination module, a blade direction determination module, an inspection point determination module and an inspection path planning module;

[0051] The UAV control module controls the UAV to fly to a predetermined height, drives the image acquisition device to take pictures of the wind turbine to be tested to obtain the original image, and controls the UAV to perform the inspection task according to the inspection path sent by the inspection path planning module;

[0052] A target image acquisition module uses a pre-trained fan structure recognition model to perform fan structure feature recognition on the original image to obtain a first target image including a blade tip area and a second target image including a blade blade area;

[0053] A blade tip coordinate determination module, which determines the coordinates of the blade tip in the first target image;

[0054] a propeller blade direction determination module, which calculates the direction range of the propeller blade in the second target image;

[0055] The inspection point determination module determines the inspection point corresponding to the wind turbine to be tested according to the position of the drone photo, the coordinates of the blade tip and the direction range of the blade.

[0056] The inspection path planning module plans the inspection path of the UAV according to the inspection points corresponding to all wind turbines to be tested, and sends the inspection path to the UAV control module.

[0057] (III) Beneficial effects

[0058] Compared with the prior art, the present invention provides a method and system for planning an inspection path for a wind turbine without stopping. The method controls a drone to fly to a predetermined height to collect an original image of the wind turbine to be tested, and uses a pre-trained wind turbine structure recognition model to identify the structural features of the wind turbine on the original image to obtain a first target image including a blade tip area and a second target image including a blade blade area, and determines the coordinates of the blade tip and calculates the direction range of the blade blade. Then, according to the drone's photo taking position, the coordinates of the blade tip and the direction range of the blade blade, the inspection point corresponding to the wind turbine to be tested is determined. Finally, the drone's inspection path is planned according to the inspection points corresponding to all the wind turbines to be tested, so that the drone can perform automatic inspections on the wind turbine blades according to the pre-planned inspection path without stopping the wind turbine, thereby effectively improving the inspection efficiency. At the same time, it can maintain the continuity of wind farm operation, reduce power grid fluctuations, and thus improve the stability of power grid operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0060] Figure 1 It is a schematic diagram of the process of the present invention;

[0061] Figure 2 It is a detailed schematic diagram of the process of determining the inspection points corresponding to the wind turbine to be tested in the present invention. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0063] A method for planning inspection paths without stopping a wind turbine, such as Figure 1 and Figure 2 As shown, S1, controlling the UAV to fly to a predetermined height, and using an image acquisition device to take a picture of the wind turbine to be tested to obtain an original image.

[0064] S2, using the pre-trained fan structure recognition model to perform fan structure feature recognition on the original image, and obtaining a first target image including a blade tip area, and a second target image including a blade blade area, specifically including:

[0065] Training a fan structure recognition model, and inputting the preprocessed original image into the pre-trained fan structure recognition model;

[0066] The wind turbine structure recognition model recognizes the wind turbine structure features of the original image by extracting contour features, and obtains a first target image including the blade tip area and a second target image including the blade blade area.

[0067] Specifically, the wind turbine structure identification model is trained, including:

[0068] S21, dividing the historical data set into a training set, a validation set, and a test set according to a preset ratio;

[0069] S22, setting a loss function and an optimizer of a wind turbine structure identification model;

[0070] S23, inputting the training set into the fan structure identification model for model training;

[0071] S24, calculating the loss value based on the loss function, and the optimizer updates the model parameters according to the loss value and the network gradient information;

[0072] S25. If the loss value is less than the preset threshold, the model training ends, and the current wind turbine structure identification model is the trained wind turbine structure identification model. Otherwise, the process returns to S23 and continues to use the training set for model training.

[0073] S26. Input the validation set into the trained wind turbine structure recognition model, evaluate the generalization ability of the model by observing the performance on the validation set, and tune the hyperparameters and structure of the model;

[0074] S27, inputting the test set into the optimized wind turbine structure identification model to evaluate the model performance;

[0075] Among them, the fan structure recognition model is built based on the convolutional neural network CNN.

[0076] Specifically, in S21, the historical data set is divided into a training set, a validation set, and a test set according to a preset ratio, including:

[0077] S211, collecting historical wind turbine images, distinguishing and marking the outlines of blade tips and blade blades in each historical wind turbine image, and setting blade tip labels and blade blade labels for each historical wind turbine image to form a historical data set;

[0078] S212. Divide the historical data set into a training set, a validation set, and a test set according to a preset ratio.

[0079] S3. In the first target image, determining the coordinates of the blade tip, specifically comprising:

[0080] S31, establishing a pixel coordinate system in the first target image, and obtaining the coordinates of the blade tip in the pixel coordinate system;

[0081] S32, performing coordinate conversion on the coordinates of the blade tip in the pixel coordinate system, acquiring the coordinates of the blade tip in the geographic coordinate system, and obtaining the coordinates of the blade tip.

[0082] Specifically, in S32, coordinate conversion is performed on the coordinates of the blade tip in the pixel coordinate system to obtain the coordinates of the blade tip in the geographic coordinate system, and the coordinates of the blade tip are obtained, including:

[0083] According to the attitude angle of the drone body and the attitude angle of the image acquisition device when the drone takes pictures, the coordinates of the blade tip in the pixel coordinate system are transformed to obtain the coordinates of the blade tip in the geographic coordinate system, thereby obtaining the coordinates of the blade tip.

[0084] S4. In the second target image, calculating the direction range of the blade, specifically including:

[0085] S41, determining a location region of the second target image in the original image, and finding two edge pixel points in the location region;

[0086] S42, calculating two deviation lengths in the space according to the distances between the two edge pixels and the central pixel of the original image respectively;

[0087] S43. Calculate the direction range of the blade according to the two deviation lengths.

[0088] Specifically, in S42, two deviation lengths in the space are calculated according to the distances between the two edge pixels and the central pixel of the original image, including:

[0089] The deviation length is calculated using the following formula:

[0090] d i '=d i ·sigmoid(ω·d i + b);

[0091] Among them, d i is the distance between the edge pixel i and the center pixel of the original image, ω is the spatial weight, b is the spatial bias, sigmoid represents the sigmoid function, d i ' is the deviation length between the edge pixel i and the center pixel of the original image.

[0092] Specifically, in S43, the direction range of the blade is calculated according to the two deviation lengths, including:

[0093] The minimum directional β of the blade is calculated using the following formula: min :

[0094]

[0095] Among them, d c ' is the deviation length between the edge pixel closest to the center pixel of the original image and the center pixel of the original image, d n is the normalization parameter, ω1 is the first direction weight;

[0096] The maximum direction β of the blade is calculated using the following formula: max :

[0097]

[0098] Among them, d f ' is the deviation length between the edge pixel point farthest from the central pixel point of the original image and the central pixel point of the original image, and ω2 is the second direction weight.

[0099] S5. Determine the inspection point corresponding to the wind turbine to be tested according to the position of the drone taking the photo, the coordinates of the blade tip and the direction range of the blade.

[0100] S6. Repeat S1 to S5 until all inspection points corresponding to the wind turbines to be tested are obtained, and the inspection path of the UAV is planned according to these inspection points.

[0101] In the technical solution of the present application, a drone is controlled to fly to a predetermined height to collect an original image of the wind turbine to be tested, and a pre-trained wind turbine structure recognition model is used to identify the structural features of the wind turbine on the original image to obtain a first target image including a blade tip area and a second target image including a blade blade area, and the coordinates of the blade tip are determined, and the directional range of the blade blade is calculated. Then, according to the drone's photo taking position, the coordinates of the blade tip and the directional range of the blade blade, the inspection point corresponding to the wind turbine to be tested is determined, and finally, the drone's inspection path is planned according to the inspection points corresponding to all the wind turbines to be tested, so that the drone can perform automatic inspections on the wind turbine blades according to the pre-planned inspection path without stopping the wind turbine, thereby effectively improving the inspection efficiency. At the same time, it can maintain the continuity of wind farm operation, reduce power grid fluctuations, and thus improve the stability of power grid operation.

[0102] On the basis of the above-mentioned method for planning a non-stop inspection path for a wind turbine, the technical solution of the present application further discloses a non-stop inspection path planning system for a wind turbine, including a drone control module, a target image acquisition module, a blade tip coordinate determination module, a blade direction determination module, an inspection point determination module, and an inspection path planning module;

[0103] The UAV control module controls the UAV to fly to a predetermined height, drives the image acquisition device to take pictures of the wind turbine to be tested to obtain the original image, and controls the UAV to perform the inspection task according to the inspection path sent by the inspection path planning module;

[0104] A target image acquisition module uses a pre-trained fan structure recognition model to perform fan structure feature recognition on the original image to obtain a first target image including a blade tip area and a second target image including a blade blade area;

[0105] A blade tip coordinate determination module, which determines the coordinates of the blade tip in the first target image;

[0106] a propeller blade direction determination module, which calculates the direction range of the propeller blade in the second target image;

[0107] The inspection point determination module determines the inspection point corresponding to the wind turbine to be tested according to the position of the drone photo, the coordinates of the blade tip and the direction range of the blade.

[0108] The inspection path planning module plans the inspection path of the UAV according to the inspection points corresponding to all wind turbines to be tested, and sends the inspection path to the UAV control module.

[0109] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements will not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for planning inspection paths without stopping a wind turbine, characterized in that: The following steps are involved: S1. Control the UAV to fly to a predetermined height, and use an image acquisition device to take a picture of the wind turbine to be tested to obtain an original image; S2. Using a pre-trained fan structure recognition model to perform fan structure feature recognition on the original image, and obtaining a first target image including a blade tip area and a second target image including a blade blade area; S3. Determine the coordinates of the blade tip in the first target image; S4, in the second target image, calculating the direction range of the blade; S5. Determine the inspection point corresponding to the wind turbine to be tested according to the position of the drone taking the photo, the coordinates of the blade tip, and the direction range of the blade; S6. Repeat S1 to S5 until all inspection points corresponding to the wind turbines to be tested are obtained, and the inspection path of the UAV is planned according to these inspection points.

2. The inspection path planning method for a wind turbine without shutting down according to claim 1 is characterized in that: In S2, the pre-trained fan structure recognition model is used to recognize the fan structure features of the original image, and a first target image including the blade tip area and a second target image including the blade blade area are obtained, including: Training a fan structure recognition model, and inputting the preprocessed original image into the pre-trained fan structure recognition model; The wind turbine structure recognition model recognizes the wind turbine structure features of the original image by extracting contour features, and obtains a first target image including the blade tip area and a second target image including the blade blade area.

3. The inspection path planning method for a wind turbine without stopping according to claim 2 is characterized in that: The training of the wind turbine structure identification model includes: S21, dividing the historical data set into a training set, a validation set, and a test set according to a preset ratio; S22, setting a loss function and an optimizer of a wind turbine structure identification model; S23, inputting the training set into the fan structure recognition model for model training; S24, calculating the loss value based on the loss function, and the optimizer updates the model parameters according to the loss value and the network gradient information; S25. If the loss value is less than the preset threshold, the model training ends, and the current wind turbine structure identification model is the trained wind turbine structure identification model. Otherwise, the process returns to S23 and continues to use the training set for model training. S26. Input the validation set into the trained wind turbine structure recognition model, evaluate the generalization ability of the model by observing the performance on the validation set, and tune the hyperparameters and structure of the model; S27, inputting the test set into the optimized wind turbine structure identification model to evaluate the model performance; Among them, the fan structure recognition model is built based on the convolutional neural network CNN.

4. The inspection path planning method for a wind turbine without shutting down according to claim 3 is characterized in that: In S21, the historical data set is divided into a training set, a validation set, and a test set according to a preset ratio, including: S211, collecting historical wind turbine images, distinguishing and marking the outlines of blade tips and blade blades in each historical wind turbine image, and setting blade tip labels and blade blade labels for each historical wind turbine image to form a historical data set; S212. Divide the historical data set into a training set, a validation set, and a test set according to a preset ratio.

5. The inspection path planning method for a wind turbine without shutting down according to claim 2 is characterized in that: In S3, in the first target image, the coordinates of the blade tip are determined, including: S31, establishing a pixel coordinate system in the first target image, and obtaining the coordinates of the blade tip in the pixel coordinate system; S32, performing coordinate conversion on the coordinates of the blade tip in the pixel coordinate system, acquiring the coordinates of the blade tip in the geographic coordinate system, and obtaining the coordinates of the blade tip.

6. The inspection path planning method for a wind turbine without shutting down according to claim 5 is characterized in that: In S32, coordinate conversion is performed on the coordinates of the blade tip in the pixel coordinate system to obtain the coordinates of the blade tip in the geographic coordinate system, and the coordinates of the blade tip are obtained, including: According to the attitude angle of the drone body and the attitude angle of the image acquisition device when the drone takes pictures, the coordinates of the blade tip in the pixel coordinate system are transformed to obtain the coordinates of the blade tip in the geographic coordinate system, thereby obtaining the coordinates of the blade tip.

7. The inspection path planning method for a wind turbine without shutting down according to claim 5 is characterized in that: In S4, in the second target image, the direction range of the blade is calculated, including: S41, determining a location region of the second target image in the original image, and finding two edge pixel points in the location region; S42, calculating two deviation lengths in the space according to the distances between the two edge pixels and the central pixel of the original image respectively; S43. Calculate the direction range of the blade according to the two deviation lengths.

8. The inspection path planning method for a wind turbine without shutting down according to claim 7 is characterized in that: In S42, two deviation lengths in the space are calculated according to the distances between the two edge pixels and the central pixel of the original image, including: The deviation length is calculated using the following formula: d i '=d i ·sigmoid(ω·d i +b); Among them, d i is the distance between the edge pixel i and the center pixel of the original image, ω is the spatial weight, b is the spatial bias, sigmoid represents the sigmoid function, d i ' is the deviation length between the edge pixel i and the center pixel of the original image.

9. The inspection path planning method for a wind turbine without shutting down according to claim 8 is characterized in that: In S43, the direction range of the blade is calculated based on the two deviation lengths, including: The minimum directional β of the blade is calculated using the following formula: min : Among them, d c ' is the deviation length between the edge pixel closest to the center pixel of the original image and the center pixel of the original image, d n is the normalization parameter, ω1 is the first direction weight; The maximum direction β of the blade is calculated using the following formula: max : Among them, d f ' is the deviation length between the edge pixel point farthest from the central pixel point of the original image and the central pixel point of the original image, and ω2 is the second direction weight.

10. A wind turbine non-stop inspection path planning system, used to execute the wind turbine non-stop inspection path planning method according to claim 1, characterized in that: It includes a UAV control module, a target image acquisition module, a blade tip coordinate determination module, a blade direction determination module, an inspection point determination module and an inspection path planning module; The UAV control module controls the UAV to fly to a predetermined height, drives the image acquisition device to take pictures of the wind turbine to be tested to obtain the original image, and controls the UAV to perform the inspection task according to the inspection path sent by the inspection path planning module; A target image acquisition module uses a pre-trained fan structure recognition model to perform fan structure feature recognition on the original image to obtain a first target image including a blade tip area and a second target image including a blade blade area; A blade tip coordinate determination module, which determines the coordinates of the blade tip in the first target image; a propeller blade direction determination module, which calculates the direction range of the propeller blade in the second target image; The inspection point determination module determines the inspection point corresponding to the wind turbine to be tested according to the position of the drone photo, the coordinates of the blade tip and the direction range of the blade. The inspection path planning module plans the inspection path of the UAV according to the inspection points corresponding to all wind turbines to be tested, and sends the inspection path to the UAV control module.

Images