Unmanned aerial vehicle autonomous inspection method and system for fan

Abstract

The invention discloses a fan unmanned aerial vehicle autonomous inspection method and system, and the method comprises the steps: building a fan three-dimensional model based on the collection data of a laser radar for a fan, and enabling the fan three-dimensional model to be constructed through a recurrent neural network; planning an inspection route of the unmanned aerial vehicle; setting an intelligent inspection APP, and determining an autonomous flight path of the unmanned aerial vehicle; automatically identifying the to-be-detected image through the fan defect detection module, determining the fan defect type and positioning the fan defect position; and counting the defect positions of the fan and outputting an analysis inspection report to finish inspection. According to the routing inspection method, the laser radar, the edge calculation module and the like are integrated, the hardware requirement of fan routing inspection is met, and unmanned routing inspection work of the wind driven generator is greatly facilitated.

Description

Technical Field

[0001] The present invention relates to the technical field of wind power generation inspection, and in particular to a wind turbine unmanned aerial vehicle autonomous inspection method and system. Background Art

[0002] With the continuous development of the new energy industry, the importance of daily inspections of wind farms is increasing. Accurately and timely judging the health status of wind turbines and ensuring efficient, stable and reliable operation of wind turbines have become the most important factors for wind power companies to have market competitiveness. Continuously reducing maintenance costs and improving the reliability of units have become urgent issues that wind power companies, R&D and manufacturing units need to solve. Among them, blades are key components for wind turbines to obtain wind energy and convert mechanical energy. During operation, they not only bear bending stress, centrifugal force, thermal stress, medium stress, aerodynamic force, etc., but also suffer from erosion by ice, frost, rain and snow and natural damage by lightning. The incoming flow and input force of wind turbine blades at high altitudes are extremely complex. When they are continuously running under rapidly changing heavy loads, it is very easy to cause cracking, deformation, shedding and fracture of the main beam, surface skin and bonding materials. The manufacturing cost of wind turbine blades accounts for 15% to 20% of the total cost. Under the long-term combined forces of the external environment and gusts, turbulence, starting, braking, etc., cracks will appear and even completely break. According to the survey, wind turbine blade accidents account for about 30% of wind turbine accidents. Sudden fracture and failure due to fatigue cracks occur from time to time. In some newly built wind farms, more than 70% of wind turbines have problems such as blade cracks. Blade damage has become a common safety hazard in wind farms. Accurately monitoring the health status of blades and quickly determining the degree of damage are of great significance to ensure the safe and efficient operation of wind turbines, extend their service life, and reduce maintenance costs.

[0003] Fatigue damage of wind turbine blades begins with the growth of very small cracks. When the cracks extend locally to the junction of the adhesive layer and the blade skin, they eventually lead to cracking of the blades and fracture of the skin. Since wind turbine blades often operate for a long time under comprehensive working conditions such as startup, yaw, pitch change, and shutdown, the complex random load distribution and natural meteorological conditions increase the difficulty of local stress calculation, and are also prone to fatigue failure of blades due to alternating loads, resulting in deterioration of blade health. Therefore, it is necessary to monitor the cracks of wind turbine blades in real time, analyze the laws of crack generation, growth, and expansion, study the relationship between blade dynamic loads and fatigue damage of wind turbine blades, and determine the degree of fatigue damage and identify the type of blade failure based on the positioning and quantitative diagnosis results of wind turbine blade cracks, thereby improving the reliability of wind turbine blades and avoiding high maintenance costs after major accidents.

[0004] In the traditional drone inspection process, the height and shooting direction of the drone need to be judged by the drone operator through the airborne camera platform or the ground manual perspective. The entire inspection process is too dependent on the flight control level of the drone operator, and there are problems such as high control difficulty, many human factors, low shooting accuracy, and short flight time, which greatly reduces the efficiency of the inspection work and exposes the difficulties in training a large number of high-level drone flight control personnel. It takes 3 to 12 months of training to take up the post, and the learning cost is high. Therefore, it is urgent to create a set of drone autonomous inspection processes to achieve one-click takeoff, automatic route planning, and autonomous inspection. Summary of the invention

[0005] (I) Purpose of the invention

[0006] The purpose of the present invention is to provide a method and system for autonomous inspection of wind turbine blades by drones, which integrates a visible light camera, a lidar and an edge computing module to meet the hardware requirements of wind turbine inspection, and is combined with an inspection APP that can perform task management and auxiliary inspection operations to form an integrated platform with functions such as task management, intelligent fault detection, fault statistics, and report output, which greatly facilitates the unmanned inspection of wind turbines.

[0007] (II) Technical solution

[0008] In order to solve the above problems, the first aspect of the present invention provides a wind turbine drone autonomous inspection method, comprising:

[0009] A laser radar is mounted on the drone, and a three-dimensional model of the wind turbine is established based on the data collected by the laser radar on the wind turbine, wherein the three-dimensional model of the wind turbine is constructed by using a recurrent neural network;

[0010] Planning an inspection route of the drone based on the three-dimensional model of the wind turbine;

[0011] Setting up a smart inspection APP, wherein the smart inspection APP has a task list for the drone;

[0012] Determining an autonomous flight path of the UAV based on the inspection route and the task list;

[0013] During autonomous flight, dynamically control the camera gimbal installed on the drone to collect images of wind turbine blades through the laser radar while the camera gimbal rotates and upload the images to the service terminal, in which a wind turbine defect detection module is provided and the laser radar is fixed on the camera gimbal;

[0014] The fan defect detection module automatically identifies the image to be detected, determines the fan defect type and locates the fan defect position;

[0015] The defect locations of the fan are counted and an analysis inspection report is output to complete the inspection.

[0016] Furthermore, the laser radar is mounted on the drone, and a three-dimensional model of the wind turbine is established based on the data collected by the laser radar on the wind turbine, including:

[0017] Equipped with a multi-line laser radar on the drone, the multi-line laser radar is used to scan different positions and directions of the wind turbine to obtain multiple laser radar point cloud data of the wind turbine;

[0018] Integrate multiple LiDAR point cloud data and build a 3D point cloud model of the wind turbine based on SLAM technology;

[0019] Construct a 3D model of the wind turbine based on the 3D point cloud model of the wind turbine.

[0020] Furthermore, constructing a three-dimensional model of the wind turbine based on the three-dimensional point cloud model of the wind turbine includes:

[0021] Introducing a recurrent neural network structure into the three-dimensional point cloud model of the wind turbine to construct a deep three-dimensional point cloud model of the wind turbine;

[0022] The wind turbine three-dimensional point cloud model is textured to generate a three-dimensional model of the wind turbine.

[0023] Furthermore, the construction of a three-dimensional model of a wind turbine based on SLAM technology includes:

[0024] First, the continuous frame data is matched in a high-frequency mode to determine the six-degree-of-freedom posture of the wind turbine and obtain a first laser odometer, wherein the laser radar point cloud data includes continuous frame data, and the high-frequency mode is 9-11 Hz;

[0025] The first laser odometer is then matched with the local SLAM map in a low-frequency mode to obtain a second laser odometer, and a three-dimensional model of the wind turbine is constructed based on the second laser odometer, wherein the low-frequency mode is 0.8 to 1.2 Hz.

[0026] Furthermore, the inspection route includes a blade inspection route, a cabin inspection route and a tower inspection route.

[0027] Furthermore, the blade inspection route includes: a leading edge windward surface, a leading edge leeward surface, a trailing edge windward surface and a trailing edge leeward surface of each blade.

[0028] Furthermore, an anomaly detection model is provided in the fan defect detection module, and the anomaly detection model is constructed based on the DAB-Net network structure and the improved EfficientNet algorithm.

[0029] Furthermore, the DAB-Net network structure adopts a 3×3 convolution module and a dual-branch structure.

[0030] Furthermore, the improved EfficientNet algorithm uses EfficientNet-B2 as the backbone network and multi-scale feature fusion BiFPN as the neck network.

[0031] A second aspect of the present invention provides a wind turbine drone autonomous inspection system, comprising:

[0032] Establish a model module, which is used to carry a laser radar on the drone and establish a three-dimensional model of the wind turbine based on the data collected by the laser radar on the wind turbine, wherein the three-dimensional model of the wind turbine is constructed by using a recurrent neural network;

[0033] A route planning module, used for planning the inspection route of the UAV based on the three-dimensional model of the wind turbine;

[0034] Setting an application module, used to set up a smart inspection APP, wherein the smart inspection APP has a task list for drones;

[0035] A flight path determination module, used to determine the autonomous flight path of the UAV based on the inspection route and the task list;

[0036] An image upload module is used to dynamically control the camera gimbal installed on the drone during autonomous flight, so as to collect the blade image of the wind turbine through the laser radar while the camera gimbal rotates and upload it to the server, in which a fan defect detection module is provided, and the laser radar is fixed on the camera gimbal;

[0037] An identification and positioning module, used to automatically identify the image to be detected through the fan defect detection module, determine the fan defect type and locate the fan defect position;

[0038] The analysis and inspection module is used to count the defect locations of the fan and output an analysis and inspection report to complete the inspection.

[0039] (III) Beneficial effects

[0040] The above technical scheme of the present invention has the following beneficial technical effects: The present invention provides a method and system for autonomous inspection of wind turbine blades by drones. The method uses drones as carriers, and obtains high-definition photos of wind turbine blades in a stationary state by carrying payload equipment such as laser radar on the drones. A three-dimensional model of the wind turbine is established based on the collected photo data. The three-dimensional model of the wind turbine is constructed by a recurrent neural network, and the depth map corresponding to each image is restored by a pre-established training model, and then the depth map is fused to reduce memory consumption and improve the applicability of large-scale three-dimensional scene reconstruction. The autonomous flight path of the drone is determined by planning the inspection route and task list of the drone. Then, the photos taken during the flight are input into the fan defect detection module. The fan defect detection module can automatically identify the photos to be detected, automatically detect the fault information in the photos, and automatically classify, locate, and store the fault data, thereby determining the type of fan defect and locating the location of the fan defect, so as to guide and assist the inspection personnel to efficiently handle the blade faults, and realize the inspection of the fan blades to replace the traditional inspection method. The inspection drone of the present invention integrates a visible light camera, a laser radar, and an edge computing module to meet the hardware requirements of wind turbine inspections. Combined with an intelligent inspection APP that can perform task management and auxiliary inspection operations, it constitutes an integrated platform with functions such as task management, intelligent fault detection, fault statistics, and report output, which greatly facilitates the unmanned inspection of wind turbines. The present invention realizes the autonomous inspection process of drones, achieving one-key takeoff, automatic route planning, and autonomous inspection. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a flow chart of a method for autonomous inspection of wind turbine blades by a UAV of the present invention;

[0042] Figure 2 It is a schematic diagram of the LOAM scheme framework in a specific embodiment of the present invention;

[0043] Figure 3 A schematic diagram of a recurrent neural network structure in a method for autonomous inspection of wind turbine blades by a drone according to the present invention;

[0044] Figure 4 This is a schematic diagram of a first type of wind turbine blade inspection waypoint in a specific embodiment of the present invention;

[0045] Figure 5 A schematic diagram of a second wind turbine blade inspection waypoint in a specific embodiment of the present invention;

[0046] Figure 6 A schematic diagram of a third wind turbine blade inspection waypoint in a specific embodiment of the present invention;

[0047] Figure 7 It is a schematic diagram of waypoints in a blade inspection route in a specific embodiment of the present invention;

[0048] Figure 8 It is a schematic diagram of waypoints in the cabin inspection route in a specific embodiment of the present invention;

[0049] Fig. 9 This is a schematic diagram of a newly added waypoint in the tower inspection route in a specific embodiment of the present invention;

[0050] Fig.10 It is a schematic diagram of the effect of adjusting the blade shooting in a specific embodiment of the present invention;

[0051] Fig.11 It is a schematic diagram of the structure of the training and detection strategy of the sub-image + the whole image in a specific embodiment of the present invention;

[0052] Fig.12 It is a diagram of the architecture of the improved EfficientDet algorithm in a specific embodiment of the present invention;

[0053] Fig.13 It is a diagram of the improved BiFPN network structure in a specific embodiment of the present invention;

[0054] Fig.14 It is a schematic diagram of the process of a wind turbine blade autonomous inspection system of a UAV of the present invention;

[0055] Fig.15 It is the first detection effect in the specific embodiment of the present invention;

[0056] Fig.16 This is the second detection effect in the specific embodiment of the present invention. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention is further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings. It should be understood that these descriptions are only exemplary and are not intended to limit the scope of the present invention. In addition, in the following description, the description of well-known structures and technologies is omitted to avoid unnecessary confusion of the concept of the present invention.

[0058] like Figure 1 As shown, the present invention provides, on one hand, a wind turbine drone autonomous inspection method, comprising:

[0059] S1, a laser radar is mounted on the drone, and a three-dimensional model of the wind turbine is established based on the data collected by the laser radar on the wind turbine, wherein the three-dimensional model of the wind turbine is constructed by using a recurrent neural network;

[0060] S2, planning an inspection route of the drone based on the three-dimensional model of the wind turbine;

[0061] S3, setting up a smart inspection APP, wherein the smart inspection APP has a task list for the drone;

[0062] S4, determining an autonomous flight path of the UAV based on the inspection route and the task list;

[0063] S5, during the autonomous flight, dynamically controlling the camera gimbal installed on the drone, so as to collect the blade image of the wind turbine through the laser radar while the camera gimbal rotates and upload it to the service terminal, wherein the service terminal is provided with a wind turbine defect detection module, and the laser radar is fixed on the camera gimbal;

[0064] S6, automatically identifying the image to be detected by the fan defect detection module, determining the fan defect type and locating the fan defect position;

[0065] S7, counting the defect locations of the fan and outputting an analysis inspection report to complete the inspection.

[0066] Furthermore, in step S1, a laser radar is mounted on the drone, and a three-dimensional model of the wind turbine is established based on the data collected by the laser radar on the wind turbine, which specifically includes:

[0067] 1.1 Equip the drone with a multi-line laser radar, use the multi-line laser radar to scan different positions and directions of the wind turbine, and obtain multiple laser radar point cloud data of the wind turbine. The present invention uses the laser radar to emit pulsed laser as the detection signal, which can well avoid the problem that the image feature extraction and matching cannot be performed due to the insufficient texture features of the wind turbine. Laser radar can be divided into single-line laser radar and multi-line laser radar according to the different laser beams. In the actual three-dimensional reconstruction of the wind turbine, the drone is equipped with a multi-line laser radar, which can obtain the three-dimensional point cloud data of the wind turbine body, and in order to scan the complete three-dimensional point cloud data of the wind turbine, it is necessary to collect data from multiple positions and different directions of the wind turbine.

[0068] 1.2 Integrate multiple laser radar point cloud data and build a 3D point cloud model of the wind turbine based on SLAM technology. Integrate the data collected from multiple positions and directions of the wind turbine in step 1.1 to generate a complete 3D point cloud model of the wind turbine. However, due to the accuracy error of the collection position and the point cloud distortion of the laser point cloud, the rigidly integrated 3D point cloud model of the wind turbine will inevitably have problems such as stratification and dislocation. Laser SLAM technology is needed to solve these problems and build a complete and detailed 3D point cloud model of the wind turbine).

[0069] Simultaneous Localization and Mapping (SLAM): Front-end scanning matching is the core module of laser SLAM. Its working content is to estimate the pose of the current frame by using the transformation relationship between adjacent frames when the pose of the previous frame is known. This transformation relationship is achieved through point cloud matching. The current mainstream approach is to first extract feature points and then use point-line matching and point-surface matching to restore the pose. Front-end scanning matching can obtain the laser odometer in a short time, but due to the inevitable error accumulation, the back-end optimization is to optimize the pose and map information of the laser odometer after a long period of incremental scanning matching. According to the different mathematical optimization methods adopted by the back-end, it can be divided into filter-based and graph-based laser SLAM, and the adjustment is performed in the form of graph optimization. Currently, there are three popular nonlinear optimization libraries: g2o, gtsam, and ceres. Closed-loop detection is responsible for eliminating the accumulated errors of some laser frames by detecting closed loops and using closed-loop constraints to correct the pose information and reduce the drift and ghosting of the point cloud map, so as to generate a globally consistent map. The map construction module is responsible for generating, optimizing and maintaining the global map.

[0070] The SLAM technology-based construction of the wind turbine 3D model in the present invention divides the complex positioning and mapping problems into a combination of two algorithms, namely, rough registration and fine registration. Figure 2 As shown in Figure 1, this solution is also called the LOAM solution, which specifically includes:

[0071] 1.2.1 First, match the continuous frame data in high-frequency mode to determine the six-degree-of-freedom posture of the wind turbine and obtain the first laser odometer. The laser radar point cloud data includes continuous frame data, and the high-frequency mode is 9-11Hz. For example, the high-frequency mode (10Hz) is used to match continuous frames (sweep-to-sweep) to roughly restore the six-degree-of-freedom (DOF) posture and obtain the first laser odometer (low-precision laser odometer).

[0072] 1.2.2 Then match the first laser odometer with the local SLAM map in low-frequency mode to obtain the second laser odometer, and build a three-dimensional model of the wind turbine based on the second laser odometer. The low-frequency mode is 0.8-1.2Hz. For example, the first laser odometer is matched with the local SLAM map (sweep-to-submap) at (1Hz) to further optimize the position and posture of the first laser odometer. The front end adopts a feature point-based matching method to make full use of the line feature points and surface feature points in the environment. It is also the mainstream method of 3D laser SLAM at present. The inertial navigation measurement data is used to remove motion distortion of each frame of laser point cloud before point cloud registration. After the camera and lidar are calibrated, the point cloud can be projected into the image.

[0073] 1.3 Construct a 3D model of the wind turbine based on the 3D point cloud model of the wind turbine, including:

[0074] 1.3.1 Introduce a recurrent neural network structure into the three-dimensional point cloud model of the wind turbine to construct a deep three-dimensional point cloud model of the wind turbine. Introduce a recurrent neural network architecture, and sequentially regularize the two-dimensional cost map through the GRU unit in the depth direction to reduce memory consumption and improve the applicability of large-scale three-dimensional scene reconstruction. The network architecture is as follows Figure 3 shown. Figure 3 In the method, the input multi-view image first extracts features in a two-dimensional feature extraction layer, and is transformed to the frontal parallel plane of the camera viewing cone of the reference image via a differentiable homography matrix. Then, the cost is calculated at different depths and regularized by a convolutional GRU unit, so that it is possible to obtain geometric and unidirectional semantic information in the depth direction. The present invention adopts a recurrent neural network structure, and a deep map fusion method based on deep learning can output a dense point cloud with higher restoration. By establishing a training model containing semantic information, highlight or reflection information can be learned, so as to better perform two-dimensional to three-dimensional matching, and effectively avoid the disadvantage that the traditional method is easily disturbed by weak texture and light reflection.

[0075] 1.3.2 Texture the three-dimensional point cloud model of the wind turbine to generate a three-dimensional model of the wind turbine. It is difficult to reflect the texture information of the scene from the sparse point cloud or dense point cloud obtained. Therefore, it is necessary to further process the three-dimensional point cloud to obtain a three-dimensional model, and then texture map the model. Only in this way can the result truly reflect the scene information. The process from three-dimensional points to three-dimensional models is called mesh reconstruction, and the process of restoring the real scene on a mesh without color information is called texture mapping.

[0076] Furthermore, the inspection routes include blade inspection routes, cabin inspection routes and tower inspection routes. During the drone inspection process, in order to achieve full coverage and refined inspection of wind turbines, it is necessary to plan routes that can be used for automated inspection of wind turbines to ensure the safety of the drone flight process and the acquisition of all wind turbine surface images. Intelligent route planning technology can not only realize the automatic generation of wind turbine inspection routes, but also perform safety checks on the generated wind turbine inspection routes (including blade inspections, cabin inspections and tower inspections). The descriptions of each inspection route are as follows:

[0077] (1) Blade inspection route: including the leading edge windward side, leading edge leeward side, trailing edge windward side and trailing edge leeward side of each blade. According to the blade recognition results, the blade points to be photographed are calculated. Blade inspection is to design a route so that the drone can automatically inspect these blade points to be photographed. When inspecting blades, it is necessary to inspect the front and rear four surfaces of each blade (the leading edge windward side, the leading edge leeward side, the trailing edge windward side and the trailing edge leeward side) respectively, such as Figure 4-6The inspection route includes not only the photography waypoints of the locations to be photographed, but also the transfer waypoints from the front to the back of the wind turbine. The waypoints in the blade inspection route are as follows: Figure 7 The black dots show:

[0078] (2) Cabin inspection route: During cabin inspection, it is necessary to inspect the five directions around the cabin (the upper rear of the cabin, the rear of the cabin, the lower rear of the cabin, the left side of the cabin, and the right side of the cabin). The waypoints in the cabin inspection route are as follows: Figure 8 The black dots show:

[0079] (3) Tower inspection route: When inspecting the tower, it is necessary to inspect the three directions around the tower (the front of the tower, the left rear of the tower, and the right rear of the tower). To avoid the drone colliding with the wind turbine tower during the inspection, the blade situation needs to be considered when switching between the three directions of the tower. Therefore, a new transfer waypoint is added when switching to avoid possible blades. The new transfer waypoints are as follows: Fig. 9 Indicated by the black dots: Intelligent route planning technology also includes route task management and route operation and maintenance management. Route task management includes: supporting the creation, editing, and deletion management of route inspection tasks. Route operation and maintenance management supports the creation, editing, and deletion of route operation and maintenance tasks, and realizes monitoring and tracking of the entire operation and maintenance process.

[0080] Furthermore, in the step S5, during the autonomous flight, the camera gimbal installed on the drone is dynamically controlled to collect the blade image of the wind turbine through the laser radar while the camera gimbal is rotating and upload it to the service terminal. The service terminal is provided with a wind turbine defect detection module, and the laser radar is fixed on the camera gimbal. In this step, the quality of the images taken during the blade inspection is fully guaranteed by using an image-based lighting analysis algorithm. During the inspection process, the wind turbine blades and tower are segmented to obtain key areas, and the images of the key areas are subjected to grayscale analysis to determine whether the lighting is appropriate, so as to give a prompt when the lighting is too dark or too bright, and adjust the exposure in time to ensure the quality of the blade shooting. The adjustment effect is as follows: Fig.10 shown.

[0081] Furthermore, in step S6, the fan defect detection module automatically identifies the image to be detected, determines the fan defect type and locates the fan defect position. In this step, an anomaly detection model is provided in the fan defect detection module, and the anomaly detection model is constructed based on the DAB-Net network structure and the improved EfficientNet algorithm. In order to reduce background false detection and weaken the negative impact of the multi-scale differences of abnormal targets on model feature learning, the idea of ​​DABNet semantic segmentation + EfficientDet target detection is adopted, and the training and detection strategy of sub-image + whole image is adopted, such as Fig.11 As shown, the model composition is introduced as follows:

[0082] 6.1 DABNet semantic segmentation: Based on DAB-Net, a deep asymmetric bottleneck (DAB module) that can be used for real-time semantic segmentation is proposed. Based on the DAB module, the DABNet structure adopts a 3×3 convolution module and a dual-branch structure, which is described as follows:

[0083] (1) First, 3×3 convolution is used to reduce the number of channels instead of 1×1 convolution. Although 1×1 convolution can reduce the amount of calculation, 1×1 convolution is used in ResNet mainly to increase the depth of the network, and here the network depth does not need to be more than 100 layers.

[0084] (2) In order to better obtain spatial information, the number of channels is reduced by half after convolution, but it is not compressed too much.

[0085] (3) Since the previous network structure has proved that layered networks are more effective for semantic segmentation tasks, DAB-Net also integrates multi-scale information and adopts a dual-branch structure. One branch obtains local information. In order to reduce the amount of calculation, it draws on ERF-Net and decomposes the convolution into a depth-wise separable convolution and a depth-wise separable convolution. In order to obtain edge semantic information, the other branch uses a dilated convolution. Without reducing the resolution, considering that the dilated convolution will increase the amount of calculation as the rate increases, DAB-Net combines it with the depth-wise asymmetric convolution. The experimental results show that compared with the dilated convolution rate {4,4,8,8,16,16},

[0086] {3,3,7,7,13,13} is 2.3 points higher. Adding the decoder does not improve the effect much, but it greatly affects the speed. In addition, adding spp can improve the effect, but it will also affect the speed.

[0087] 6.2 Ideas for Improving EfficientDet Target Detection

[0088] Considering that the imaging distance of wind turbine blades photographed by drones is far due to the influence of GPS positioning deviation, in order to effectively detect small defects on wind turbine blades, EfficientDet was improved in the wind turbine anomaly detection project. The improved EfficientDet algorithm architecture is shown in the figure below. Fig.12 As shown, the process is described as follows:

[0089] (1) The improved EfficientNet-B2 is selected as the backbone feature extraction network for preliminary feature extraction, and five feature layers {P1, P2, P3, P4, P5} are obtained by downsampling 1 to 5 times;

[0090] (2) The improved BiFPN is connected in series as the neck network, so that multiple feature layers can be fused in a bidirectional multi-scale manner from top to bottom and from bottom to top, and five effective feature layers are obtained;

[0091] (3) The five effective feature layers obtained after fusion are passed into the classifier and regressor in EfficientHead to classify the prior frame and adjust the position of the prior frame to obtain the predicted frame;

[0092] Among them, the improved BiFPN is an enhanced version of FPN. It removes the nodes with only one input edge that contributes less to the fusion of features at different levels, then adds an extra edge to the original input and output in the same layer, and adds an extra edge to the original input and output in the upper and lower layers, respectively. Finally, each top-down and bottom-up bidirectional path is repeated to achieve more advanced feature fusion. The network structure is as follows: Fig.13 The present invention combines multiple data enhancement methods such as Mosaic and CutMix to train the detection model, and uses four types of targets that are easy to be misdetected, such as common lifting signs, drainage holes, center of gravity positions and hooks, as training categories. Through the classification ability of the network, they are identified and eliminated in the project to ensure the accuracy of the identified targets. The effect spectrum is shown in Fig.15 The algorithm can detect tiny detached targets on the blades. Tiny gel coat detachments can be detected normally with high confidence. The experimental results can verify the rationality and superiority of the algorithm. Fig.16 .

[0093] Furthermore, in step S7, the defect locations of the fan are counted and an analysis inspection report is output, and completing the inspection specifically includes: counting the fan defects and outputting an analysis report, and automatically generating an inspection report, including flight operation records, abnormal statistics and summaries, defect descriptions, etc.

[0094] Furthermore, the smart inspection APP interface has three modules on the homepage of the APP software: route parameters, wind turbine list, and task list, which are used to set drone parameters, execute tasks, and query historical task information. The functions of each module are as follows:

[0095] (1) Route parameters: used to set the flight route parameters of the drone, including flight speed, inspection distance, first flight safety altitude, etc.

[0096] (2) Fan List: This is a fan management module that can create or import fan data and perform inspection tasks on target fans.

[0097] (3) Task list: This is a task management module that records all executed tasks. You can view the detailed parameters of the task under the task name.

[0098] like Fig.14 As shown, another aspect of the present invention provides a wind turbine UAV autonomous inspection system, the system comprising:

[0099] 31. Establish a model module, which is used to carry a laser radar on the drone, and establish a three-dimensional model of the wind turbine based on the data collected by the laser radar on the wind turbine, wherein the three-dimensional model of the wind turbine is constructed by using a recurrent neural network;

[0100] 22. A route planning module, used to plan the inspection route of the UAV based on the three-dimensional model of the wind turbine;

[0101] 23. Setting an application module, used to set up an intelligent inspection APP, wherein the intelligent inspection APP has a task list for the drone;

[0102] 24 a flight path determination module, used to determine the autonomous flight path of the UAV based on the inspection route and the task list;

[0103] 25 image upload module, used for dynamically controlling the camera gimbal installed on the UAV during autonomous flight, so as to collect the blade image of the wind turbine through the laser radar while the camera gimbal rotates and upload it to the server, wherein the server is provided with a fan defect detection module, and the laser radar is fixed on the camera gimbal;

[0104] 26 identification and positioning module, used to automatically identify the image to be detected through the fan defect detection module, determine the fan defect type and locate the fan defect position;

[0105] 27 Analysis and inspection module, used to count the defect locations of the fan and output an analysis and inspection report to complete the inspection.

[0106] Based on any of the wind turbine drone autonomous inspection methods described above, the implementation process of drone autonomous inspection is as follows:

[0107] (1) Collect the blade length, tower height, and GPS coordinates of the base below the wind turbine hub;

[0108] (2) Place the aircraft in a clear position in the field of view (make sure there are no obstacles such as blades directly above the aircraft). The system will automatically generate a bird's-eye view route. The remote controller will confirm the execution and the aircraft will automatically fly to hover directly above the wind turbine.

[0109] (3) The aircraft will automatically take a bird's-eye view of the wind turbine when it is directly above the wind turbine. The system will automatically calculate the wind turbine's heading angle. The remote controller will correct and determine the wind turbine's heading angle. The system will automatically generate a front view route, and the aircraft will automatically fly to the front of the wind turbine and hover.

[0110] (4) The aircraft will automatically take a front view of the wind turbine when it is directly in front of it. The system will automatically calculate the blade position angle. The remote controller will correct and determine the blade position angle. The system will automatically generate a wind turbine inspection route and perform inspection operations on the wind turbine.

Claims

1. A wind turbine drone autonomous inspection method, characterized in that: include: A laser radar is mounted on the drone, and a three-dimensional model of the wind turbine is established based on the data collected by the laser radar on the wind turbine, wherein the three-dimensional model of the wind turbine is constructed by using a recurrent neural network; Planning an inspection route of the drone based on the three-dimensional model of the wind turbine; Setting up a smart inspection APP, wherein the smart inspection APP has a task list for the drone; Determining an autonomous flight path of the UAV based on the inspection route and the task list; During autonomous flight, dynamically control the camera gimbal installed on the drone to collect images of wind turbine blades through the laser radar while the camera gimbal rotates and upload the images to the service terminal, in which a wind turbine defect detection module is provided and the laser radar is fixed on the camera gimbal; The fan defect detection module automatically identifies the image to be detected, determines the fan defect type and locates the fan defect position; The defect locations of the fan are counted and an analysis inspection report is output to complete the inspection.

2. The wind turbine drone autonomous inspection method according to claim 1 is characterized in that: The method of carrying a laser radar on the drone and establishing a three-dimensional model of the wind turbine based on the data collected by the laser radar on the wind turbine includes: Equipped with a multi-line laser radar on the drone, the multi-line laser radar is used to scan different positions and directions of the wind turbine to obtain multiple laser radar point cloud data of the wind turbine; Integrate multiple LiDAR point cloud data and build a 3D point cloud model of the wind turbine based on SLAM technology; Construct a 3D model of the wind turbine based on the 3D point cloud model of the wind turbine.

3. The wind turbine drone autonomous inspection method according to claim 2 is characterized in that: The constructing of the three-dimensional model of the wind turbine based on the three-dimensional point cloud model of the wind turbine comprises: Introducing a recurrent neural network structure into the three-dimensional point cloud model of the wind turbine to construct a deep three-dimensional point cloud model of the wind turbine; The wind turbine three-dimensional point cloud model is textured to generate a three-dimensional model of the wind turbine.

4. The wind turbine drone autonomous inspection method according to claim 2 is characterized in that: The construction of a three-dimensional model of a wind turbine based on SLAM technology includes: First, the continuous frame data is matched in a high-frequency mode to determine the six-degree-of-freedom posture of the wind turbine and obtain a first laser odometer, wherein the laser radar point cloud data includes continuous frame data, and the high-frequency mode is 9-11 Hz; The first laser odometer is then matched with the local SLAM map in a low-frequency mode to obtain a second laser odometer, and a three-dimensional model of the wind turbine is constructed based on the second laser odometer, wherein the low-frequency mode is 0.8 to 1.2 Hz.

5. The wind turbine drone autonomous inspection method according to claim 1 is characterized in that: The inspection routes include blade inspection routes, cabin inspection routes and tower inspection routes.

6. The wind turbine drone autonomous inspection method according to claim 5 is characterized in that: The blade inspection route includes: the leading edge windward surface, the leading edge leeward surface, the trailing edge windward surface and the trailing edge leeward surface of each blade.

7. The wind turbine drone autonomous inspection method according to claim 1 is characterized in that: The fan defect detection module is provided with an anomaly detection model, and the anomaly detection model is constructed based on the DAB-Net network structure and the improved EfficientNet algorithm.

8. The wind turbine drone autonomous inspection method according to claim 7 is characterized in that: The DAB-Net network structure adopts a 3×3 convolution module and a dual-branch structure.

9. The wind turbine drone autonomous inspection method according to claim 7, characterized in that: In the improved EfficientNet algorithm, EfficientNet-B2 is used as the backbone network, and multi-scale feature fusion BiFPN is used as the neck network.

10. A wind turbine drone autonomous inspection system, characterized in that: include: Establish a model module, which is used to carry a laser radar on the drone and establish a three-dimensional model of the wind turbine based on the data collected by the laser radar on the wind turbine, wherein the three-dimensional model of the wind turbine is constructed by using a recurrent neural network; A route planning module, used for planning the inspection route of the UAV based on the three-dimensional model of the wind turbine; Setting an application module, used to set up a smart inspection APP, wherein the smart inspection APP has a task list for drones; A flight path determination module, used to determine the autonomous flight path of the UAV based on the inspection route and the task list; An image upload module is used to dynamically control the camera gimbal installed on the drone during autonomous flight, so as to collect the blade image of the wind turbine through the laser radar while the camera gimbal rotates and upload it to the server, in which a fan defect detection module is provided, and the laser radar is fixed on the camera gimbal; An identification and positioning module, used to automatically identify the image to be detected through the fan defect detection module, determine the fan defect type and locate the fan defect position; The analysis and inspection module is used to count the defect locations of the fan and output an analysis and inspection report to complete the inspection.

Images