A wind turbine tower blade detection system based on drone

Abstract

The present invention provides a wind turbine tower blade detection system based on an unmanned aerial vehicle, the system comprising an airframe control module, an image acquisition module, a communication module, a detection and diagnosis module and an energy management module; the airframe control module is used for controlling the stable flight and precise navigation of the unmanned aerial vehicle; the communication module is used for completing data transmission between the unmanned aerial vehicle and a ground control station; the image acquisition module is used for collecting images of wind turbine tower blades during the flight of the unmanned aerial vehicle; the detection and diagnosis module is used for completing the detection and diagnosis of wind turbine tower blades; the energy management module is used for monitoring and managing the energy usage of the unmanned aerial vehicle during flight; the present invention realizes comprehensive monitoring and maintenance of wind turbine tower blades by integrating advanced airframe control, efficient image acquisition and processing and detailed detection and diagnosis schemes, thereby significantly improving the operating safety and maintenance efficiency of wind power equipment.

Description

Technical Field

[0001] The present invention relates to the field of wind turbine blade detection systems, and in particular to a wind turbine tower blade detection system based on an unmanned aerial vehicle. Background Art

[0002] With the rapid development of the renewable energy industry, wind power energy, as an important part of renewable energy, the maintenance and efficiency optimization of its related equipment have become particularly important; in particular, the blades of the wind turbine tower, as a key part of the wind turbine, their health status directly affects the performance and safety of the entire power generation system; since wind turbine blades are often exposed to a complex and changing external environment, they are prone to wear, damage or aging, and timely and effective inspection and maintenance are particularly critical; traditional wind turbine blade inspection methods often rely on manual inspection or ground infrastructure, which is not only costly but also inefficient, and it is difficult to meet the needs of the rapidly developing wind power industry; therefore, finding a more efficient and cost-effective wind turbine blade inspection technology has become an important demand for the development of the industry.

[0003] According to the related disclosed technical solutions, the technology with publication number CN117386567A proposes a method and system for detecting fan blades, which includes: obtaining a real-time aerial perspective image of a flying robot system, extracting data from the real-time aerial perspective image through a center voting method to construct a simplified blade model of the wind turbine blade to be tested, generating a safe flight reference trajectory based on the simplified blade model and a safe flight offset distance, calculating an optimal placement position based on the safe flight reference trajectory and the simplified blade model, and controlling the flying robot system to place a crawling robot on the fan blade at a hovering position corresponding to the optimal placement position, controlling the crawling robot to detect the fan blade, and obtaining detection result information, and detecting the fan blade through the fan blade detection in combination with posture information, safe flight reference trajectory and optimal placement position, thereby improving the detection efficiency of the fan blade; however, the solution requires the use of an additional robot system for precise placement and operation, resulting in high operation complexity and significantly increased costs; in addition, due to the lack of the integration and splicing function of the blade image, the solution may not be able to fully capture all key detection structure details, limiting the comprehensiveness and accuracy of the detection. Summary of the invention

[0004] The purpose of the present invention is to propose a wind turbine tower blade detection system based on a drone to address the current deficiencies.

[0005] The present invention adopts the following technical solution:

[0006] A wind turbine tower blade detection system based on an unmanned aerial vehicle, the system comprising a body control module, an image acquisition module, a communication module, a detection and diagnosis module and an energy management module;

[0007] The body control module is used to control the stable flight and precise navigation of the UAV; the communication module is used to complete the data transmission between the UAV and the ground control station; the image acquisition module is used to collect the image of the wind turbine tower blades during the flight of the UAV; the detection and diagnosis module is used to complete the detection and diagnosis of the wind turbine tower blades; the energy management module is used to monitor and manage the energy usage of the UAV during flight;

[0008] The body control module includes a flight control unit and an automatic obstacle avoidance unit; the flight control unit is used to control the UAV to fly along a set detection route, and the automatic obstacle avoidance unit is used to control the UAV to detect obstacles in the flight path in real time during flight and adjust the flight path of the UAV to achieve real-time obstacle avoidance;

[0009] The detection and diagnosis module includes an image preprocessing unit, a real-time detection unit and an early warning unit; the image preprocessing unit is used to preprocess and store the wind turbine tower blade images collected by the drone in each flight mission; the real-time detection unit is used to detect the condition of the wind turbine tower blades when the drone performs the flight mission; the early warning unit is used to complete the early warning of abnormal conditions of the wind turbine tower blades in the future according to the analysis of the detection results of the real-time detection unit in the historical records;

[0010] Furthermore, the image preprocessing unit further includes an image acquisition subunit, an image stitching subunit, and an image storage subunit; the image acquisition subunit is used to acquire the wind turbine tower blade images continuously taken by the image acquisition module, the image stitching subunit is used to correct and stitch the continuously taken wind turbine tower blade images; the image storage subunit is used to complete the storage of the stitched blade images;

[0011] Furthermore, the workflow of the image stitching subunit is as follows:

[0012] S11: Acquire continuously photographed wind turbine tower blade images;

[0013] S12: Correct the brightness, contrast and color of all leaf images;

[0014] S13: for all corrected leaf images, identify and extract key feature points of each leaf image, and generate a feature descriptor for each key feature point; the feature descriptor is obtained based on the gradient information of the area around the corresponding feature point;

[0015] S14: matching the key feature points extracted from each image according to the distance measurement between the feature descriptors of the key feature points in each image;

[0016] S15: performing geometric transformation and alignment operations on each image to stitch them together according to the matched key feature points, and performing fusion processing on the stitching area of ​​the image;

[0017] S16: Generate and output the spliced ​​leaf image;

[0018] Furthermore, the real-time detection unit completes the detection of the blade condition by inputting the spliced ​​blade image into a pre-trained blade defect recognition model; the output of the blade defect recognition model includes various defect types and defect severity levels corresponding to the defect types;

[0019] Furthermore, the real-time detection unit sets an alarm threshold for each defect type. During the detection of the blade condition, when the severity of a defect type exceeds the set alarm threshold, the real-time detection unit will automatically trigger an alarm and send an alarm message to the staff; the alarm message includes the defect type, the severity of the defect and the specific location of the defect obtained according to the blade image;

[0020] Furthermore, the early warning unit completes the prediction alarm for the possible defect types of the blades by comprehensively analyzing the defect severity corresponding to each defect type in multiple historical detection results, and sends a warning message to the staff when it is predicted that abnormal defects may occur in the blades in the future.

[0021] The beneficial effects achieved by the present invention are:

[0022] The present invention realizes an efficient image processing process through the combination of an image acquisition subunit, an image stitching subunit, and an image storage subunit. By integrating, correcting, and stitching continuously captured local blade images, a complete blade image is generated and output, providing a more comprehensive blade view for subsequent analysis and identification, and more accurately identifying and analyzing blade surface defects. The real-time detection unit identifies blade defects in real time, and the early warning unit predicts possible blade defects in the future, so that measures can be taken in advance to prevent further development of defects. This predictive maintenance strategy not only reduces the frequency and cost of emergency repairs, but also improves the operating efficiency and safety of the overall equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The present invention can be further understood from the following description in conjunction with the accompanying drawings. The components in the figures are not necessarily drawn to scale, but the emphasis is placed on illustrating the principles of the embodiments. In different views, the same reference numerals designate corresponding parts.

[0024] Figure 1 It is a schematic diagram of the overall module of the present invention.

[0025] Figure 2Schematic diagram of the workflow of the image stitching subunit of the present invention.

[0026] Figure 3 It is a schematic flow chart of the method for fusion processing of the splicing area according to the present invention. DETAILED DESCRIPTION

[0027] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with its embodiments; it should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention; for those skilled in the art, other systems, methods and / or features of the present embodiment will become apparent after reviewing the following detailed description; it is intended that all such additional systems, methods, features and advantages are included in this specification; included within the scope of the present invention and protected by the appended claims; additional features of the disclosed embodiments are described in the following detailed description, and these features will be apparent from the following detailed description.

[0028] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if the terms "upper", "lower", "left", "right" and the like indicate directions or positional relationships based on the directions or positional relationships shown in the drawings, it is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or component referred to must have a specific direction, be constructed and operated in a specific direction. Therefore, the terms describing the positional relationship in the drawings are only used for illustrative purposes and cannot be understood as limitations on this patent. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0029] Embodiment 1.

[0030] like Figure 1 As shown, this embodiment provides a wind turbine tower blade detection system based on a drone, the system comprising a body control module, an image acquisition module, a communication module, a detection and diagnosis module and an energy management module;

[0031] The body control module is used to control the stable flight and precise navigation of the UAV; the communication module is used to complete the data transmission between the UAV and the ground control station; the image acquisition module is used to collect the image of the wind turbine tower blades during the flight of the UAV; the detection and diagnosis module is used to complete the detection and diagnosis of the wind turbine tower blades; the energy management module is used to monitor and manage the energy usage of the UAV during flight;

[0032] The body control module includes a flight control unit and an automatic obstacle avoidance unit; the flight control unit is used to control the UAV to fly along a set detection route, and the automatic obstacle avoidance unit is used to control the UAV to detect obstacles in the flight path in real time during flight and adjust the flight path of the UAV to achieve real-time obstacle avoidance;

[0033] The detection and diagnosis module includes an image preprocessing unit, a real-time detection unit and an early warning unit; the image preprocessing unit is used to preprocess and store the wind turbine tower blade images collected by the drone in each flight mission; the real-time detection unit is used to detect the condition of the wind turbine tower blades when the drone performs the flight mission; the early warning unit is used to complete the early warning of abnormal conditions of the wind turbine tower blades in the future according to the analysis of the detection results of the real-time detection unit in the historical records;

[0034] Furthermore, the image preprocessing unit further includes an image acquisition subunit, an image stitching subunit, and an image storage subunit; the image acquisition subunit is used to acquire the wind turbine tower blade images continuously taken by the image acquisition module, the image stitching subunit is used to correct and stitch the continuously taken wind turbine tower blade images; the image storage subunit is used to complete the storage of the stitched blade images;

[0035] Further, such as Figure 2 As shown, the workflow of the image stitching subunit is as follows:

[0036] S11: Acquire continuously photographed wind turbine tower blade images;

[0037] S12: Correct the brightness, contrast and color of all leaf images;

[0038] S13: for all corrected leaf images, identify and extract key feature points of each leaf image, and generate a feature descriptor for each key feature point; the feature descriptor is obtained based on the gradient information of the area around the corresponding feature point;

[0039] S14: matching the key feature points extracted from each image according to the distance measurement between the feature descriptors of the key feature points in each image;

[0040] S15: performing geometric transformation and alignment operations on each image to stitch them together according to the matched key feature points, and performing fusion processing on the stitching area of ​​the image;

[0041] S16: Generate and output the spliced ​​leaf image;

[0042] Furthermore, in step S13, the SIFT algorithm is used to identify and extract key feature points of each leaf image;

[0043] Further, such as Figure 3 As shown, in step S15, the fusion processing of the spliced ​​area is completed in the following manner:

[0044] S151: for two images to be stitched, decompose the two images into multiple levels containing different detail features through Gaussian filtering and down sampling, wherein the top level contains the most detail features and the bottom level contains the least detail features;

[0045] S152: For each level of the two stitched images, a fusion weight at each level is calculated according to the gradient amplitude of its pixels;

[0046] S153: using the fusion weights obtained in the previous step, effectively fuse the corresponding levels of the two stitched images, and reconstruct the fused levels layer by layer to generate a complete stitched leaf image;

[0047] Specifically, in step S151, firstly, Gaussian filtering is applied to the two images to smooth the images to reduce high-frequency noise and retain the main structure and low-frequency information; after applying the Gaussian filtering, the two images are downsampled to reduce the resolution of the images; after each Gaussian filtering and downsampling operation is performed, a level of the two images is obtained, and the Gaussian filtering and downsampling operations are repeatedly performed to obtain multiple levels of the two images containing different detail features, wherein the detail features of each level are image features such as edges, textures and structures contained in the level;

[0048] Furthermore, in step S152, the fusion weights of each layer are obtained by:

[0049] S1521: For two stitched images, calculate the gradient amplitude of the pixels of each image at each level:

[0050]

[0051]

[0052] Among them, F 1,i (x, y) is the gradient amplitude of the first stitched image at the i-th level, x represents the horizontal direction, y represents the vertical direction, I 1,x,i is the lateral gradient of the first spliced ​​image at level i, I 1,y,i is the longitudinal gradient of the first spliced ​​image at level i; F 2,i (x, y) is the gradient amplitude of the second stitched image at the i-th level, I 2,x,i is the lateral gradient of the second stitched image at level i, I 2,y,i is the longitudinal gradient of the second spliced ​​image at level i; let F 1,,i(x,y)≥F 2,,i (x,y);

[0053] S1522: Calculate the initial weight of each image at each level:

[0054]

[0055] Among them, W 1,i (x, y) is the initial weight of the first spliced ​​image at the i-th level, W 2,i (x,y) is the initial weight of the second spliced ​​image at level i;

[0056] S1523: Calculate the fusion weight of each image at each level:

[0057] W′ 1,i (x,y)=W 1,i (x,y)+ΔW i (x,y)×W 2,i (x,y);

[0058] W′ 2,i (x,y)=1-W′ 1,i (x,y);

[0059] Among them, W′ 1,i (x, y) is the fusion weight of the first spliced ​​image at the i-th level, W′ 2,i (x, y) is the fusion weight of the second stitched image at the i-th level, ΔW i (x,y) is the weight adjustment factor, satisfying:

[0060]

[0061] Where k is the adjustment coefficient, and its value range is [1,10];

[0062] Furthermore, in step S53, the complete spliced ​​leaf image is generated by fusion and reconstruction in the following manner:

[0063] S1531: For each level, generate the fusion output of that level:

[0064] L out,i =W′ 1,i (x,y)×L 1,i +W′ 2,i (x,y)×L 2,i

[0065] Among them, L out,i is the fusion output of the i-th level, L 1,i is the detail feature of the first image at level i; L 2,iis the detail feature of the second image at level i;

[0066] S1532: Starting from the lowest level, use the L of each layer obtained by fusion out,i Reconstructing layer by layer until the top level is reached, thereby generating a spliced ​​leaf image;

[0067] The blades of a wind turbine tower are huge in size, and a single image is difficult to cover the entire area. Using an effective stitching method to reconstruct a complete blade view from multiple images is crucial for maintenance and safety inspections. This embodiment decomposes the image into different levels for image fusion, and can effectively capture detailed information of the wind turbine blades at different levels, including detailed information from the overall outline to tiny cracks or wear. By dynamically adjusting the weights when fusing two stitched images at each level, more weights are set on the stitched images with more details based on the gradient amplitude analysis and calculation of the images at each level, thereby enhancing the level of detail during image fusion, thereby improving the detail and clarity of the stitched blade image, and providing a more accurate visual basis for subsequent blade condition analysis.

[0068] Embodiment 2:

[0069] This embodiment should be understood to include at least all the features of any of the above embodiments, and further improve upon them;

[0070] This embodiment provides a wind turbine tower blade detection system based on a drone, the system comprising a body control module, an image acquisition module, a communication module, a detection and diagnosis module, and an energy management module;

[0071] The body control module is used to control the stable flight and precise navigation of the UAV; the communication module is used to complete the data transmission between the UAV and the ground control station; the image acquisition module is used to collect the image of the wind turbine tower blades during the flight of the UAV; the detection and diagnosis module is used to complete the detection and diagnosis of the wind turbine tower blades; the energy management module is used to monitor and manage the energy usage of the UAV during flight;

[0072] The body control module includes a flight control unit and an automatic obstacle avoidance unit; the flight control unit is used to control the UAV to fly along a set detection route, and the automatic obstacle avoidance unit is used to control the UAV to detect obstacles in the flight path in real time during flight and adjust the flight path of the UAV to achieve real-time obstacle avoidance;

[0073] The detection and diagnosis module includes an image preprocessing unit, a real-time detection unit and an early warning unit; the image preprocessing unit is used to preprocess and store the wind turbine tower blade images collected by the drone in each flight mission; the real-time detection unit is used to detect the condition of the wind turbine tower blades when the drone performs the flight mission; the early warning unit is used to complete the early warning of abnormal conditions of the wind turbine tower blades in the future according to the analysis of the detection results of the real-time detection unit in the historical records;

[0074] Furthermore, the real-time detection unit completes the detection of the blade condition by inputting the spliced ​​blade image into a pre-trained blade defect recognition model; the output of the blade defect recognition model includes various defect types and defect severity levels corresponding to the defect types;

[0075] Furthermore, the real-time detection unit sets an alarm threshold for each defect type. During the detection of the blade condition, when the severity of a defect type exceeds the set alarm threshold, the real-time detection unit will automatically trigger an alarm and send an alarm message to the staff; the alarm message includes the defect type, the severity of the defect and the specific location of the defect obtained according to the blade image;

[0076] Furthermore, after the early warning unit has completed the analysis of the detection results of multiple real-time detection units in the historical records, it sends early warning information to the user to complete the early warning of abnormal conditions of wind turbine tower blades in the future; the early warning information includes the repair early warning priority information of the defect type of each wind turbine tower blade;

[0077] Furthermore, the specific working process of the early warning unit is as follows:

[0078] S21: for all wind turbine tower blades that have been inspected, obtain n inspection results closest to the current time, wherein the inspection results are various defect types and defect severity levels corresponding to the defect types output by the real-time inspection unit;

[0079] S22: For each defect type of each wind turbine tower blade, calculate the corresponding defect repair priority coefficient:

[0080]

[0081] Among them, K is the defect repair priority coefficient of a certain defect type, Q j is the alarm accumulation coefficient of the jth detection result, W avg is the average defect severity of all test results that do not reach the set alarm threshold, max(E j ) is the maximum value of the defect severity among all the test results that do not reach the set alarm threshold, and γ is the set alarm threshold;

[0082] For Q j satisfy:

[0083]

[0084] S23: sorting all defect types of each wind turbine tower blade from high to low according to the defect repair priority coefficient, wherein a defect type with a higher defect repair priority coefficient has a higher repair warning priority; and sending the sorting information of the defect types in each wind turbine tower blade as warning information to the user;

[0085] Due to the high operating and maintenance costs of wind turbine tower blades, it is more critical to detect and prevent blade defects in the early stages. By integrating multiple historical test results and analyzing the defect development trends of blades according to the severity of various defect types, early warning and repair priorities are provided for various defect types of each blade. Through this comprehensive monitoring and early warning, staff can better control operating and maintenance costs, respond to potential risks in advance, improve equipment reliability and safety, and ultimately reduce repair costs and extend equipment life.

[0086] The contents disclosed above are only preferred feasible embodiments of the present invention, and do not limit the protection scope of the present invention. Therefore, all equivalent technical changes made using the contents of the present invention specification and drawings are included in the protection scope of the present invention. In addition, the elements therein can be updated as technology develops.

Claims

1. A wind turbine tower blade detection system based on drone, characterized in that: The system includes a body control module, an image acquisition module, a communication module, a detection and diagnosis module and an energy management module; the body control module is used to control the stable flight and precise navigation of the UAV; the communication module is used to complete the data transmission between the UAV and the ground control station; the image acquisition module is used to collect wind turbine tower blade images during the flight of the UAV; the detection and diagnosis module is used to complete the detection and diagnosis of wind turbine tower blades; the energy management module is used to monitor and manage the energy usage of the UAV during flight; the body control module includes a flight control unit and an automatic obstacle avoidance unit; the flight control unit is used to control the UAV to fly along the set detection route, and the automatic obstacle avoidance unit is used to control the UAV to detect obstacles in the flight path in real time during the flight and adjust the flight path of the UAV to achieve real-time obstacle avoidance; the detection and diagnosis module includes an image preprocessing unit, a real-time detection unit and an early warning unit; the image preprocessing unit is used to preprocess and store the wind turbine tower blade images collected in each flight mission of the UAV; the real-time detection unit is used to detect the wind turbine tower blade conditions when the UAV performs the flight mission; the early warning unit is used to complete the early warning of abnormal conditions of wind turbine tower blades in the future according to the analysis of the detection results of multiple real-time detection units in the historical records; The image preprocessing unit includes an image stitching subunit for correcting and stitching the continuously photographed wind turbine tower blade images; the workflow of the image stitching subunit includes: S1521: For two stitched images, calculate the gradient amplitude of the pixels of each image at each level: ; ;in, The first stitched image is The gradient magnitude of the level, Indicates the horizontal direction, Indicates the vertical direction, The first stitched image is The lateral gradient of the level, The first stitched image is The longitudinal directional gradient of the hierarchy; For the second stitched image in The gradient magnitude of the level, For the second stitched image, The lateral gradient of the level, For the second stitched image, The vertical gradient of the level; ; S1522: Calculate the initial weight of each image at each level: ; ;in, The first stitched image is The initial weights of the layers, For the second stitched image, The initial weight of the layer; S1523: Calculate the fusion weight of each image at each level: ； ； in, The first stitched image is The fusion weight of the layer, For the second stitched image, The fusion weight of the layer, is the weight adjustment factor, satisfying: ;in, is the adjustment coefficient, the value range is .

2. The wind turbine tower blade detection system based on drone according to claim 1 is characterized in that: The image preprocessing unit also includes an image acquisition subunit, an image stitching subunit and an image storage subunit; the image acquisition subunit is used to acquire the wind turbine tower blade images continuously taken by the image acquisition module, and the image stitching subunit is used to correct and stitch the continuously taken wind turbine tower blade images; the image storage subunit is used to complete the storage of the stitched blade images.

3. The wind turbine tower blade detection system based on drone according to claim 1, characterized in that: The workflow of the image stitching subunit is as follows: S11: Acquire continuously photographed wind turbine tower blade images; S12: Correct the brightness, contrast and color of all leaf images; S13: for all corrected leaf images, identify and extract key feature points of each leaf image, and generate a feature descriptor for each key feature point; the feature descriptor is obtained based on the gradient information of the area around the corresponding feature point; S14: matching the key feature points extracted from each image according to the distance measurement between the feature descriptors of the key feature points in each image; S15: performing geometric transformation and alignment operations on each image to stitch them together according to the matched key feature points, and performing fusion processing on the stitching area of ​​the image; S16: Generate and output the stitched leaf image.

4. The wind turbine tower blade detection system based on drone according to claim 1, characterized in that: The real-time detection unit completes the detection of the blade condition by inputting the spliced ​​blade image into a pre-trained blade defect recognition model; the output of the blade defect recognition model includes various defect types and the defect severity corresponding to the defect type.

5. The wind turbine tower blade detection system based on drone according to claim 1, characterized in that: The real-time detection unit sets an alarm threshold for each defect type. During the detection of the blade condition, when the defect severity of a certain defect type exceeds the set alarm threshold, the real-time detection unit will automatically trigger an alarm and send the alarm information to the staff; the alarm information includes the defect type, the defect severity and the specific location of the defect obtained based on the blade image.

6. The wind turbine tower blade detection system based on drone according to claim 1, characterized in that: The early warning unit completes the prediction alarm for the possible defect types of the blade by analyzing the defect severity corresponding to each defect type in the comprehensive historical detection results, and sends a warning message to the staff when it is predicted that abnormal defects may occur in the blade in the future.

Images