Unmanned aerial vehicle-based fan defect diagnosis method and system, and electronic device

Abstract

The application discloses a fan defect diagnosis method and system based on a UAV and electronic equipment, and relates to the technical field of fans. The method comprises the following steps: a UAV is used to inspect the fan, a fan component inspection image set is obtained, and a fan defect is identified according to the fan component inspection image set; when the fan defect comprises a fan blade defect, at least one target blade with a defect in the fan is positioned according to the fan blade defect, and a blade image subset corresponding to each target blade is obtained from the fan component inspection image set; for each target blade, the images in the blade image subset are spliced according to the position and height when the UAV captures the images in the blade image subset, a rough spliced image is obtained, and a blade panorama of the target blade is obtained according to the rough spliced image; and a fan defect diagnosis result is generated according to the blade panorama and the fan defect. Therefore, the accurate fan defect diagnosis result can be automatically generated.

Description

Technical Field

[0001] This invention relates to the field of wind turbine technology, and in particular to a method, system and electronic equipment for wind turbine defect diagnosis based on unmanned aerial vehicles (UAVs). Background Technology

[0002] In the field of wind power generation, regular inspections of key components such as wind turbine blades and towers to identify defects such as cracks, lightning strike damage, and contamination are crucial for ensuring equipment safety and power generation efficiency.

[0003] The proposed solution employs a "drone data acquisition + manual interpretation" model. Specifically, this approach first uses drones equipped with high-definition visible light cameras and infrared thermal imagers to automatically or semi-automatically fly over wind turbines along pre-set routes, collecting high-definition images or video data of the turbine surface. After the flight mission, inspection personnel bring the massive amount of image data stored in the drone's storage device back to a ground workstation. Then, professional analysis engineers manually browse, filter, and interpret thousands of images one by one on a computer. Engineers rely on their professional knowledge and experience to visually identify potential defects in the images, such as cracks on the blades or leading-edge corrosion, classifying, labeling, and recording them to ultimately generate an inspection report.

[0004] However, the solutions in the relevant technologies rely heavily on manual defect diagnosis, resulting in low inspection efficiency and high costs. Furthermore, the subjective nature of manual interpretation makes it difficult to guarantee the accuracy and consistency of diagnostic results. In addition, the solution is essentially an "offline" analysis, which cannot achieve real-time defect identification and early warning while the drone is conducting inspections. This leads to the loss of the best opportunity to immediately conduct re-inspection or take emergency measures when serious defects are discovered, and the degree of intelligence and automation is insufficient. Summary of the Invention

[0005] This invention aims to at least partially solve one of the technical problems in related technologies. Therefore, the first objective of this invention is to propose a method for diagnosing wind turbine defects based on unmanned aerial vehicles (UAVs), so as to automatically generate accurate wind turbine defect diagnosis results.

[0006] The second objective of this invention is to provide an electronic device.

[0007] The third objective of this invention is to propose a wind turbine defect diagnosis system based on unmanned aerial vehicles (UAVs).

[0008] To achieve the above objectives, a first aspect of the present invention proposes a method for diagnosing wind turbine defects based on unmanned aerial vehicles (UAVs). The method includes: inspecting a wind turbine using a UAV to obtain a set of inspection images of wind turbine components, and identifying wind turbine defects based on the set of inspection images; when the wind turbine defects include wind turbine blade defects, locating at least one target blade in the wind turbine with defects based on the wind turbine blade defects, and obtaining a subset of blade images corresponding to each target blade from the set of inspection images of wind turbine components; for each target blade, stitching the images in the subset of blade images according to the position and height when the UAV captured the images, obtaining a coarse stitched image, and obtaining a panoramic view of the target blade based on the coarse stitched image; and generating a wind turbine defect diagnosis result based on the panoramic view of the blade and the wind turbine defects.

[0009] In addition, the wind turbine defect diagnosis method based on UAV according to embodiments of the present invention may also have the following additional technical features:

[0010] According to one embodiment of the present invention, when the wind turbine defect does not include the wind turbine blade defect, the method further includes: generating a wind turbine defect diagnosis result based on the wind turbine defect.

[0011] According to one embodiment of the present invention, the step of stitching the leaf images in the leaf image subset based on the position and height of the images captured by the UAV to obtain a coarse stitched image includes: obtaining a reference image and a stitched image based on the leaf image subset, wherein the reference image is a leaf root image or a leaf tip image in the leaf image subset, and the stitched image is other images in the leaf image subset besides the reference image; obtaining a first actual distance in the horizontal direction between the leaf region in the reference image and the leaf region in the stitched image based on the position of the UAV when capturing the reference image and the position when capturing the stitched image; obtaining a second actual distance in the vertical direction between the leaf region in the reference image and the leaf region in the stitched image based on the height of the UAV when capturing the reference image and the height when capturing the stitched image; converting the first actual distance into a first pixel distance and the second actual distance into a second pixel distance; stitching the images in the leaf image subset based on the first pixel distance and the second pixel distance to obtain the coarse stitched image.

[0012] According to an embodiment of the present invention, obtaining a panoramic image of the target leaf from the coarsely stitched image includes: obtaining a search radius based on the size of the reference image; for each stitched image, obtaining a search range based on the position of the center point of the leaf region in the stitched image in the coarsely stitched image and the search radius, obtaining multiple search positions based on the search range and a preset search step size, obtaining multiple scaling factors based on a preset scaling range and a preset scaling step size, and obtaining an optimal scaling factor and an optimal position of the leaf region in the stitched image based on the multiple search positions and the multiple scaling factors; and obtaining the panoramic image of the leaf based on the optimal scaling factor and the optimal position.

[0013] According to an embodiment of the present invention, before determining the search range for each stitched image based on the position of the center point of the blade region in the stitched image in the coarse stitched image and the search radius, the method further includes: moving the wind turbine region in the reference image to the image stitching region, so that the center point of the wind turbine region in the reference image is located at a preset position on the image stitching region; obtaining the optimal scaling factor and the optimal position of the blade region in the stitched image on the image stitching region based on multiple search positions and multiple scaling factors includes: obtaining multiple search parameter combinations based on multiple search positions and multiple scaling factors; for each search parameter combination... The placement position of the blade region center point in the blade image on the image stitching area is determined based on the search position corresponding to the search parameter combination, the position of the wind turbine region center point in the reference image in the coarse stitching image, and the preset position. The blade region in the blade image is processed according to the scaling factor corresponding to the search parameter combination, and the processed blade region is moved so that the center point of the blade region is located at the placement position. The intersection-union ratio (IUU) between the blade region and the existing blade regions in the image stitching area is obtained. When the maximum IUU is greater than zero, the placement position and scaling factor corresponding to the maximum IUU are taken as the optimal position and the optimal scaling factor.

[0014] According to one embodiment of the present invention, when the maximum crossover ratio is less than or equal to zero, the method further includes: obtaining the optimal position based on the position of the center point of the blade region in the coarse stitched image.

[0015] According to one embodiment of the present invention, generating a wind turbine defect diagnosis result based on the blade panoramic image and the wind turbine defect includes: obtaining the average scaling factor based on all the optimal scaling factors, and obtaining a conversion relationship based on the average scaling factor and the camera parameters of the UAV; obtaining the pixel distance between the wind turbine blade defect and the blade root in the blade panoramic image, and converting the pixel distance into an actual physical distance according to the conversion relationship; and generating the wind turbine defect diagnosis result based on the blade panoramic image, the wind turbine defect, and the actual physical distance.

[0016] According to one embodiment of the present invention, the step of identifying wind turbine defects based on the wind turbine component inspection image set includes: inputting images from the wind turbine component inspection image set into a pre-trained wind turbine defect identification model to obtain the wind turbine defects.

[0017] To achieve the above objectives, a second aspect of the present invention provides an electronic device including a memory, a processor, and a computer program stored in the memory and running on the processor. When the computer program is executed by the processor, it implements the above-described method for diagnosing wind turbine defects based on unmanned aerial vehicles (UAVs).

[0018] To achieve the above objectives, a third aspect of the present invention provides a wind turbine defect diagnosis system based on unmanned aerial vehicles (UAVs), including the aforementioned electronic equipment.

[0019] According to embodiments of the present invention, a method, system, and electronic equipment for wind turbine defect diagnosis based on unmanned aerial vehicles (UAVs) are used to inspect wind turbines, obtaining a set of inspection images of wind turbine components. Wind turbine defects are then identified based on these images. When the wind turbine defect includes blade defects, at least one target blade with a defect is located based on the blade defect. A subset of blade images corresponding to each target blade is obtained from the set of inspection images. For each target blade, the images in the subset are stitched together based on the position and height of the images captured by the UAV, resulting in a coarsely stitched image. A panoramic view of the target blade is then obtained from the coarsely stitched image. Finally, a wind turbine defect diagnosis result is generated based on the panoramic image and the wind turbine defect. This method enables the automatic generation of accurate wind turbine defect diagnosis results, offering high efficiency, low cost, and high accuracy, consistency, and real-time performance.

[0020] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0021] Figure 1 This is a flowchart of a wind turbine defect diagnosis method based on unmanned aerial vehicles (UAVs) according to an embodiment of the present invention;

[0022] Figure 2 This is a flowchart of a wind turbine defect diagnosis method based on a drone according to an embodiment of the present invention;

[0023] Figure 3 This is a flowchart of another embodiment of the wind turbine defect diagnosis method based on unmanned aerial vehicles (UAVs) of the present invention;

[0024] Figure 4 This is a flowchart of another embodiment of the wind turbine defect diagnosis method based on unmanned aerial vehicles (UAVs) of the present invention;

[0025] Figure 5 This is a flowchart of another embodiment of the wind turbine defect diagnosis method based on unmanned aerial vehicles (UAVs) of the present invention;

[0026] Figure 6 This is a panoramic view of the blades of an example of the present invention;

[0027] Figure 7 This is a flowchart of another embodiment of the wind turbine defect diagnosis method based on unmanned aerial vehicles (UAVs) of the present invention;

[0028] Figure 8 This is a flowchart of another embodiment of the wind turbine defect diagnosis method based on unmanned aerial vehicles (UAVs) of the present invention;

[0029] Figure 9 This is a structural block diagram of an electronic device according to an embodiment of the present invention;

[0030] Figure 10 This is a structural block diagram of a wind turbine defect diagnosis system based on unmanned aerial vehicles (UAVs) according to an embodiment of the present invention. Detailed Implementation

[0031] The following description, with reference to the accompanying drawings, outlines a method, system, and electronic device for diagnosing wind turbine defects based on unmanned aerial vehicles (UAVs). Throughout the description, the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions. The embodiments described with reference to the accompanying drawings are exemplary and should not be construed as limiting the invention.

[0032] Figure 1 This is a flowchart of a wind turbine defect diagnosis method based on unmanned aerial vehicles (UAVs) according to an embodiment of the present invention.

[0033] like Figure 1 As shown, the wind turbine defect diagnosis method based on UAVs includes:

[0034] S11. Using drones to inspect wind turbines, a set of inspection images of wind turbine components is obtained, and wind turbine defects are identified based on the set of inspection images of wind turbine components.

[0035] S12, when the wind turbine defect includes the wind turbine blade defect, locate at least one target blade in the wind turbine with a defect based on the wind turbine blade defect, and obtain a subset of blade images corresponding to each target blade from the wind turbine component inspection image set.

[0036] S13. For each target blade, the images in the blade image subset are stitched together based on the position and height of the images captured by the drone to obtain a coarse stitched image. A panoramic view of the target blade is then obtained based on the coarse stitched image.

[0037] S14 generates wind turbine defect diagnosis results based on the panoramic view of the blades and wind turbine defects.

[0038] The process of identifying wind turbine defects based on a set of wind turbine component inspection images can include: inputting images from the set of wind turbine component inspection images into a pre-trained wind turbine defect identification model to obtain the wind turbine defects.

[0039] Specifically, a set of inspection images of wind turbine components is first obtained, which is a set of images collected using a drone. Since the inspection of wind turbine components is carried out using a drone, each image taken by the drone only needs to contain a part of the wind turbine component, thus allowing the drone to get close to the wind turbine, thereby improving the accuracy of subsequent wind turbine defect diagnosis.

[0040] As an example, if the drone flies to the initial cruise position and the battery level is within a preset range, it can acquire an image of the top of the wind turbine, where the initial cruise position includes the wind turbine's coordinates and safe altitude. Based on the image of the top of the wind turbine, the drone is controlled to fly to the center point of the wind turbine blades and acquire an image of the wind turbine's swept surface. Based on the image of the swept surface, the actual error distance of the swept surface is determined. If the actual error distance of the swept surface meets the preset conditions of the swept surface, the reference flight direction of the wind turbine components is determined. Based on the component's safe distance, the overlap rate of the component inspection images, and the number of component inspection shots, the set of sequence of single-step movement distances of the components and the total movement distance of the components are determined. Images are acquired at the component's reference flight direction, the set of sequence of single-step movement distances of the components, and the total movement distance of the components to obtain a set of wind turbine component inspection images, including blade and tower inspection data.

[0041] This wind turbine component inspection image set includes images of all wind turbine components. This image set can then be input into the wind turbine defect identification model to identify wind turbine defects.

[0042] To obtain the above wind turbine defect identification model, see [link / reference]. Figure 2 First, it is necessary to receive the second wind turbine component inspection image set. The method for obtaining the second wind turbine component inspection image set can be found in the above wind turbine component inspection image set.

[0043] After obtaining the inspection image set of the second wind turbine component, perform data import and task initialization.

[0044] Specifically, the second wind turbine component inspection image set and its corresponding metadata are imported into the data annotation tool. The metadata includes the latitude and longitude, altitude, focal length, timestamp, etc., corresponding to each image in the second wind turbine component inspection image set. The latitude and longitude can be the latitude and longitude of the drone when the image was taken by the drone, and the altitude can be the altitude of the drone when the image was taken by the drone.

[0045] After importing the second wind turbine component inspection image set and corresponding metadata into the data annotation tool, the integrity of the dataset can be verified first, and a unique identifier can be generated for each image based on the inspection task ID to initialize the annotation task.

[0046] Furthermore, a set of structured annotation specifications needs to be predefined in the platform based on the defect type standards of wind turbine components (such as cracks, detachment, lightning damage, oil stains, corrosion, etc.). This specification clarifies the visual feature definitions of various defects, annotation tools (annotation tasks are defined as polygon tasks to facilitate the subsequent generation of rotated target boxes), and attribute labels (such as length and severity level).

[0047] After initializing the annotation task, load the defect annotation specification.

[0048] Specifically, the above annotation specifications will be loaded into the current annotation task.

[0049] Furthermore, the first round of manual annotation and review was carried out.

[0050] Specifically, the annotators identify and delineate defects in the images according to the specifications and add corresponding labels. Subsequently, another reviewer examines and corrects the annotation results.

[0051] Furthermore, determine whether the quality sampling inspection meets the standards.

[0052] Specifically, based on the previously labeled images, a certain proportion of labeled images are randomly selected and submitted to senior experts for quality inspection. Labeled samples that do not meet the specifications are returned and corrected until the overall labeling accuracy of this batch of data reaches a preset threshold (e.g., ≥98%).

[0053] Perform data augmentation on the labeled images of wind turbine components that have passed quality inspection.

[0054] Specifically, offline data augmentation techniques are applied to generate augmented samples in batches, including random rotation, horizontal / vertical flipping, brightness and contrast adjustment, scaling, and adding random noise, to expand the data scale and improve the model's generalization ability. Finally, the augmented complete dataset is divided into training, validation, and test sets according to a preset ratio (e.g., 8:1:1), and labeled datasets, namely the training, validation, and test sets, are output for subsequent model training.

[0055] For details on the training of the above model, please refer to [link / reference]. Figure 3 The example shown.

[0056] A1, a polygonal rotating rectangle annotation.

[0057] Specifically, after obtaining the above-mentioned labeled images, since the labeling content includes wind turbine defects, that is, for each wind turbine component image, if there is a wind turbine defect in the image, the wind turbine defect in the image has been outlined by the labeler using polygons. Therefore, for each defect outline, the minimum area of ​​the circumscribed rectangle of the defect outline can be obtained first.

[0058] The minimum area circumscribed rectangle of revolution mentioned above can be obtained using the following formula:

[0059] ,

[0060] The constraint condition for this expression is:

[0061] ,

[0062] in, It is all polygons that can be enclosed The set of rotating rectangles, where P is a polygonal region of wind turbine defects consisting of n vertices, i.e., the region of the aforementioned wind turbine defects, where the number of n is determined by the shape of the defect profile. , Let be the i-th vertex of the defect profile of the wind turbine defect. "Being restricted to" means Meaning each circumscribed rectangle All must include the defective area P of the wind turbine. This means obtaining the circumscribed rectangle. area, This means finding the circumscribed rectangle with the minimum area. .

[0063] In other words, the minimum area circumscribed rectangle of the aforementioned defect contour is a rotated rectangle. , is the rectangle with the smallest area among all rectangles that include the defect profile, that is, the smallest bounding rectangle of the defect profile.

[0064] After obtaining the minimum bounding rectangle of the defect profile, the rectangle parameters of the minimum bounding rectangle can also be obtained, including the x-coordinate of the center point. , center point ordinate ,Width ,high Angles relative to preset straight lines (hereinafter referred to as rotation angle).

[0065] Using the above method, we can obtain the smallest bounding rectangle that can most tightly enclose the irregular defect shape. The smallest bounding rectangle is defined by its center point. ,Width ,high and rotation angle Definition. This method ensures that the minimum bounding rectangle adaptively fits along the principal axis of the defect. Compared to converting from or directly using a horizontal box, it can more thoroughly eliminate interference from irrelevant background pixels, providing cleaner feature learning samples for subsequent model training.

[0066] A2, Construct the YOLO-OBB rotation detection model.

[0067] Specifically, the rotation detection framework YOLO-OBB was selected. The framework was specifically improved to address the characteristics of the wind turbine data: a rotation region proposal network was added after the feature extraction network to generate directional candidate regions; in the detection head, the bounding box regression parameters of the traditional YOLO-OBB rotation detection framework were expanded to five parameters (center point x-coordinate, center point y-coordinate, width, height, and angle) regression. Simultaneously, based on the statistical characteristics of the aspect ratio of wind turbine defects, a set of rotation anchor frame proportions suitable for slender targets was preset.

[0068] Alternatively, the above rotation detection model can also be implemented in other possible ways, which are not limited in this application.

[0069] A3, Design a composite loss function.

[0070] This function includes: 1) an angle-sensitive regression loss for angle prediction, ensuring that the model can accurately learn the orientation of defects; and 2) an improved rotation intersection-over-union loss, which directly optimizes the overlap between the predicted bounding box and the actual rotated bounding box.

[0071] Alternatively, the above loss function may also be implemented in other possible ways, which are not limited in this application.

[0072] A4, phased training: horizontal box warm-up → rotating box fine-tuning.

[0073] Specifically, the training employs a phased strategy: first, pre-training is performed on horizontal bounding box labeled data, followed by fine-tuning on rotated bounding box datasets. Through iterative optimization, the model can not only identify defect categories but also accurately perceive their spatial orientation and shape.

[0074] A5, determine whether the average accuracy of the rotating frame meets the standard.

[0075] Based on the model after the above training iterations, the average accuracy of the rotated frame was used as the core evaluation index on an independent test set to verify the model's detection efficiency for slender, non-directional defects.

[0076] A6, lightweight model processing.

[0077] After successful verification, the fully trained rotation detection model is combined with lightweight technology to compress the model size, resulting in the final wind turbine defect identification model.

[0078] A7 is packaged for rotational defect identification services.

[0079] A8, Output fan defect identification model.

[0080] Furthermore, the aforementioned wind turbine defect identification model can be set up to output the x-coordinate of the center point, y-coordinate of the center point, width, height, and rotation angle of the minimum bounding rectangle of the wind turbine defect after the defect is identified.

[0081] When the wind turbine defects do not include wind turbine blade defects, the above-mentioned UAV-based wind turbine defect diagnosis method also includes: generating wind turbine defect diagnosis results based on wind turbine defects.

[0082] In other words, when there are no defects in the turbine blades, the turbine defect diagnosis results can be generated directly based on the turbine defects.

[0083] When wind turbine defects include wind turbine blade defects, in order to obtain wind turbine defect diagnosis results, it is necessary to first identify the blade that has the above-mentioned wind turbine blade defect, identify the blade as the target blade, and then find the blade image obtained by taking pictures of the target blade from the above-mentioned wind turbine component inspection image set to obtain the blade image subset corresponding to the target blade.

[0084] Since the images in the above wind turbine component inspection image set are all partial images, for example, many images may need to be stitched together to obtain a panoramic image of a wind turbine blade, in order to generate wind turbine defect diagnosis results, it is necessary to stitch together the images in each blade image subset to obtain a panoramic image of the blade.

[0085] To stitch together the images in the leaf image subset, the coordinates and altitude of the drone were first obtained when the drone took each image in the leaf image subset. The drone's coordinates were then used as its position, which could be obtained through a positioning device such as GPS on the drone.

[0086] It should be noted that the coarse stitched image corresponding to each target blade is obtained from a subset of the blade images corresponding to that target blade.

[0087] Furthermore, the relationship between any two images in the leaf image subset can be obtained in the horizontal and vertical directions based on the position and altitude of the UAV corresponding to each image in the leaf image subset. The images in the leaf image subset can then be stitched together using this relationship to obtain a coarse stitched image. Finally, a panoramic view of the target leaf can be obtained based on the coarse stitched image.

[0088] It is clear that the number of coarsely stitched images and the panoramic images of the leaves are consistent with the number of target leaves.

[0089] This allows for the automatic generation of accurate wind turbine defect diagnosis results, which is highly efficient, low-cost, and offers high accuracy, consistency, and real-time performance.

[0090] In some embodiments of the present invention, the leaf images are stitched together based on the position and height of the images captured by the UAV in a subset of leaf images to obtain a coarse stitched image. This process includes: obtaining a reference image and a stitched image based on the subset of leaf images, wherein the reference image is a leaf root image or a leaf tip image from the subset of leaf images, and the stitched image consists of other images from the subset of leaf images besides the reference image; obtaining a first actual distance in the horizontal direction between the leaf region in the reference image and the leaf region in the stitched image based on the position of the UAV when capturing the reference image and the position when capturing the stitched image; obtaining a second actual distance in the vertical direction between the leaf region in the reference image and the leaf region in the stitched image based on the height of the UAV when capturing the reference image and the height when capturing the stitched image; converting the first actual distance into a first pixel distance and the second actual distance into a second pixel distance; and stitching the images in the subset of leaf images together based on the first pixel distance and the second pixel distance to obtain a coarse stitched image.

[0091] See Figure 4 B1, Data loading pre-parameter initialization.

[0092] Specifically, after identifying wind turbine defects using the aforementioned wind turbine defect identification model to process the wind turbine component inspection image set, the first step is to initialize the data loading pre-parameters by loading the wind turbine component inspection image set and its corresponding metadata into the processing center. Simultaneously, pre-configured camera and stitching parameters are loaded. Camera parameters include the vertical distance from the camera to the wind turbine blades and the physical width of the sensor; stitching parameters include the search translation ratio and zoom range.

[0093] B2, blade region segmentation.

[0094] Specifically, a pre-trained deep learning model for wind turbine blade segmentation is used to perform batch inference on images in a subset of blade images to obtain the blade regions in the images in the subset of blade images. A binary mask can be generated for each image in the subset of blade images to cover the background parts of the image, thereby eliminating the interference of complex backgrounds such as the sky, tower, and ground on subsequent feature matching, and ensuring that the stitching result focuses on the blade body.

[0095] B3, coarse positioning calculation.

[0096] Specifically, after obtaining the leaf region from each image in the leaf image subset, a leaf root image or a leaf tip image is selected from the leaf image subset as a reference image. Since the coarse stitched image corresponding to each target leaf is obtained from the leaf image subset corresponding to that target leaf, a reference image needs to be selected from each leaf image subset.

[0097] After determining the reference image in the blade image subset, the remaining images in the wind turbine component inspection image set, excluding the reference image, are used as the stitched images.

[0098] Furthermore, for each stitched image, a first actual distance in the horizontal direction between the blade region in the stitched image and the blade region in the reference image is obtained based on the corresponding drone position in the stitched image and the corresponding drone position in the reference image. Moreover, a second actual distance in the vertical direction between the blade region in the stitched image and the blade region in the reference image is obtained based on the corresponding drone height in the stitched image and the corresponding drone height in the reference image.

[0099] B4, Offset Calculation and Canvas Determination.

[0100] Specifically, the ground sampling distance of the image is obtained according to the following formula:

[0101] ,

[0102] in, is the ground sampling distance, in meters per pixel, and f is the focal length of the camera mounted on the UAV that captured the images in the subset of leaf images mentioned above. Let D be the number of pixels in the width direction of the image captured by the camera, and D be the vertical distance from the camera to the wind turbine blades. This refers to the physical width of the aforementioned sensor.

[0103] Furthermore, after obtaining the first actual distance, the second actual distance, and the ground sampling distance, the first actual distance is converted into a first pixel distance, and the second actual distance is converted into a second pixel distance according to the following formula:

[0104] ,

[0105] ,

[0106] in, The distance is the first pixel. The distance is the second pixel. The first actual distance, This is the second actual distance. The direction factor is determined based on the flight direction and blade orientation, and its value is... or Specifically, the value is -1 when the drone flies from the tip of the leaf to the root, and 1 when it flies from the tip to the root. The Y-axis is inverted to adapt to the image coordinate system (the Y-axis of the above-mentioned preset image coordinate system is positive downwards).

[0107] After calculating the first pixel distance and second pixel distance for each stitched image, the minimum and maximum values ​​of the first pixel distance, the minimum and maximum values ​​of the second pixel distance, and then the width and height of the stitched canvas can be obtained according to the following formula:

[0108] ,

[0109] ,

[0110] in, To splice the canvas width, To adjust the canvas height, The maximum value of the distance to the first pixel. The minimum distance to the first pixel. The maximum value of the second pixel distance. The minimum distance to the second pixel. The number of pixels in the width direction of the reference image. This refers to the number of pixels in the height direction of the reference image.

[0111] It should be noted that the above-mentioned splicing canvas is a virtual canvas used to splice images from the above leaf image subset on the canvas to obtain a coarse spliced ​​image.

[0112] B5, canvas stitching and result output.

[0113] Specifically, after obtaining the splicing canvas, the leaf region in the aforementioned reference image is extracted from the reference image, and the extracted leaf region is transferred to the splicing canvas. The specific position of the leaf region in the reference image on the splicing canvas is determined by the sign of the first pixel distance and the second pixel distance corresponding to each splicing image. For example, if the sign of the first pixel distance and the second pixel distance corresponding to each splicing image determines that each splicing image is in the upper right of the reference image, then the reference image can be placed in the lower left of the splicing canvas.

[0114] Furthermore, for each stitched image, the leaf region in the stitched image is extracted (for example, by using the mask mentioned above), and the position of the leaf region in the stitched image in the stitched canvas is determined based on the first pixel distance, the second pixel distance corresponding to the stitched image and the position of the leaf region in the reference image in the stitched canvas. Then, each leaf region is moved to its corresponding position in the stitched canvas.

[0115] It should be noted that when moving a leaf area to the splicing canvas, if two leaf areas overlap, the pixel value of the overlapping area is taken from the pixel value of the leaf area that was moved to the splicing canvas first.

[0116] This allows for coarse splicing of multiple blade regions.

[0117] In some embodiments of the present invention, obtaining a panoramic view of the target leaf from a coarsely stitched image includes: obtaining a search radius based on the size of a reference image; for each stitched image, obtaining a search range based on the position of the center point of the leaf region in the stitched image in the coarsely stitched image and the search radius, obtaining multiple search positions based on the search range and a preset search step size, obtaining multiple scaling factors based on a preset scaling range and a preset scaling step size, and obtaining the optimal scaling factor and the optimal position of the leaf region in the stitched image based on the multiple search positions and the multiple scaling factors; and obtaining a panoramic view of the leaf based on the optimal scaling factor and the optimal position.

[0118] Specifically, for each stitched image, before determining the search range based on the position and search radius of the blade region center point in the coarse stitched image, the method further includes: moving the wind turbine region in the reference image to the image stitching region so that the center point of the wind turbine region in the reference image is located at a preset position in the image stitching region; obtaining the optimal scaling factor and optimal position of the blade region in the stitched image in the image stitching region based on multiple search positions and multiple scaling factors, including: obtaining multiple search parameter combinations based on multiple search positions and multiple scaling factors; for each search parameter combination, determining the placement position of the blade region center point in the blade image in the image stitching region based on the search position corresponding to the search parameter combination, the position of the wind turbine region center point in the reference image in the coarse stitched image, and the preset position; processing the blade region in the blade image based on the scaling factor corresponding to the search parameter combination; moving the processed blade region so that the center point of the blade region is located at the placement position; and obtaining the intersection-union ratio (IUGR) between the blade region and existing blade regions in the image stitching region; when the maximum IUGR is greater than zero, using the placement position and scaling factor corresponding to the maximum IUGR as the optimal position and optimal scaling factor.

[0119] See Figure 5 C1, Data preparation and parameter initialization.

[0120] Specifically, the coarse stitched image obtained in the above stages, the corresponding leaf segmentation binary mask, and the initial pixel offset are loaded. Simultaneously, preset fine stitching search parameters, including the search translation ratio, are loaded. Preset zoom range Preset search step size (pixels) and preset scaling steps .

[0121] Using the reference image as a baseline, the size of the image stitching region used for fine-grained searching is determined based on the minimum and maximum values ​​of the distance between the first and second pixels. ,in, The width of the image stitching area, The height of the image stitching area.

[0122] C2, the reference image is placed in the parameter record.

[0123] Specifically, the blade area of ​​the aforementioned reference image is placed in the image stitching area, and the center point of the wind turbine area in the reference image needs to be located at a preset position in the image stitching area. This preset position can be set so that the center point of the wind turbine area in the reference image is located at the center point of the image stitching area.

[0124] Simultaneously, the search parameter combination corresponding to the reference image is recorded. This search parameter combination includes the difference between the position of the wind turbine region in the reference image and the preset position in the coarse stitched image. For example, it can be the difference between the coordinates of the center point of the wind turbine region in the reference image in the image stitching area and the coordinates in the coarse stitched image. It also includes the scaling factor used by the reference image, specifically 1, that is, the size of the wind turbine region in the reference image in the image stitching area is the same as its size in the stitching canvas.

[0125] For each stitched image, the following steps are performed.

[0126] C3, Search Space Construction.

[0127] Specifically, the search radius is obtained according to the following formula:

[0128] ,

[0129] in, For the search radius, , This represents the search translation ratio.

[0130] After obtaining the search radius, set the x-coordinate of the search position. ,in, Let x be the x-coordinate of the center point of the leaf region in the stitched image within the coarse stitched image, and let x be the preset search step size in the x-coordinate direction. Pixel.

[0131] Set the search position y-coordinate ,in, Let be the ordinate of the center point of the leaf region in the stitched image within the aforementioned coarse stitched image, and let be the preset search step size in the ordinate direction. Pixel.

[0132] scaling factor , The preset scaling range and preset scaling step size are: .

[0133] Furthermore, based on the above exist Select the x-coordinate value of the specific search location, for example... , , According to the above exist Select the y-coordinate value of the specific search location, for example... , , Then, based on the x-coordinate and y-coordinate of the selected search location, the search position is obtained. ).

[0134] And, according to the above exist Select a specific scaling factor s value, for example , , The scaling factor s can be used as the magnification factor for subsequent resampling.

[0135] After obtaining the search location ( After setting the scaling factor s, the search position ( x in ) and the above The difference between them is the difference in the x-axis. Search location ( ) in the above The difference between them is the difference in the ordinate. Difference of x-coordinates Difference between the vertical and horizontal axes As a displacement factor, it forms a three-dimensional parameter search space. It can be seen that the three-dimensional parameter search space is the combination of search parameters corresponding to the stitched image.

[0136] C4, Parallel scaling calculation.

[0137] Specifically, after obtaining the three-dimensional parameter search space, the intersection-union ratio corresponding to each combination of search parameters is obtained.

[0138] Using the j-th search parameter combination Let's take an example to illustrate.

[0139] Specifically, after obtaining the j-th search parameter combination Then, the leaf region in the stitched image is scaled according to the scaling factor. Bilinear interpolation resampling is performed to obtain the transformed blade region. .

[0140] Transformed blade region The placement of the center point within the image stitching area depends on the search location. The position of the wind turbine region in the reference image within the coarsely stitched image is determined by a preset position. For example, assuming the coordinates of the center point of the blade region in the reference image on the coarsely stitched image are the same as its coordinates in the image stitching region, then the transformed blade region... The search position is the location where the center point is placed within the image stitching area. Assuming the coordinates of the center point of the leaf region in the reference image are inconsistent on the coarse stitched image and in the image stitched region, the search position can be adjusted based on the coordinate difference between the center point of the leaf region in the reference image and the coarse stitched image and the image stitched region. The transformed blade region is obtained. The placement of the center point within the image stitching area.

[0141] Furthermore, the transformed blade region The center point is placed at the placement position, and the transformed blade region is calculated. The image stitching area already contains a leaf region. Crossover ratio:

[0142] ,

[0143] in, This is the intersection-union ratio. Therefore, the search position can be obtained. and scaling factor The corresponding crossover / union ratio.

[0144] This calculation process can be performed in parallel using multi-threading to accelerate the search.

[0145] C5, the optimal parameters are determined.

[0146] Specifically, after traversing the entire three-dimensional parameter search space, the maximum cross-union ratio is determined from all the obtained cross-union ratios. If the maximum cross-union ratio is greater than zero, the search parameter combination corresponding to the maximum cross-union ratio is used as the optimal transformation parameter, that is, the displacement factor corresponding to the maximum cross-union ratio is used as the optimal displacement factor. Then, the optimal position on the image stitching area is obtained according to the optimal displacement factor, and the scaling factor corresponding to the maximum cross-union ratio is used as the optimal scaling factor.

[0147] When the maximum crossover ratio (CVR) is less than or equal to zero, the UAV-based wind turbine defect diagnosis method also includes: obtaining the optimal position based on the position of the center point of the blade region in the coarse stitched image. For example, assuming that the coordinates of the center point of the blade region in the reference image are consistent with the coordinates in the image stitching region, then for the stitched image, the position of the center point of the blade region in the coarse stitched image is the optimal position. If the coordinates of the center point of the blade region in the reference image are inconsistent with the coordinates in the image stitching region, then the position of the center point of the blade region in the stitched image can be adjusted according to the coordinate difference between the center point of the blade region in the reference image and the image stitching region to obtain the optimal position.

[0148] Furthermore, the optimal scaling factor corresponding to this blade region is set to 1, meaning no scaling is performed.

[0149] C6, Image Transformation and Fusion.

[0150] Specifically, it is necessary to pre-generate a color canvas and a mask canvas, the size and coordinate system of which are consistent with the above image stitching area.

[0151] After obtaining the optimal transformation parameters, the stitched image is resampled using the scaling factor corresponding to these parameters. The optimal position within the stitched image region is then determined based on the displacement factor corresponding to the optimal transformation parameters. The resampled stitched image is moved to the color canvas, and the leaf region within the resampled stitched image is moved to the mask canvas. The center point of the leaf region in the resampled stitched image is located at the optimal position on both the color canvas and the mask canvas. During fusion, pixel values ​​in overlapping areas follow a "first-come, first-served" principle to avoid mixing artifacts.

[0152] Place the reference image on the colored canvas, and place the fan area in the reference image on the mask canvas, with the center point of the fan area in the reference image located at the preset position on both the colored canvas and the mask canvas.

[0153] After obtaining the optimal scaling factor, the size of the corresponding stitched image is adjusted using the optimal scaling factor. After obtaining the optimal position, the stitched image adjusted using the optimal scaling factor is moved to the color canvas, and the leaf area in the stitched image adjusted using the optimal scaling factor is moved to the mask canvas, so that the center point of the leaf area in the stitched image is located at the optimal position on the color canvas and the mask canvas.

[0154] C7, Iterative Optimization and Result Output.

[0155] Specifically, after performing the processing described in C6 on each stitched image, the color canvas is cropped based on the effective area of ​​the mask canvas to remove excess background, generating a precisely stitched panoramic image of the leaf. (See the image for details.) Figure 6 The example shown. Output the panorama, a list of optimized transform parameters for each image, and the corresponding sequence of maximum IOU values.

[0156] In some embodiments of the present invention, generating wind turbine defect diagnosis results based on the blade panoramic image and wind turbine defects includes: obtaining the average scaling factor based on all optimal scaling factors, and obtaining a conversion relationship based on the average scaling factor and the camera parameters of the UAV; obtaining the pixel distance between the wind turbine blade defect and the blade root in the blade panoramic image, and converting the pixel distance into the actual physical distance based on the conversion relationship; and generating wind turbine defect diagnosis results based on the blade panoramic image, wind turbine defects, and the actual physical distance.

[0157] See Figure 7 D1, coordinate mapping relationship construction.

[0158] Specifically, for each blade region in the above panoramic image of the blade, the following affine transformation matrix is ​​obtained based on the optimal transformation parameters:

[0159] ,

[0160] in, Let i be the i-th affine transformation matrix. Let be the optimal scaling factor corresponding to the i-th blade region. This represents the optimal abscissa difference corresponding to the i-th blade region. This represents the optimal ordinate difference corresponding to the i-th blade region.

[0161] D2, Defect Vertex Transformation.

[0162] Specifically, after obtaining the aforementioned affine transformation matrix, for each wind turbine defect, the coordinates of the four vertices of the minimum bounding rectangle in the splicing canvas are obtained. ,in, Let these be the coordinates of the first vertex. The coordinates of the second vertex. The coordinates of the third vertex. The coordinates of the fourth vertex.

[0163] Among them, the above , , , All are 3x3 matrices, with the first row representing the x-coordinate, the second row representing the y-coordinate, and the third row representing a scaling factor fixed at 1, as described above. It is a matrix with three rows and four columns.

[0164] Furthermore, assuming that the center point of the leaf region in the above reference image is at position (0,0) in both the above stitched canvas and the above image stitched region, the global pixel coordinate vertex set can be obtained according to the following formula:

[0165] ,

[0166] in, It is the set of vertices with global pixel coordinates.

[0167] D3, Physical Scale Calibration.

[0168] Specifically, first, the average scaling factor is obtained based on the scaling factors corresponding to the images in all leaf image subsets, and then the conversion relationship is obtained according to the following formula:

[0169] ,

[0170] in, For conversion relationships, This represents the average of the scaling factors.

[0171] D4, Defect location calculation.

[0172] After obtaining the conversion relationship, select a reference point from the above panoramic view of the leaf, for example, it can be set as the root of the leaf.

[0173] After determining the reference point, calculate the pixel distance between the center point of the minimum bounding rectangle and the reference point within the panoramic view of the blade. Then, obtain the actual physical distance between the center point of the minimum bounding rectangle and the reference point according to the following formula:

[0174] ,

[0175] Where L is the actual physical distance between the center point of the minimum bounding rectangle and the reference point. The pixel distance between the center point of the minimum bounding rectangle and the reference point is given.

[0176] This enables the coordinate transformation of the defect location from a local image to a panoramic view of the entire blade.

[0177] D5, whether the defect originates from multiple overlapping images.

[0178] Specifically, by obtaining the overlap of the minimum bounding box of the wind turbine blade defects corresponding to the inspection images of different wind turbine components on the panoramic image, if it is higher than the threshold, it is considered as the same defect.

[0179] D6, calculate the weighted average position.

[0180] Specifically, for the same defect that appears in multiple wind turbine component inspection images due to image overlap in the above-mentioned panoramic image of the blade, the multiple identification results will be automatically merged. That is, the weighted average of the physical location coordinates of the defect or the smallest bounding rectangle of the defect in the above-mentioned panoramic image of the blade (with the identification confidence level as the weight) will be taken as the final result.

[0181] D7, record directly, generate a unique ID, and add to the defect list.

[0182] Specifically, if a defect does not originate from multiple wind turbine component inspection images, an ID for that defect is generated and added to the defect list.

[0183] D8 generates a structured list of curve locations.

[0184] Specifically, a structured list of defect locations is generated, with each record containing: a unique defect ID, type, global pixel coordinates in the panoramic image, physical location (in meters) from the leaf root, direction of extension, estimated size (length, width), and the original image number to which it belongs.

[0185] The above method generates wind turbine defect diagnosis results based on the panoramic view of the blades, wind turbine defects, and actual physical distances. For detailed steps, please refer to [link to relevant documentation]. Figure 8 .

[0186] E1, Information Integration and Structured Storage.

[0187] Specifically, the defect identification results, including defect type, rotation frame size, confidence level, and extension direction, are automatically correlated and aggregated with the precise physical location information of the defects, including the actual distance from the blade root and the global pixel coordinates in the panoramic image. Simultaneously, basic information related to this inspection task, such as turbine identification, inspection timestamps, and environmental parameters, are also included in the management. All information is organized and stored according to a predefined structured data model, forming a complete data record set with a unique defect identifier at its core, laying the data foundation for subsequent visualization and report generation.

[0188] E2, panoramic image defect visualization annotation.

[0189] Specifically, using the panoramic image of the blade generated in the preceding steps as the base image, and based on the calculated global pixel coordinates of the defects, a layered rendering strategy is employed for automatic annotation. Specifically, within the rotating rectangular area corresponding to the defect, a colored semi-transparent overlay layer associated with the defect type is filled to visually represent the shape and category of the defect; at the center point of the defect, a directional icon pointing in the same direction as the defect's extension is overlaid; and a text label containing the defect type and a unique identifier is displayed next to the icon. This layered annotation method generates a comprehensive visual appendix integrating all defect spatial distribution and key attribute information.

[0190] E3, generation of structured diagnostic reports.

[0191] Specifically, based on industry-standard templates, a structured, comprehensive diagnostic report is automatically generated and supports export in multiple formats. The report content is automatically populated and includes the following parts: a report summary containing an overview of the inspection and a summary of defect statistics; detailed attributes of the inspected fan and metadata of the task for this inspection; a detailed list of all defects clearly presented in tabular form, including their identifier, type, precise physical location, size, confidence level, and preliminary handling priority assessment; an embedded panoramic annotation map generated in the aforementioned steps as a core appendix; and preliminary maintenance suggestions automatically generated based on a pre-built expert rule base and combined with the specific type, size, and location information of the defects.

[0192] E4, select the export format.

[0193] Specifically, the report generation method supports exporting the complete content to various standard document formats with one click, including portable document format (PDF) for archiving and distribution, document format (DOCX) for subsequent editing, and structured data format (JSON) for inter-system data exchange, thereby meeting the needs of different application scenarios.

[0194] E5, Report Output and Archiving.

[0195] In summary, the wind turbine defect diagnosis method based on UAVs in this invention utilizes UAVs to inspect wind turbines, obtaining a set of inspection images of wind turbine components, and identifying wind turbine defects based on these images. When the wind turbine defect includes blade defects, at least one target blade with a defect is located based on the blade defect, and a subset of blade images corresponding to each target blade is obtained from the wind turbine component inspection image set. For each target blade, the images in the blade image subset are stitched together based on the position and height of the images captured by the UAV, resulting in a coarse stitched image. A panoramic view of the target blade is then obtained from the coarse stitched image. A wind turbine defect diagnosis result is generated based on the panoramic image and the wind turbine defect. Therefore, accurate wind turbine defect diagnosis results can be automatically generated, with high efficiency, low cost, and high accuracy, consistency, and real-time performance. Moreover, by setting a specific method for obtaining a panoramic image of the blade from a coarsely stitched image, the image registration problem can be transformed into an optimal search problem in a discrete parameter space by using the intersection-union ratio of the blade region segmentation mask as the optimization objective function. A three-dimensional (translation + scaling) search strategy is adopted to effectively compensate for scale changes during UAV shooting. The introduction of a parallel computing mechanism significantly improves the efficiency of large-scale search, ultimately achieving sub-pixel-level blade image alignment accuracy, providing a high-fidelity geometric benchmark for accurate subsequent defect localization.

[0196] Furthermore, the present invention proposes an electronic device.

[0197] Figure 9 This is a structural block diagram of an electronic device according to an embodiment of the present invention.

[0198] like Figure 9 As shown, the electronic device 500 includes a processor 501 and a memory 503. The processor 501 and the memory 503 are connected, for example, via a bus 502. Optionally, the electronic device 500 may also include a transceiver 504. It should be noted that in practical applications, the transceiver 504 is not limited to one type, and the structure of this electronic device 500 does not constitute a limitation on the embodiments of the present invention.

[0199] Processor 501 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this invention. Processor 501 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0200] Bus 502 may include a pathway for transmitting information between the aforementioned components. Bus 502 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 502 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 9 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0201] The memory 503 stores a computer program corresponding to the UAV-based wind turbine defect diagnosis method of the above embodiments of the present invention. This computer program is executed by the processor 501. The processor 501 executes the computer program stored in the memory 503 to implement the content shown in the aforementioned method embodiments.

[0202] in, Figure 9 The electronic device 500 shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0203] The electronic device according to the embodiments of the present invention, by implementing the UAV-based wind turbine defect diagnosis method of the above embodiments, can automatically generate accurate wind turbine defect diagnosis results, which is highly efficient, low-cost, and has high accuracy, consistency and real-time performance.

[0204] Furthermore, this invention proposes a wind turbine defect diagnosis system based on unmanned aerial vehicles (UAVs).

[0205] Figure 10 This is a structural block diagram of a wind turbine defect diagnosis system based on unmanned aerial vehicles (UAVs) according to an embodiment of the present invention.

[0206] like Figure 10 As shown, the wind turbine defect diagnosis system 100 based on UAV includes the aforementioned electronic equipment 500.

[0207] According to the UAV-based wind turbine defect diagnosis system of the present invention, the electronic equipment described above can automatically generate accurate wind turbine defect diagnosis results, which is highly efficient, low-cost, and has high accuracy, consistency and real-time performance.

[0208] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein can be considered as a ordered list of executable instructions for implementing logical functions, which can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0209] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0210] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0211] In the description of this specification, the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and should not be construed as limiting the present invention.

[0212] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0213] In this specification, unless otherwise stated, the terms "installation," "connection," "joining," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise explicitly defined. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0214] In this invention, unless otherwise explicitly specified and limited, "above" or "below" the second feature can mean that the first feature is in direct contact with the second feature, or that the first feature is in indirect contact with the second feature through an intermediate medium. Furthermore, "above," "over," and "on top" of the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.

[0215] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for diagnosing wind turbine defects based on unmanned aerial vehicles (UAVs), characterized in that, The method includes: By using drones to inspect wind turbines, a set of inspection images of wind turbine components is obtained, and wind turbine defects are identified based on the set of inspection images of wind turbine components. When the wind turbine defect includes a wind turbine blade defect, at least one target blade in the wind turbine with a defect is located based on the wind turbine blade defect, and a subset of blade images corresponding to each target blade is obtained from the wind turbine component inspection image set. For each target blade, the images in the subset of blade images are stitched together based on the position and height of the images captured by the UAV to obtain a coarse stitched image, and a panoramic image of the target blade is obtained based on the coarse stitched image. Based on the panoramic view of the blades and the wind turbine defects, a wind turbine defect diagnosis result is generated; The step of stitching the leaf images based on their position and height when captured by the UAV into a subset of leaf images to obtain a coarse stitched image includes: A reference image and a stitched image are obtained from the subset of leaf images, wherein the reference image is a leaf root image or a leaf tip image from the subset of leaf images, and the stitched image is other images from the subset of leaf images other than the reference image; The first actual horizontal distance between the leaf region in the reference image and the leaf region in the stitched image is obtained based on the position of the drone when it takes the reference image and the position when it takes the stitched image. The second actual distance in the vertical direction between the leaf region in the reference image and the leaf region in the stitched image is obtained based on the height at which the UAV captures the reference image and the height at which it captures the stitched image. Convert the first actual distance into a first pixel distance, and convert the second actual distance into a second pixel distance; The images in the subset of leaf images are stitched together based on the first pixel distance and the second pixel distance to obtain the coarse stitched image.

2. The wind turbine defect diagnosis method based on unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, When the wind turbine defect does not include the wind turbine blade defect, the method further includes: The wind turbine defect diagnosis results are generated based on the aforementioned wind turbine defects.

3. The wind turbine defect diagnosis method based on unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, The step of obtaining the panoramic image of the target blade based on the coarsely stitched image includes: The search radius is obtained based on the dimensions of the reference image; For each of the stitched images, a search range is obtained based on the position of the center point of the leaf region in the stitched image in the coarse stitched image and the search radius. Multiple search positions are obtained based on the search range and a preset search step size. Multiple scaling factors are obtained based on a preset scaling range and a preset scaling step size. Finally, the optimal scaling factor of the leaf region in the stitched image and its optimal position on the image stitching area are obtained based on the multiple search positions and the multiple scaling factors. The panoramic view of the blade is obtained based on the optimal scaling factor and the optimal position.

4. The wind turbine defect diagnosis method based on unmanned aerial vehicles (UAVs) according to claim 3, characterized in that, Before determining the search range for each stitched image based on the position of the center point of the leaf region in the stitched image within the coarse stitched image and the search radius, the method further includes: The wind turbine area in the reference image is moved to the image stitching area so that the center point of the wind turbine area in the reference image is located at a preset position on the image stitching area; The step of obtaining the optimal scaling factor and the optimal position of the leaf region in the stitched image based on multiple search positions and multiple scaling factors includes: Multiple combinations of search parameters are obtained based on multiple search locations and multiple scaling factors; For each combination of search parameters, the placement position of the center point of the blade region in the blade image on the image stitching area is determined according to the search position corresponding to the search parameter combination, the position of the center point of the wind turbine region in the reference image in the coarse stitching image, and the preset position. The blade region in the blade image is processed according to the scaling factor corresponding to the search parameter combination, and the processed blade region is moved so that the center point of the blade region is located at the placement position. The intersection-union ratio between the blade region and the existing blade regions in the image stitching area is obtained. When the maximum crossover-union ratio is greater than zero, the placement position and scaling factor corresponding to the maximum crossover-union ratio are taken as the optimal position and the optimal scaling factor.

5. The wind turbine defect diagnosis method based on unmanned aerial vehicles (UAVs) according to claim 4, characterized in that, When the maximum intersection-union ratio is less than or equal to zero, the method further includes: The optimal position is obtained based on the location of the center point of the blade region in the coarse stitched image.

6. The wind turbine defect diagnosis method based on unmanned aerial vehicles (UAVs) according to claim 4, characterized in that, The step of generating wind turbine defect diagnosis results based on the panoramic view of the blades and the wind turbine defects includes: The average scaling factor is obtained based on all the optimal scaling factors, and a conversion relationship is obtained based on the average scaling factor and the camera parameters of the UAV. Obtain the pixel distance between the wind turbine blade defect and the blade root in the panoramic image of the blade, and convert the pixel distance into the actual physical distance according to the conversion relationship; The wind turbine defect diagnosis result is generated based on the panoramic view of the blades, the wind turbine defect, and the actual physical distance.

7. The wind turbine defect diagnosis method based on unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, The step of identifying wind turbine defects based on the wind turbine component inspection image set includes: The images in the wind turbine component inspection image set are input into a pre-trained wind turbine defect recognition model to obtain the wind turbine defects.

8. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and running on the processor, wherein when the computer program is executed by the processor, it implements the unmanned aerial vehicle-based wind turbine defect diagnosis method according to any one of claims 1-7.

9. A wind turbine defect diagnosis system based on unmanned aerial vehicles (UAVs), characterized in that, Including the electronic device according to claim 8.

Images