Wind power generation component defect detection method, system and equipment and storage medium

Abstract

The invention discloses a wind power generation component defect detection method, system and equipment and a storage medium, and the method comprises the steps: carrying out the automatic cruise detection through employing an airborne inspection system, so as to collect an original image of a wind power generation component, carrying out the preprocessing of the collected original image, so as to obtain a preprocessed image, and carrying out the detection of the defect of the wind power generation component; and performing defect detection on the preprocessed image by using a preset defect detection model so as to identify the type of a defect target and locate the position of the defect target in the original image, outputting the type of the defect target, and selecting the position of the defect target in the original image. According to the method, the capability of detecting the small target defects of the wind power generation component is effectively enhanced, and the completeness and accuracy of defect detection of the wind power generation component are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of wind power generation, and in particular relates to a method, system, equipment and storage medium for detecting defects in wind power generation components. Background Art

[0002] As a vital component of renewable energy, the stable operation of wind turbine equipment is crucial for ensuring the reliability and security of energy supply. However, research on wind turbine equipment, particularly deep learning-based wind turbine equipment detection technology, is still in its infancy, and research results are relatively scarce. Wind turbine defect detection faces numerous technical challenges, including scarcity of dataset samples, complex and variable scenarios, and the fact that power equipment defects often manifest as small objects.

[0003] While traditional image detection algorithms have found some application in power equipment inspection, they also have significant limitations. For example, while the improved Canny algorithm achieves relatively good edge detection results, it fails to fully consider adaptability in complex scenarios, resulting in reduced detection performance in these challenging environments. While the Hough algorithm can be used to detect ice thickness on transmission lines, it cannot accurately locate defective transmission lines, making it difficult to meet actual O&M requirements. The Canny edge detection algorithm for ultraviolet images suffers from long detection times, making it inadequate for real-time monitoring. With the continuous advancement of deep learning in computer-based inspection technology, its application in the power industry is becoming increasingly widespread. Deep learning image recognition technologies are primarily categorized into single-stage and two-stage detectors, with common examples including SSD, YOLO, SSP-Net, and Faster R-CNN. Single-stage detection algorithms offer the advantages of high flexibility and real-time performance, but at the expense of some accuracy. Two-stage detection algorithms offer higher accuracy, but are more complex and have poor real-time performance. In actual application scenarios, the complex background of the original image will cause certain interference in the detection algorithm's extraction of target features, thereby reducing detection accuracy. When the VGG16 network model detects deep-level feature images, adding a feature detection layer may cause the gradient vanishing problem, affecting the network's convergence and detection accuracy. The NMS algorithm uses a hard threshold screening strategy. When there is partial occlusion due to equipment defects, it is easy to mistakenly suppress overlapping target windows, resulting in missed detections.

[0004] Therefore, the existing algorithms have many shortcomings in wind power equipment defect detection and are difficult to meet the needs of actual engineering applications. It is urgent to develop an algorithm that can effectively improve the accuracy and robustness of wind power equipment defect detection, which has important practical significance and application value. Summary of the Invention

[0005] The purpose of the present invention is to provide a method, system, equipment and storage medium for defect detection of wind power components. The DenseNet network is innovatively applied to the feature detection link. At the same time, a Soft-PNMS algorithm based on a penalty factor is introduced to deal with the overlapping of target windows during the detection process, thereby solving the technical problems of low accuracy and poor real-time performance of existing wind power component defect detection.

[0006] To solve the above technical problems, the present invention is achieved through the following technical solutions:

[0007] The present invention provides a method for detecting defects in wind power generation components, which comprises:

[0008] Use an airborne inspection system to conduct automated patrol inspections to collect original images of wind turbine components;

[0009] Performing a preprocessing operation on the collected original image to obtain a preprocessed image;

[0010] Performing defect detection on the preprocessed image using a preset defect detection model to identify the type of defect target and locate the position of the defect target in the original image;

[0011] The type of the defective object is output, and the position of the defective object is framed in the original image.

[0012] In one embodiment of the present invention, the preset defect detection model is established in the following manner:

[0013] Obtaining original image samples of wind power generation components;

[0014] Constructing a sample dataset of wind power component defects based on the original image samples;

[0015] Dividing the sample data set into a training set and a test set for model training and model testing respectively;

[0016] Build an initial detection model;

[0017] Performing model training on the initial detection model using the training set to obtain an intermediate detection model;

[0018] The intermediate detection model is tested using the test set, and the model parameters of the intermediate detection model are adjusted according to the test results to obtain the final preset defect detection model.

[0019] In one embodiment of the present invention, the sample data set of wind power component defects includes defect data samples of insulator breakage, flashover, and blade breakage, dirt, and paint peeling.

[0020] In one embodiment of the present invention, obtaining original image samples of wind power generation components includes:

[0021] An airborne inspection system is used to perform automated patrol inspections to collect original image samples of wind turbine components.

[0022] In one embodiment of the present invention, constructing a sample dataset of wind power component defects based on the original image samples includes:

[0023] constructing an initial sample dataset of wind power component defects based on the original image samples;

[0024] The original image samples are subjected to data enhancement processing to expand the size of the initial sample data set, thereby obtaining a high-quality sample data set for constructing the preset defect detection model, wherein the data enhancement processing includes distortion correction simulation, multi-angle rotation and noise injection.

[0025] In one embodiment of the present invention, the constructing of the initial detection model includes:

[0026] Using the SSD target detection algorithm as the framework, build the initial network model;

[0027] The VGG16 convolutional neural network used for feature extraction in the SSD target detection algorithm is replaced with a DenseNet convolutional neural network. The dense connection mechanism of the DenseNet convolutional neural network is used to enhance feature reuse and gradient propagation capabilities, thereby improving the feature extraction accuracy of small target defects in wind turbine components.

[0028] The NMS algorithm used for prediction box screening in the SSD target detection algorithm is replaced with a Soft-PNMS algorithm based on a penalty factor. By dynamically adjusting the confidence of overlapping prediction boxes, the detection stability of densely arranged or partially occluded defects is improved to complete the construction of the initial detection model.

[0029] In one embodiment of the present invention, the dynamically adjusting the confidence of the overlapping prediction boxes to improve the detection stability of densely arranged or partially occluded defects includes:

[0030] When the intersection-over-union ratio between two prediction boxes increases, the Soft-PNMS algorithm nonlinearly enhances the penalty weight of the low-confidence prediction box to improve the detection stability of densely arranged or partially occluded defects.

[0031] Based on the same inventive concept, another embodiment of the present invention further provides a wind power component defect detection system, comprising:

[0032] An image acquisition module, used for acquiring original images of wind power generation components;

[0033] An image processing module is used to perform a preprocessing operation on the collected original image to obtain a preprocessed image;

[0034] A defect detection module is used to perform defect detection on the preprocessed image using a preset defect detection model to identify the type of defect target and locate the position of the defect target in the original image;

[0035] The result output module is used to output the type of the defect target and select the position of the defect target in the original image.

[0036] Based on the same inventive concept, another embodiment of the present invention further provides an electronic device, the electronic device comprising:

[0037] one or more processors;

[0038] A storage device is used to store one or more programs, and when the one or more programs are executed by the one or more processors, the electronic device implements the wind power generation component defect detection method as described in any of the above embodiments.

[0039] Based on the same inventive concept, another embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a computer processor, the computer executes the wind power component defect detection method as described in any of the above embodiments.

[0040] As described above, the present invention provides a method for detecting defects in wind power components, comprising: using an airborne inspection system to perform automated patrol inspection to collect original images of wind power components, performing preprocessing operations on the collected original images to obtain preprocessed images, using a preset defect detection model to perform defect detection on the preprocessed images to identify the type of defective targets and locate the positions of the defective targets in the original images, outputting the type of defective targets, and boxing out the positions of the defective targets in the original images. This method improves the backbone network of the traditional SSD (Single Shot MultiBox Detector) target detection algorithm, replacing the VGG16 convolutional neural network used in the traditional SSD target detection algorithm with a DenseNet convolutional neural network for feature detection. The DenseNet convolutional neural network uses its unique dense connection mechanism to connect features in the channel dimension to achieve feature reuse. This feature reuse strategy can not only capture the features of the input image more comprehensively, but also enables the gradient to be propagated more stably during the training process, thereby slowing down the gradient disappearance phenomenon. At the same time, the DenseNet convolutional neural network reduces the number of network parameters and the complexity of the model through dense connections while ensuring the feature extraction capability, which helps to improve the generalization ability and training efficiency of the model. In addition, during the detection process, when faced with the situation where two defective targets of wind power generation components are partially occluded by each other, a Soft-PNMS algorithm based on a penalty factor is introduced. The Soft-PNMS algorithm is optimized on the basis of the traditional NMS algorithm and adjusts the score of the candidate target window by introducing a penalty factor. During the screening process, the Soft-PNMS algorithm does not directly suppress overlapping windows by a pre-set intersection over union (IOU) threshold, but dynamically adjusts the score of the window according to the IOU value and penalty factor between the windows, so that even if the IOU value between the two windows is large, as long as the defective targets corresponding to them have a high confidence level, they will not be easily suppressed, thereby effectively enhancing the detection capability of small target defects and improving the integrity and accuracy of wind power component defect detection. Of course, any product implementing the present invention does not necessarily need to achieve all the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0042] Figure 1 A schematic flow chart of a method for detecting defects in wind power generation components provided as an exemplary embodiment of the present application.

[0043] Figure 2 A schematic diagram of a network model of a traditional SSD target detection algorithm provided as an exemplary embodiment of the present application.

[0044] Figure 3 A schematic diagram of a network model of a traditional SSD target detection algorithm provided as an exemplary embodiment of the present application.

[0045] Figure 4 A schematic structural diagram of a wind power component defect detection platform provided by an exemplary embodiment of the present application.

[0046] Figure 5 A schematic diagram of the structure of the DenseNet convolutional neural network provided as an exemplary embodiment of the present application.

[0047] Figure 6 A schematic diagram of the structure of the densely connected module of the DenseNet convolutional neural network provided as an exemplary embodiment of the present application.

[0048] Figure 7 A schematic diagram of different function penalty strengths provided by an exemplary embodiment of the present application.

[0049] Figure 8 A schematic structural diagram of a wind power component defect detection system provided by another exemplary embodiment of the present application.

[0050] Figure 9 A schematic structural diagram of an electronic device provided as another exemplary embodiment of the present application. DETAILED DESCRIPTION

[0051] The following describes the embodiments of the present invention through specific examples. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments. The details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other unless they conflict.

[0052] It should be noted that the illustrations provided in the following embodiments are merely schematic illustrations of the basic concept of the present invention. Therefore, the illustrations only show components related to the present invention and are not drawn according to the number, shape, and size of components in actual implementation. In actual implementation, the type, quantity, and proportion of each component may be changed arbitrarily, and the component layout may also be more complex.

[0053] In the following description, numerous details are discussed to provide a more thorough explanation of the embodiments of the present invention. However, it is obvious to those skilled in the art that the embodiments of the present invention can be implemented without these specific details. In other embodiments, publicly known structures and devices are shown in block diagram form rather than in detail to avoid making the embodiments of the present invention difficult to understand.

[0054] In order to solve the technical problem that the existing wind power component defect detection technology is difficult to achieve real-time positioning and accurate identification of small target defects when dealing with complex scenarios, the present invention proposes a wind power component defect detection method, which is applied to the field of wind power generation technology. The method innovatively optimizes the backbone network of the traditional SSD target detection algorithm by replacing the traditional VGG16 convolutional neural network with a DenseNet convolutional neural network with stronger feature extraction capabilities to improve the feature detection performance of small target defects in wind power components in complex scenarios. In addition, the Soft-PNMS algorithm based on the penalty factor is introduced to re-define the number and ratio thresholds of the prediction boxes to reduce the missed detection phenomenon caused by excessive suppression and ensure the retention of more potential defect targets.

[0055] See also Figure 1 As shown, in an exemplary embodiment of the present application, the wind power component defect detection method includes the following steps:

[0056] S100: Automated patrol inspection using an airborne inspection system to capture raw images of wind turbine components;

[0057] S200: performing a preprocessing operation on the collected original image to obtain a preprocessed image;

[0058] S300: performing defect detection on the preprocessed image using a preset defect detection model to identify the type of defect target and locate the position of the defect target in the original image;

[0059] S400: Outputting the type of the defective object and selecting the position of the defective object in the original image.

[0060] The steps in the above-mentioned wind power component defect detection method will be discussed in detail below.

[0061] It should be noted that before officially carrying out the wind power component defect detection task, the preset defect detection model needs to be pre-built and trained.

[0062] In an exemplary embodiment of the present application, the preset defect detection model is established in the following manner:

[0063] S10: obtaining original image samples of wind power generation components;

[0064] S20: constructing a sample dataset of wind power component defects based on the original image samples;

[0065] S30: Divide the sample data set into a training set and a test set for model training and model testing respectively;

[0066] S40: constructing an initial detection model;

[0067] S50: Performing model training on the initial detection model using the training set to obtain an intermediate detection model;

[0068] S60: Performing a model test on the intermediate detection model using the test set, and adjusting the model parameters of the intermediate detection model according to the test results to obtain the final preset defect detection model.

[0069] It should be noted that in this embodiment, the sample dataset of wind turbine component defects includes multiple types of defect data samples, including insulator breakage, insulator flashover, blade damage, blade surface dirt accumulation, and blade paint peeling. Of course, in other embodiments, to further improve the generalization ability and detection accuracy of the model, the sample dataset can be further expanded to include more types of defect data samples. For example, the sample dataset can also include other types of defect data samples, such as tower tilt and cable aging.

[0070] In an exemplary embodiment of the present application, in step S10, the obtaining of original image samples of wind power generation components includes: performing automated patrol inspection using an airborne inspection system to collect original image samples of wind power generation components.

[0071] Specifically, see Figure 2 As shown, in the practice of wind turbine component defect inspection, accurate image acquisition is the foundation for subsequent defect detection and analysis. To achieve efficient and comprehensive image acquisition, in this embodiment, a wind turbine component defect detection platform was built. The platform consists of two core components: an onboard inspection system and a monitoring system.

[0072] The airborne inspection system, serving as the image acquisition front-end, consists of a drone and an intelligent vehicle, which work in tandem to cover every critical component of a wind turbine. The drone, with its flexible flight capabilities, can reach high altitudes above wind turbine towers and capture high-definition images of hard-to-reach areas like blades and the nacelle. The intelligent vehicle, which operates on the ground or on a specific platform within the wind turbine, captures detailed images of the tower, foundation, and other areas. Through the collaboration of the drone and intelligent vehicle, comprehensive and comprehensive raw image samples of every key component of a wind turbine can be acquired.

[0073] The monitoring system serves as the backend of the entire detection platform and is responsible for the motion control of the airborne inspection system and the display of detection data. It can remotely control the movement trajectory and shooting angle of the drone and smart car according to the preset inspection route and tasks to ensure the accuracy and efficiency of image acquisition. At the same time, it can receive and display the original image samples collected by the airborne inspection system in real time, providing data support for subsequent defect detection and analysis. In addition, in order to achieve efficient data communication between the airborne inspection system and the monitoring system, a signal station is set up in this embodiment. The signal station serves as a data transmission hub and is responsible for transmitting the original image samples collected by the airborne inspection system to the monitoring system in real time and stably to ensure the integrity and timeliness of the data.

[0074] In an exemplary embodiment of the present application, in step S20, constructing a sample dataset of defects of wind power components based on the original image samples includes: constructing an initial sample dataset of defects of wind power components based on the original image samples, performing data enhancement processing on the original image samples to expand the scale of the initial sample dataset, thereby obtaining a high-quality sample dataset for constructing the preset defect detection model, wherein the data enhancement processing includes distortion correction simulation, multi-angle rotation and noise injection.

[0075] Specifically, to address issues such as the limited size and insufficient data diversity of the initial sample dataset, in this embodiment, data augmentation processing is performed on the collected raw image samples. This is done to comprehensively simulate various image variations that may occur in actual application scenarios, thereby effectively expanding the size of the initial sample dataset and ultimately constructing a sample dataset suitable for model training. It should be noted that in this embodiment, the data augmentation processing includes distortion correction simulation, multi-angle rotation, and noise injection. In the actual operation of wind turbine components, imaging equipment may experience geometric distortion in captured images due to issues such as installation position deviation, lens quality, or complex environmental factors. To ensure that the model can adapt to such distortion, distortion correction simulation technology is used to deeply process the raw images. By simulating different types of geometric distortion, such as barrel distortion and pincushion distortion, and generating corresponding before-and-after correction image pairs, the sample dataset is enriched with information about image distortion features, helping the model learn more robust feature representations and thereby improving its ability to detect defects in distorted images. Furthermore, considering the varying placement angles of wind turbine components in space and the diverse shooting angles of imaging equipment, in this embodiment, the original image is subjected to multi-angle rotation processing. By rotating the image around a central point at different angles, a series of image samples from different perspectives are generated. This processing method enables the model to learn the characteristic information of wind turbine components at different angles, enhancing the model's adaptability to angular variations of the target object, effectively avoiding missed or false detections caused by angular deviations, and thus improving detection accuracy and stability. Furthermore, the image acquisition process is often subject to interference from various noise sources, such as electronic noise and noise caused by changes in ambient lighting. To simulate these noise conditions, in this embodiment, different types of noise, including Gaussian noise and salt and pepper noise, are injected into the original image. By controlling the noise intensity and distribution, image samples with different noise levels are generated. This allows the model to be exposed to more images with noise interference during subsequent training, thereby improving the model's noise tolerance and ensuring that good detection performance is maintained even when faced with noisy images in practical applications.

[0076] In an exemplary embodiment of the present application, in step S30, constructing the initial detection model includes the following steps:

[0077] S31: Using the SSD target detection algorithm as a framework, build the initial network model;

[0078] S32: replacing the VGG16 convolutional neural network used for feature extraction in the SSD target detection algorithm with a DenseNet convolutional neural network, and utilizing the dense connection mechanism of the DenseNet convolutional neural network to enhance feature reuse and gradient propagation capabilities, thereby improving the feature extraction accuracy of small target defects in wind power generation components;

[0079] S33: Replace the NMS algorithm used for prediction box screening in the SSD target detection algorithm with a Soft-PNMS algorithm based on a penalty factor, and improve the detection stability of densely arranged or partially occluded defects by dynamically adjusting the confidence of overlapping prediction boxes to complete the construction of the initial detection model.

[0080] It should be noted that in an exemplary embodiment of the present application, in step S33, the confidence of the overlapping prediction frame is dynamically adjusted to improve the detection stability of densely arranged or partially occluded defects, specifically including: when the intersection-over-union ratio between the two prediction frames increases, the Soft-PNMS algorithm nonlinearly enhances the penalty weight of the low-confidence prediction frame to improve the detection stability of densely arranged or partially occluded defects.

[0081] Specifically, see Figure 3As shown in the figure, the SSD object detection algorithm is a single-stage object detection algorithm. Its network model inherits the anchor box strategy of the Faster R-CNN network. By presetting anchor prediction boxes of different length and width ratios, each output feature layer is usually configured with 4 or 6 detection boxes to cover objects of different shapes. By adopting different scale detection methods, shallow feature maps are used to detect small objects, and deep feature maps are used to detect large objects. The process of the SSD target detection algorithm includes: determining the size of the input image, performing necessary preprocessing, and then inputting the preprocessed image into the SSD network; the preprocessed image is processed by a series of convolutional neural networks (i.e., VGG convolutional neural networks) to generate multiple feature maps of different scales (i.e., SSD feature layers), and the scales of the feature maps are usually 19*19, 10*10, 5*5, 3*3, 1*1, etc. The shallow feature map (such as 19*19) has a smaller receptive field and is suitable for detecting small targets, and the deep feature map (such as 1*1) has a larger receptive field and is suitable for detecting large targets; for each detection target in the output feature layer, a default box of different sizes is generated and its position coordinates are calculated. The SSD target detection algorithm inherits the Faster The anchor feature in the R-CNN algorithm; the position offset and classification accuracy of the boundary of each default box are calculated; the network model is trained on the feature image; the prediction box output by the model is post-processed, and the intersection-over-union (IOU) value of the prediction box is compared through the non-maximum suppression (NMS) algorithm. If it meets the preset threshold, it is retained, otherwise it is eliminated; finally, the defect type of the detection target is output and the location of the defect is selected on the input image to complete the target detection task. In addition, the NMS algorithm's screening process for the bounding box includes: calculating the area size of each candidate bounding box at the target position, sorting and confidence scoring from large to small according to the calculated results; cyclically comparing the area of ​​the bounding boxes, calculating the area size of the intersection area of ​​the first and second ranked bounding boxes, and calculating the ratio of the area of ​​the intersection area of ​​the two boxes to the area of ​​the union of the two boxes (i.e., the IOU value); comparing the preset IOU threshold with the calculated IOU value. If the preset IOU threshold is larger, the second ranked bounding box is removed, otherwise it is retained. Follow the above steps to select the remaining bounding boxes in turn.

[0082] See also Figure 4As shown, in this embodiment, the VGG16 convolutional neural network used for feature extraction in the traditional SSD object detection algorithm is replaced with a DenseNet convolutional neural network. This DenseNet convolutional neural network, through its unique dense connection mechanism, concatenates features along the channel dimension, rather than the linear superposition of traditional networks. That is, the input of each layer is the concatenation of the outputs of all previous layers, thus achieving feature reuse. Furthermore, the densely connected structure of the DenseNet convolutional neural network shortens the gradient propagation path by directly connecting shallow and deep layers, effectively alleviating the vanishing gradient problem in deep networks. Furthermore, compared to the VGG convolutional neural network, the DenseNet convolutional neural network significantly reduces the number of network parameters and model complexity through feature reuse and efficient feature extraction. Furthermore, in this embodiment, to address the problem of missed detections due to target overlap during wind turbine component defect detection, a penalty-based Soft-PNMS algorithm is introduced. When two wind turbine component defects partially occlude each other, the traditional NMS algorithm may mistakenly delete valid boxes due to the IOU value of the candidate boundary window being less than a preset IOU threshold, resulting in missed detections. The densely connected structure of the DenseNet convolutional neural network allows shallow layer features to be directly transferred to deep layers, enhancing the ability to fuse multi-scale features. Shallow layer features are reused in subsequent feature modules, reducing redundant calculations and improving detection efficiency. The DenseNet convolutional neural network has a performance advantage over ResNet and has more optimized calculation parameters and less computational data. Figure 5 As shown, Figure 5 This is a schematic diagram of the structure of the DenseNet convolutional neural network, which contains the DenseNet convolutional neural network structure of three densely connected modules. Figure 5 It can be seen that the feature input of each layer is the feature detected by all the previous convolutional layers as output. Figure 6 As shown, Figure 6 The figure is a schematic diagram of the structure of the dense connection module, which consists of multiple bottleneck layers, wherein each bottleneck layer includes BN, ReLU, 1×1 convolution, BN, ReLU and 3×3 convolution. Figure 6 As shown in the schematic diagram of the densely connected module structure, each densely connected module directly fuses the input of the current layer with the output of all previous layers in the channel dimension through channel splicing. In this embodiment, the DenseNet convolutional neural network is typically composed of four densely connected modules in series. As the network depth requirement increases, the number of densely connected modules will also increase accordingly.

[0083] Furthermore, during the prediction phase of wind turbine component defect detection, a large number of candidate detection target windows and corresponding window scores are generated. This requires the use of the NMS algorithm to compare and rank the confidence scores of the predicted boxes, calculate the intersection-over-union ratio of the candidate windows, and remove low-confidence candidate windows to obtain the wind turbine component defect detection windows. When dealing with situations where two wind turbine component defects partially obscure each other, the NMS algorithm uses a hard threshold screening strategy, which can easily lead to missed detections. Therefore, in this embodiment, a Soft-PNMS algorithm is proposed to improve the detection rate.

[0084] The Soft-NMS algorithm adds a small penalty factor to the prediction box with a low IOU value, and adds a large penalty factor to the prediction box with a high IOU value, thereby re-formulating the confidence score of the prediction window. Formula (1) is the expression of the Soft-NMS algorithm and the NMS algorithm. In formula (1), when f(I(b M ,b i When ))=0, the NMS algorithm is used for threshold screening.

[0085]

[0086] Among them, s i Represents the confidence score of the i-th default box of the detected target; b M Indicates the predicted box with the highest confidence score among all default boxes of the detected target; b i Represents the remaining i-th prediction box; I(b M ,b i ) is the calculation of b M and b i The value of the intersection-over-union (IOU) of .

[0087] Formula (2) is the expression of Soft NMS-linear, which is a linear penalty function. A linear function is used to penalize the prediction box with a large threshold. At this time, the penalty factor increases with b. m and b i The intersection-combination ratio changes with the change of .

[0088]

[0089] From the above analysis, we can see that although a linear penalty function is used to penalize prediction boxes with large thresholds, there are still defects. The screening threshold needs to be added manually, and when overlapping prediction boxes are calculated, the value will exceed the threshold, resulting in a sudden change in confidence. Therefore, it is necessary to select a nonlinear function, namely Soft NMS-Gaussian, to penalize prediction boxes with large thresholds and re-score the prediction boxes. As shown in formula (3):

[0090]

[0091] See also Figure 7 As shown in Figure 1, when the value range of the intersection-union ratio is large, such as in the interval [0.7, 1], the penalty strength of the constraint function is weakened. When the intersection-union ratio is 1, the penalty strength of the constraint function decreases, which can easily lead to false detection. To address the above problems, in this embodiment, a Soft-PNMS algorithm, namely a soft penalty non-maximum suppression algorithm, is proposed, such as formula (4) as the penalty factor of the NMS function.

[0092] s i =s i (1-I 2 (b M ,b i )) (4)

[0093] When the IOU value between two prediction windows increases, the weight of the prediction box penalty in the Soft-PNMS algorithm will also increase accordingly. At the same time, it solves the problem that the value of the intersection-over-union ratio when calculating overlapping prediction boxes will exceed the threshold, resulting in a sudden change in confidence. The penalty weight is smoother and more reasonable.

[0094] It is worth noting that after successfully building and optimizing the preset defect detection model, the model is capable of being officially put into the task of defect detection of wind turbine components. First, step S100 is executed, that is, using the onboard inspection system to perform automated patrol inspection to collect original images of wind turbine components.

[0095] Specifically, the image acquisition process uses an airborne inspection system consisting of a drone and an intelligent vehicle. The drone and the intelligent vehicle are used to synchronously collect images of components in wind turbine equipment that need to be inspected for defects, thereby completely obtaining the original images of the wind turbine components to be inspected.

[0096] Next, step S200 is executed, that is, a preprocessing operation is performed on the collected original image to obtain a preprocessed image.

[0097] Specifically, a series of preprocessing operations are performed on the captured raw image. These preprocessing operations include, but are not limited to, image denoising, contrast enhancement, and image resizing. Image denoising is used to eliminate environmental interference, contrast enhancement is used to improve image clarity, and image resizing is used to adapt to the input requirements of a subsequent pre-set defect detection model. In this embodiment, preprocessing operations are used to optimize the quality of the captured raw image, thereby improving the accuracy and efficiency of subsequent defect detection.

[0098] Next, step S300 is executed, that is, performing defect detection on the pre-processed image using a preset defect detection model to identify the type of defect target and locate the position of the defect target in the original image.

[0099] Specifically, the preprocessed image data obtained in step S200 is used as the input of the preset defect detection model. After the preset defect detection model receives the input image, the convolution layer inside it will perform feature extraction on the input image. The convolution layer slides on the image through a series of convolution kernels to calculate the convolution result, thereby extracting feature information at different levels in the image. After completing the feature extraction, the preset defect detection model locates and classifies the defect target and determines the type of defect target to which it belongs. It should be noted that after locating and classifying the defect target, the preset defect detection model may output multiple overlapping detection frames. These overlapping detection frames may lead to multiple detections of the same defect target, thereby affecting the accuracy and readability of the detection results. Therefore, an improved non-maximum suppression method is used for post-processing to further improve the accuracy of detection.

[0100] Finally, step S400 is executed, that is, the type of the defective object is output and the position of the defective object is framed in the original image.

[0101] In summary, the present invention provides a method for defect detection of wind power components, including: using an airborne inspection system to perform automated patrol inspection to collect original images of wind power components, performing preprocessing operations on the collected original images to obtain preprocessed images, using a preset defect detection model to perform defect detection on the preprocessed images to identify the type of defective targets and locate the positions of the defective targets in the original images, outputting the type of defective targets, and boxing out the positions of the defective targets in the original images. The method improves the backbone network of the traditional SSD target detection algorithm, replaces the VGG16 convolutional neural network used in the traditional SSD target detection algorithm with a DenseNet convolutional neural network for feature detection, and the DenseNet convolutional neural network uses its unique dense connection mechanism to connect features in the channel dimension, thereby achieving feature reuse. This feature reuse strategy can not only capture the features of the input image more comprehensively, but also enables more stable propagation of gradients during training, thereby slowing down the phenomenon of gradient disappearance. At the same time, the DenseNet convolutional neural network reduces the number of network parameters through dense connections while ensuring feature extraction capabilities, thereby reducing the complexity of the model and helping to improve the generalization ability and training efficiency of the model. In addition, during the detection process, when faced with a situation where two defective targets of wind power generation components partially occlude each other, a Soft-PNMS algorithm based on a penalty factor is introduced. The Soft-PNMS algorithm is optimized on the basis of the traditional NMS algorithm and adjusts the scores of candidate target windows by introducing a penalty factor. During the screening process, the Soft-PNMS algorithm does not directly suppress overlapping windows through a pre-set intersection-over-union ratio threshold, but dynamically adjusts the scores of windows based on the IOU value and penalty factor between the windows. Even if the IOU value between the two windows is large, as long as the defective targets corresponding to them have a high confidence level, they will not be easily suppressed, thereby effectively enhancing the detection capability of small target defects and improving the integrity and accuracy of wind power component defect detection.

[0102] Based on the same inventive concept, please refer to Figure 8 As shown, another embodiment of the present invention further provides a wind power component defect detection system 100, the system 100 comprising:

[0103] An image acquisition module 110 is used to acquire original images of wind power generation components;

[0104] The image processing module 120 is used to perform a preprocessing operation on the collected original image to obtain a preprocessed image;

[0105] The defect detection module 130 is used to perform defect detection on the preprocessed image using a preset defect detection model to identify the type of defect target and locate the position of the defect target in the original image;

[0106] The result output module 140 is configured to output the type of the defective object and select the location of the defective object in the original image.

[0107] It should be noted that the wind power component defect detection system 100 includes the wind power component defect detection method described in any of the above embodiments. Since the wind power component defect detection system 100 provided in this embodiment and the wind power component defect detection method provided in any of the above embodiments are based on the same inventive concept and therefore have at least the same beneficial effects, they will not be described in detail here.

[0108] Based on the same inventive concept, please refer to Figure 9 As shown, another embodiment of the present invention further provides an electronic device 11, which may include a memory 111, a processor 112 and a bus, and may also include a computer program stored in the memory 111 and executable on the processor 112, such as a wind power component defect detection program.

[0109] Among them, the memory 111 includes at least one type of readable storage medium, and the readable storage medium includes a flash memory, a mobile hard disk, a multimedia card, a card-type memory (for example, SD or DX memory, etc.), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the memory 111 can be an internal storage unit of the electronic device 11, such as a mobile hard disk of the electronic device 11. In other embodiments, the memory 111 can also be an external storage device of the electronic device 11, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device 11. Furthermore, the memory 111 can also include both an internal storage unit of the electronic device 11 and an external storage device. The memory 111 can not only be used to store application software and various types of data installed in the electronic device 11, such as codes for defect detection of wind power components, etc., but can also be used to temporarily store data that has been output or is to be output.

[0110] In some embodiments, the processor 112 may be composed of an integrated circuit, such as a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and a combination of various control chips. The processor 112 is the control core (Control Unit) of the electronic device 11. It utilizes various interfaces and circuits to connect the various components of the entire electronic device 11. It executes or runs programs or modules stored in the memory 111 (such as a wind power component defect detection program) and calls data stored in the memory 111 to perform various functions of the electronic device 11 and process data.

[0111] The processor 112 executes the operating system and various installed application programs of the electronic device 11. The processor 112 executes the application programs to implement the steps in the above-mentioned wind power generation component defect detection method.

[0112] Exemplarily, the computer program may be divided into one or more modules, which are stored in the memory 111 and executed by the processor 112 to implement the present application. The one or more modules may be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program in the electronic device 11. For example, the computer program may be divided into an image acquisition module 110, an image processing module 120, a defect detection module 130, and a result output module 140.

[0113] The above-mentioned integrated unit implemented in the form of a software functional module can be stored in a computer-readable storage medium, which can be either non-volatile or volatile. The above-mentioned software functional module is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, computer equipment, or network equipment, etc.) or a processor to perform part of the functions of the wind power component defect detection method described in various embodiments of this application.

[0114] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the present invention. Anyone skilled in the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by one of ordinary skill in the art without departing from the spirit and technical principles disclosed herein are intended to be covered by the claims of the present invention.

Claims

1. A method for detecting defects in wind power generation components, characterized in that: include: Use an airborne inspection system to conduct automated patrol inspections to collect original images of wind turbine components; Performing a preprocessing operation on the collected original image to obtain a preprocessed image; Performing defect detection on the preprocessed image using a preset defect detection model to identify the type of defect target and locate the position of the defect target in the original image; The type of the defective object is output, and the position of the defective object is framed in the original image.

2. The wind power generation component defect detection method according to claim 1, characterized in that: The preset defect detection model is established in the following way: Obtaining original image samples of wind power generation components; Constructing a sample dataset of wind power component defects based on the original image samples; Dividing the sample data set into a training set and a test set for model training and model testing respectively; Build an initial detection model; Performing model training on the initial detection model using the training set to obtain an intermediate detection model; The intermediate detection model is tested using the test set, and the model parameters of the intermediate detection model are adjusted according to the test results to obtain the final preset defect detection model.

3. The method for detecting defects in wind power generation components according to claim 2, wherein: The sample data set of wind power component defects includes defect data samples of insulator breakage, flashover, and blade breakage, dirt, and paint peeling.

4. The method for detecting defects in wind power generation components according to claim 2, wherein: The obtaining of original image samples of wind power generation components includes: An airborne inspection system is used to perform automated patrol inspections to collect original image samples of wind turbine components.

5. The method for detecting defects in wind power generation components according to claim 2, wherein: The step of constructing a sample dataset of defects of wind power generation components based on the original image samples includes: constructing an initial sample dataset of wind power component defects based on the original image samples; The original image samples are subjected to data enhancement processing to expand the size of the initial sample data set, thereby obtaining a high-quality sample data set for constructing the preset defect detection model, wherein the data enhancement processing includes distortion correction simulation, multi-angle rotation and noise injection.

6. The method for detecting defects in wind power generation components according to claim 2, wherein: The constructing of the initial detection model includes: Using the SSD target detection algorithm as the framework, build the initial network model; The VGG16 convolutional neural network used for feature extraction in the SSD target detection algorithm is replaced with a DenseNet convolutional neural network. The dense connection mechanism of the DenseNet convolutional neural network is used to enhance feature reuse and gradient propagation capabilities, thereby improving the feature extraction accuracy of small target defects in wind turbine components. The NMS algorithm used for prediction box screening in the SSD target detection algorithm is replaced with a Soft-PNMS algorithm based on a penalty factor. By dynamically adjusting the confidence of overlapping prediction boxes, the detection stability of densely arranged or partially occluded defects is improved to complete the construction of the initial detection model.

7. The method for detecting defects in wind power generation components according to claim 6, wherein: The dynamic adjustment of the confidence level of the overlapping prediction boxes improves the detection stability of densely arranged or partially occluded defects, including: When the intersection-over-union ratio between two prediction boxes increases, the Soft-PNMS algorithm nonlinearly enhances the penalty weight of the low-confidence prediction box to improve the detection stability of densely arranged or partially occluded defects.

8. A wind power component defect detection system, characterized in that: include: An image acquisition module, used for acquiring original images of wind power generation components; An image processing module is used to perform a preprocessing operation on the collected original image to obtain a preprocessed image; A defect detection module is used to perform defect detection on the preprocessed image using a preset defect detection model to identify the type of defect target and locate the position of the defect target in the original image; The result output module is used to output the type of the defect target and select the position of the defect target in the original image.

9. An electronic device, characterized in that: The electronic device comprises: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the electronic device to implement the wind power generation component defect detection method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that A computer program is stored thereon, and when the computer program is executed by a processor of a computer, the computer is caused to execute the wind power generation component defect detection method according to any one of claims 1 to 7.

Images