A Defect Detection Method for Wind Turbine Blades Based on Real-Time Image Cropping and Attention Mechanism

Abstract

This invention relates to a method for detecting defects in wind turbine blades based on real-time image cropping and an attention mechanism. The method includes: inputting an image to be detected into a constructed defect detection model; the defect detection model first resizing the input image, then using a feature extractor to extract basic feature information; after feature extraction, an attention probability map is obtained through an attention mechanism module; based on the results of the attention probability map, an online image cropping module cropps the original image to its original size, and the cropped image patch is used as input to the detector to focus attention on the region most meaningful for defect detection; then, the image patch and the attention probability map are multiplied and fed into the defect detector, which analyzes the cropped image; the defect detection model outputs the analysis result, i.e., the defect detection result. This method is beneficial for improving the robustness, accuracy, and detection speed of blade defect detection.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to a method for detecting defects in wind turbine blades based on real-time image cropping and attention mechanisms. Background Technology

[0002] Blades are a crucial component of wind turbine systems; however, the complexity of their environment can lead to damage or defects, increasing maintenance costs and safety risks. Therefore, defect detection technology for wind turbine blades is essential for ensuring stable operation and reducing failures. Currently, commonly used blade defect detection technologies include acoustic emission, ultrasonic, vibration, thermal imaging, and machine vision methods. Acoustic emission-based blade detection requires installing numerous sensors on the blade to cover potentially damaged areas, but there is no physical connection between sensor placement and blade damage. Furthermore, acoustic emission signals are very difficult to distinguish from signals in noisy environments, requiring high-sampling-frequency data acquisition systems, which increases data processing complexity and cost. Computer vision-based detection technology has become an important method for wind turbine blade defect detection, capable of detecting external defects such as cracks and scratches. However, the limited range of detectable defects and high dependence on the detection environment result in poor robustness. Summary of the Invention

[0003] The purpose of this invention is to provide a method for detecting defects in wind turbine blades based on real-time image cropping and attention mechanisms. This method is beneficial to improving the robustness, accuracy, and speed of blade defect detection.

[0004] To achieve the above objectives, the technical solution adopted by this invention is: a method for detecting defects in wind turbine blades based on real-time image cropping and attention mechanisms, comprising:

[0005] Input the image to be detected into the constructed defect detection model;

[0006] The defect detection model first resizes the input image, then uses a feature extractor to extract basic feature information after scaling. Following feature extraction, the attention probability map y is obtained through the attention mechanism module A-net. a The attention probability map indicates the importance of different regions in the image; based on the attention probability map y aThe result is that the original size of the image is cropped by the online image cropping module, and the cropped image patch is used as input to the detector to focus attention on the region most meaningful for defect detection; then the image patch and the attention probability map are multiplied and fed into the defect detector, and the defect detector analyzes the cropped image.

[0007] The defect detection model outputs analysis results, which are the results of defect detection.

[0008] Furthermore, the attention mechanism module A-net consists of 6 residual blocks, with each channel set to 16, 32, 64, 32, 16, and 1 respectively; the attention mechanism module transforms the linear output into a probability graph y by applying an element-wise softmax layer. a This ensures that each element's value ranges between 0 and 1; then, the confidence level of the detection results is compared with the probability graph y. a The probability values ​​corresponding to the center point of the detection box are multiplied to suppress false detections in the background area.

[0009] Furthermore, the attention mechanism module A-net is trained through supervised learning, and the supervised real values ​​are obtained through image segmentation of wind turbine blades;

[0010] Let the width of the image be w and the height be h, then the distance matrix D from each point on the image to the center point can be obtained:

[0011]

[0012] Among them, D i，j Let (i, j) represent the Euclidean distance from point (i, j) to the image center point; based on this distance matrix, the expression for the high-pass filter is obtained:

[0013]

[0014] Where, σ 2 The value M represents the cutoff frequency of the low-pass filter; therefore, the true value M of the attention mechanism module... gt as follows:

[0015]

[0016] Among them, F m It is the result of Fourier transforming the masked image of the wind turbine blades. IDFT is the inverse Fourier transform function.

[0017] Furthermore, based on the attention probability map y a The result is that the original image size is cropped using an online image cropping module, and the implementation method is as follows:

[0018] In the probability diagram ya The non-zero probability point is circumscribed by a rectangle, and the gradient direction of the diagonal of the rectangle is the linear equation of the leaf in the image; based on the image height H, width W, and the set n... p Given the number of blocks, the gradients of the image along the x-axis and y-axis are calculated as follows:

[0019]

[0020]

[0021] During the cropping process, select the point (x) with the highest probability in the probability graph. c y c The center point of the i-th block is used as the cropping center point, and cropping is performed bidirectionally along the gradient direction. The coordinates of the center point of the i-th block are calculated as follows:

[0022]

[0023]

[0024] Where α is the fill value during cutting; since the height of each block is fixed at h and the width is fixed at w, all blocks can be obtained quickly after the center point is determined.

[0025] Compared with existing technologies, the present invention has the following advantages: The present invention provides a method for detecting defects in wind turbine blades that combines real-time image cropping and attention mechanisms. This method, by designing an attention network A-net and training it with supervised learning, can suppress background information and enhance the detector's attention to the wind turbine blades. At the same time, through online image cropping, most of the background information can be removed in the early stage of detection, avoiding unnecessary detection of background information, reducing processing time, and improving detection speed. It exhibits better performance than other detection methods. Attached Figure Description

[0026] Figure 1 This is a flowchart illustrating the method implementation of an embodiment of the present invention;

[0027] Figure 2 This is an architecture diagram of the attention mechanism module in an embodiment of the present invention;

[0028] Figure 3 This is the actual value M in the embodiments of the present invention. gt The visualization results;

[0029] Figure 4 This is a schematic diagram of the online image cropping processing results in an embodiment of the present invention. Detailed Implementation

[0030] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0031] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0032] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0033] like Figure 1 As shown, this embodiment provides a method for detecting defects in wind turbine blades based on real-time image cropping and attention mechanisms, including:

[0034] 1) Input the image to be detected into the constructed defect detection model.

[0035] 2) The defect detection model first resizes the input image, then uses a feature extractor to extract basic feature information. After feature extraction, the attention probability map y is obtained through the attention mechanism module A-net. a The attention probability map indicates the importance of different regions in an image; based on the attention probability map y a The result is that the original image size is cropped by an online image cropping module, and the cropped image patch is used as input to the detector to focus attention on the region most meaningful for defect detection; then the image patch and the attention probability map are multiplied and fed into the defect detector, and the defect detector analyzes the cropped image.

[0036] 3) The defect detection model outputs the analysis results, that is, the defect detection results are obtained.

[0037] The architecture of the attention mechanism module A-net is as follows: Figure 2 As shown, the attention mechanism module A-net consists of 6 residual blocks, with channel settings of 16, 32, 64, 32, 16, and 1 respectively. The attention mechanism module transforms the linear output into a probability map y by applying an element-wise softmax layer. a This ensures that each element's value ranges between 0 and 1. Then, the confidence level of the detection results is compared with the probability graph y. aThe probability values ​​corresponding to the center points of the detection boxes are multiplied, effectively suppressing false detections in background areas. This approach introduces an attention probability map into the detection process, making the detection results more accurate and reliable. By combining the attention mechanism with the detector output, defects in images can be better identified and located.

[0038] In this embodiment, the attention mechanism module A-net is trained through supervised learning, and the supervised ground truth is obtained by image segmentation of wind turbine blades.

[0039] Let the width of the image be w and the height be h, then the distance matrix D from each point on the image to the center point can be obtained:

[0040]

[0041] Among them, D i，j Let (i, j) represent the Euclidean distance from point (i, j) to the image center. Based on this distance matrix, the expression for the high-pass filter is obtained:

[0042]

[0043] Where, σ 2 This represents the cutoff frequency of the low-pass filter. The actual value M of the attention mechanism module... gt as follows:

[0044]

[0045] Among them, F m It is the result of Fourier transforming the masked image of the wind turbine blades. IDFT is the inverse Fourier transform function.

[0046] In this embodiment, the true value M gt The visualization results are as follows Figure 3 As shown.

[0047] Common object detection methods involve image preprocessing, specifically reducing the image resolution to fit the input of a convolutional neural network. However, for detecting defects in wind turbine blades, simply reducing the image resolution is insufficient for accuracy, as the defects are difficult to detect in the reduced image. Therefore, this paper proposes an innovative online image cropping method based on the attention probability map y. a The result is that the original image size is cropped into blocks, thereby reducing the image size and removing unnecessary information. This can help improve the efficiency of convolutional neural networks and reduce training time and resource requirements. The cropped image still retains enough information for tasks such as target recognition and classification. The specific implementation method is described below.

[0048] By observing the linear characteristic of wind turbine blades in the image, in the probability graph y a The non-zero probability points are circumscribed by a rectangle, and the gradient direction of the rectangle's diagonal is the linear equation of the leaf in the image. This is based on the image's height H, width W, and a set n. p Given the number of blocks, the gradients of the image along the x-axis and y-axis are calculated as follows:

[0049]

[0050]

[0051] During the cropping process, select the point (x) with the highest probability in the probability graph. c y c The center point of the i-th block is used as the cropping center point, and cropping is performed bidirectionally along the gradient direction. The coordinates of the center point of the i-th block are calculated as follows:

[0052]

[0053]

[0054] Where α is the padding value during trimming. Since the height of each block is fixed at h and the width at w, after determining the center point, all blocks can be quickly obtained, such as... Figure 4 As shown, this online cropping method based on the geometric features of wind turbine blades can crop images more efficiently, reduce processing time, and avoid unnecessary detection of background information.

[0055] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0056] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0057] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0058] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0059] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A method for detecting defects in wind turbine blades based on real-time image cropping and attention mechanisms, characterized in that, include: Input the image to be detected into the constructed defect detection model; The defect detection model first resizes the input image, then uses a feature extractor to extract basic feature information after scaling. Following feature extraction, an attention probability map is obtained through the A-net attention mechanism module. The attention probability map indicates the importance of different regions in the image; based on the attention probability map... As a result, the original image size is cropped by an online image cropping module, and the cropped image patch is used as input to the detector to focus attention on the area most meaningful for defect detection. The image patch and the attention probability map are then multiplied and fed into the defect detector, which analyzes the cropped image. The defect detection model outputs analysis results, that is, the defect detection results are obtained; The attention mechanism module A-net consists of 6 layers of residual blocks; the attention mechanism module transforms the linear output into a probability graph by applying an element-wise softmax layer. This ensures that each element's value ranges between 0 and 1; then, the confidence level of the detection results is compared with the probability graph. The probability values ​​corresponding to the center point of the detection box are multiplied to suppress false detections in the background area; According to the attention probability map The result is that the original image size is cropped using an online image cropping module, and the implementation method is as follows: In probability diagram The non-zero probability points are circumscribed by a rectangle, and the gradient direction of the rectangle's diagonal is the linear equation of the leaf in the image; based on the image's height H, width W, and set parameters... Given the number of blocks, the gradients of the image along the x-axis and y-axis are calculated as follows: During the cropping process, select the point with the highest probability from the probability graph. The center point of the cropped image is used as the center point, and cropping is performed bidirectionally along the gradient direction; the coordinates of the center point of the i-th block are calculated as follows: in, This is the fill value during cutting; since the height of each block is fixed at h and the width is fixed at w, all blocks can be quickly obtained after the center point is determined.

2. The method for detecting defects in wind turbine blades based on real-time image cropping and attention mechanism according to claim 1, characterized in that, The channels of the 6-layer residual blocks of the attention mechanism module A-net are set to 16, 32, 64, 32, 16 and 1, respectively.

3. The method for detecting defects in wind turbine blades based on real-time image cropping and attention mechanism according to claim 1, characterized in that, The attention mechanism module A-net is trained through supervised learning, and the real values ​​of the supervision are obtained by image segmentation of wind turbine blades. Let the width of the image be... The height is Then we can obtain the distance matrix from the points on the image to the center point. : in, Point The Euclidean distance to the image center point; based on this distance matrix, the expression for the high-pass filter is obtained: in, This represents the cutoff frequency of the low-pass filter; the actual value of the attention mechanism module... as follows: in, It is the result of Fourier transforming the masked image of the wind turbine blades. IDFT is the inverse Fourier transform function.

Images