Fan blade crack defect detection method and system based on improved Yolov8-obb

Abstract

The invention discloses an improved Yolov8-obb-based fan blade crack defect detection method and system. The system comprises a visual module, a transceiver module and a processing module. The detection method comprises the following steps: acquiring fan blade surface state image information; the collected fan blade state image is preprocessed; the improved Yolov8-obb network model is trained, and a weight is generated; and detecting the fan blade state image. According to the method, rotating target detection and deep learning are combined, the real-time state image of the fan blade can be shot through the unmanned aerial vehicle to detect the crack defect of the fan blade, the image with the crack defect is marked, and a good effect is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of fan blade crack defect detection, and in particular to a fan blade crack defect detection method and system based on improved Yolov8-obb. Background Art

[0002] Wind energy is a widely available renewable energy source. By collecting and utilizing wind energy, the energy and environmental pressures of economic and social development can be alleviated. In order to obtain more energy, wind power generation devices are often installed in windy areas, which are often accompanied by extreme weather such as hail, acid rain and sandstorms. As a key component of wind turbines, blades are prone to crack defects when running for a long time in such severe weather.

[0003] The development and innovation of computer technology have given rise to detection methods based on computer vision. Many effective detection algorithms have been developed and applied, which can be divided into two categories: detection algorithms based on image processing and detection algorithms based on machine learning.

[0004] The detection algorithm based on image processing takes the inherent properties of the image as the processing object, and combines various filtering algorithms and image processing techniques to realize detection and analysis. This detection method only expands the inherent properties of the image with the help of mathematical operations, which has significant limitations. For changing environments, parameters need to be changed frequently to complete the detection task. The Chinese patent "A method for automatic detection of fan blade defects based on images" (application number: 202111355749.4) proposes the use of image methods to detect defects in fan blades, but this method has low detection accuracy and is difficult to meet actual conditions.

[0005] Deep learning technology abandons manual intervention in feature extraction, completes autonomous training of input data by constructing a neural network, and uses the computer's "perspective" to extract deep features of the data. Compared with traditional machine learning algorithms, this algorithm has better expression capabilities. The literature "Li Bing, Bai Yunshan, Zhao Kuan, et al. Wind turbine blade surface defect detection algorithm based on HSCA-YOLOv7 [J]. China Electric Power, 2023, 56(10): 43-52." proposed an improved Yolov7 network for wind turbine blade surface defect detection, but the detection frame of the network is difficult to fit the actual defects compared with the rotating target detection frame.

[0006] In summary, current wind turbine blade defect detection methods often only focus on improving detection accuracy. The detection target frame usually contains a large range of irrelevant information, and the defect position cannot be accurately located and fitted, resulting in poor results. Summary of the invention

[0007] In order to solve the problem that the current fan blade crack defect detection method is difficult to accurately locate the defect. The present invention provides a fan blade crack defect detection method and system based on improved Yolov8-obb, which combines rotating target detection with deep learning, can detect fan blade crack defects by taking real-time state images of fan blades by drone, and mark pictures with crack defects, which has a good effect.

[0008] The technical solution adopted by the present invention is:

[0009] A fan blade crack defect detection system based on improved Yolov8-obb, the system includes:

[0010] A visual module is used to record the surface status of wind turbine blades;

[0011] The transceiver module is responsible for the communication and information transmission between the visual module and the processing module;

[0012] The processing module is used to control the UAV and detect whether there are defects on the surface of the wind turbine blades, and save the detection results.

[0013] The fan blade crack defect detection method based on the improved Yolov8-obb ​​includes the following steps:

[0014] Step 1: Collecting image information of the surface state of the fan blade;

[0015] Step 2: preprocessing the collected fan blade status image;

[0016] Step 3: Train the improved Yolov8-obb ​​network model and generate weights;

[0017] Step 4: Detect the status image of the fan blade;

[0018] Step 5: Analyze and summarize the detection results and count the number of images with crack defects.

[0019] In the step 1, image information of the surface state of the fan blades is collected, including normal fan blades and fan blades with crack defects, wherein 3000 images of each type are used as the initial data set.

[0020] In the step 2, the collected fan blade surface state images are preprocessed, and the final fan blade state data set is obtained by preprocessing the fan blade state images in the initial data set.

[0021] The MSRCR algorithm is used to preprocess the surface state image of the wind turbine blade; based on the MSRCR algorithm, it is first decomposed into images of different scales, and then the adaptive gain function is used to enhance the image of each scale, and finally the image is reconstructed to achieve the enhancement effect. The following steps are included:

[0022] Step 2.1: Read the original image data and convert the integer to double type using the log function according to the formula s(x,y)=log[S(x,y)];

[0023] Step 2.2: According to the formula Obtain the logarithmic domain enhancement result r(x,y);

[0024] Step 2.3: According to the formula R(x,y)=exp[r(x,y)], the value of r(x,y) is converted from the logarithmic domain to the real number domain R(x,y) through the natural exponential function operation;

[0025] Step 2.4: Introduce adaptive enhancement function C k (x,y), enhance the image:

[0026]

[0027] Step 2.5: According to formula R MSRCR (x,y)=C k (x,y)iR(x,y) to get the final output result.

[0028] In the above formula, S(x,y) represents the original image data, s(x,y) represents the converted image data, and r(x,y) represents the logarithmic domain enhancement result. It means summing the results of taking k from 1 to 3, q k is the weight corresponding to the k value, log represents the logarithmic calculation of the data, exp represents the natural exponential function calculation of the data, R(x,y) represents the real domain enhancement result, C k (x, y) is the adaptive enhancement function, β is the gain constant, α is the nonlinear transformation adjustment coefficient, s k (x, y) represents the image data when k takes the corresponding value, R MSRCR (x,y) is the final output result.

[0029] In step 3, the wind turbine blade status data set is divided into a training set and a test set in a ratio of 8:2, and input into the improved Yolov8-obb ​​network model for training, and finally the weight with the highest accuracy is selected and saved for performing subsequent defect target detection tasks.

[0030] In step 3, the OBB (Oriented Bounding Box) used by the Yolov8-obb ​​network is a rotation detection box. The Yolov8-obb ​​algorithm consists of 5 parameters, and its target detection box is represented by OBB(x obb ,y obb ,w obb ,h obb ,θ obb ), where (x obb ,y obb ) is the center coordinate of the OBB rotation detection frame, w obb is the long side of the OBB rotation detection box, h obb With the long side w obb The adjacent sides can also be called short sides, θ obb is the positive direction of the x-axis and the long side w obb The angle between θ∈[-π / 2,π / 2];

[0031] The improved Yolov8-obb ​​network model structure improves the C2f module in the original Yolov8-obb ​​network to the C3-BiFormer module; improves the SPPF module in the original Yolov8-obb ​​network to the SPPFCSPC module; and improves the original loss function CIoU to SIoU with angle loss.

[0032] In the Yolov8-obb ​​network, the C3 module is fused with the BiFormer attention mechanism to replace the C2f module in the original network to enhance the network's attention to small target features.

[0033] The SPPF module in the Yolov8-obb ​​network is replaced by the SPPFCSPC module, and multiple convolution and cascade techniques are introduced to extract and fuse features of different scales, thereby expanding the model receptive field.

[0034] The SPPFCSPC module integrates two submodules: SPP and fully connected spatial pyramid convolution (FCSPC). SPP acts as a pooling layer to process input feature maps of different scales; fully connected spatial pyramid convolution (FCSPC) is an improved convolution layer used to optimize the representation of feature maps. By performing multi-scale spatial pyramid pooling on the input feature map, the SPP module captures information about objects and scenes at different scales. Subsequently, the FCSPC module convolves the feature maps of different scales output by the SPP module and divides the input feature map into blocks. These blocks are then merged and connected, and then convolved, so that the receptive field of the model is enhanced while retaining key feature information, thereby improving the accuracy of the model.

[0035] The calculation formula of SPPFCSPC module is as follows:

[0036]

[0037]

[0038] S 4 =S 1 *S 2 *S 3

[0039] Among them, R represents the input feature layer, S 1 (R) represents the pooling layer result of the minimum pooling kernel, p represents the padding value of the pooling operation, k represents the size of the pooling kernel of the pooling operation, Indicates that the input feature layer is pooled with a padding value of 2 and a pooling kernel size of 5. 2 (S 1 ) represents the pooling layer result of a medium-sized pooling kernel, Indicates that the pooling operation with a fill value of 2 and a pooling kernel size of 5 is performed on the pooling layer result of the minimum pooling kernel. 3 (S 2 ) represents the pooling layer result of the maximum pooling kernel, Indicates that the pooling operation with a fill value of 2 and a pooling kernel size of 5 is performed on the pooling layer result of the medium-sized pooling kernel. 4 Indicates the final output result.

[0040] In step 3, during network training, the improved Yolov8-obb ​​network model uses the SIoU loss function. It takes into account the vector angle between the real detection box and the predicted detection box, and redefines the penalty index. The SIoU loss function consists of 4 cost functions: ①, Angle cost; ②, Distance cost; ③, Shape cost; ④, IoU cost.

[0041] The calculation formula is as follows:

[0042] ①、Angle cost:

[0043]

[0044] In the above formula, Λ represents the calculation result of Anglecost, α represents the angle between the center point of the real detection box and the predicted detection box, x represents the sine function value of the angle α, and c h represents the height difference between the center point of the real detection box and the predicted detection box, σ represents the horizontal distance between the center point of the real detection box and the predicted detection box, Indicates the ordinate of the center point of the real detection box, b cy Represents the ordinate of the center point of the predicted detection box, express With b cy The maximum value between express With b cy The minimum value between Indicates the horizontal coordinate of the center point of the real detection frame; b cx Represents the horizontal coordinate of the center point of the predicted detection box.

[0045] ②、Distance cost:

[0046]

[0047]

[0048] In the above formula, Δ represents the calculation result of Distance cost. It means to calculate and sum the cases of t=x or t=y respectively, x represents the horizontal direction, y represents the vertical direction, γ represents the calculation parameter, ρ t It means that the case of t = x or t = y is substituted into the calculation, e (-γρt) Represents -γρ t Perform exponential function operation, ρ x Represents the calculation result for the horizontal direction, ρ y Indicates the calculation result for the vertical direction, w c Indicates the width of the minimum bounding rectangle of the real detection box and the predicted detection box, h c Indicates the height of the minimum bounding rectangle of the real detection box and the predicted detection box.

[0049] ③、Shape cost:

[0050]

[0051] In the above formula, Ω represents the calculation result of Shapecost, w represents the width of the predicted detection box, and h represents the height of the predicted detection box. It means to calculate and sum the cases of t=w or t=h, ω t represents the value of ω when t = w or t = h, θ represents the degree of concern for shape loss, which is generally between [2, 6], ω w represents the calculation result in the width direction, ω h Indicates the calculation result in the height direction, w gt Indicates the width of the real detection box; h gt Indicates the height of the real detection box; max(w,w gt ) represents w and w gt The maximum value between max(h,h gt) represents h and h gt The maximum value between .

[0052] ④, IoU cost:

[0053]

[0054] In the above formula, A represents the real detection box, B represents the predicted detection box, A∩B represents the intersection of A and B, and A∪B represents the union of A and B.

[0055] In summary, the expression of SIoU loss function is:

[0056]

[0057] Repeat the training for multiple rounds, and finally select the weight with the highest accuracy on the test set to save. During the training process, the training parameter setting will generate a weight file that can be used for subsequent detection tasks every ten rounds of training. In order to make the subsequent detection accuracy higher, each generated weight will be tested in the test set to verify the correctness of the detection, and then the weight accuracy p will be obtained. Among them, TP is a correctly detected positive result, and FP is an erroneously detected positive result. Finally, the weight with the highest accuracy is selected for subsequent defect detection work.

[0058] In step 4, during the process of detecting crack defects on the surface of the fan blade, the surface state of the fan blade is recorded using the visual module, and after detection by the processing module, the detection result is obtained.

[0059] In step 5, according to the detection result obtained in step 4, the surface state image of the wind turbine blade with crack defects is statistically analyzed to obtain a detection report.

[0060] The present invention discloses a fan blade crack defect detection method and system based on improved Yolov8-obb, and the technical effects are as follows: 1) The present invention adopts rotating target detection and uses a rotating target frame to mark the crack defect position, so that the target frame can locate the crack defect position more accurately, and the effect is better.

[0061] 2) The present invention adopts an improved Yolov8-obb ​​deep learning network training data set to obtain the detection weight. The improved Yolov8-obb ​​deep learning network can more effectively enhance the network's learning ability for image features by improving the C2f module in the original network to the C3-BiFormer module; by improving the SPPF module in the original network to the SPPFCSPC module, the network reasoning speed can be more effectively improved; the original loss function CIoU is improved to SIoU with angle loss. The improved loss function can better describe the regression of the rotating target, improve the real-time and accuracy of detection, and has good practical value.

[0062] 3) The fan blade surface crack defect detection method of the present invention can effectively detect the crack defects on the fan blade surface and fit the direction and width of the crack with good accuracy. The present invention can effectively improve the monitoring of the fan blade status, reduce the probability of accidents, and reduce maintenance costs.

[0063] 4) In the process of detecting crack defects on the surface of wind turbine blades, the present invention introduces rotating target detection and an improved Yolov8-obb ​​deep learning network, so that the detection frame fits the actual defect position better than other detection algorithms. At the same time, the accuracy and reliability of the detection method are improved by improving the network. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 It is a schematic diagram of the process of the present invention.

[0065] Figure 2 This is an example diagram of the final wind turbine blade status data set described in the present invention.

[0066] Figure 3 This is a flow chart of the MSRCR algorithm described in the present invention.

[0067] Figure 4 This is a diagram showing the OBB detection effect of the present invention.

[0068] Figure 5 It is a schematic diagram of the principle of the OBB detection frame of the present invention.

[0069] Figure 6 This is the improved Yolov8-obb ​​network structure diagram described in the present invention.

[0070] Figure 7 This is a schematic diagram of the C3-BiFormer module of the present invention.

[0071] Figure 8 This is a schematic diagram of the SPPFCSPC module described in the present invention.

[0072] FIG9( a ) is a first image of the surface state of a fan blade according to an embodiment;

[0073] FIG9( b ) is a second image of the surface state of the fan blade of the embodiment.

[0074] FIG. 10( a ) is a first detection result of crack defects on the surface of a fan blade of an embodiment;

[0075] FIG. 10( b ) is a second detection result of crack defects on the surface of the fan blade of the embodiment. DETAILED DESCRIPTION

[0076] A fan blade crack defect detection system based on improved Yolov8-obb, the system includes:

[0077] A visual module is used to record the surface status of wind turbine blades. The visual module consists of a drone and a camera on board.

[0078] The transceiver module is responsible for the communication and information transmission between the visual module and the processing module. It uses radio communication to carry out two-way communication between the visual module and the processing module through radio frequency waves to transmit flight control instructions and image information.

[0079] The processing module is composed of a computer equipped with a UAV control system, wind turbine blade defect detection weights and data storage functions. It is used to control the UAV and detect whether there are defects on the surface of the wind turbine blades, and save the detection results.

[0080] The crack defect detection method of fan blades based on improved Yolov8-obb ​​is as follows: Figure 1 As shown, the following steps are included:

[0081] Step 1: Collecting image information of the surface state of the fan blade;

[0082] Step 2: preprocessing the collected fan blade status image;

[0083] Step 3: Train the improved Yolov8-obb ​​network model and generate weights;

[0084] Step 4: Detect the status image of the fan blade;

[0085] Step 5: Analyze and summarize the detection results and count the number of images with crack defects.

[0086] In the step 1, image information of the surface state of the fan blades is collected, including normal fan blades and fan blades with crack defects, wherein 3000 images of each type are used as the initial data set.

[0087] In step 2, the collected fan blade surface state image is preprocessed. The purpose of image preprocessing is to improve the effect and accuracy of image processing, and then improve the stability and accuracy of subsequent processing. By preprocessing the fan blade state image in the initial data set, the final fan blade state data set is obtained. The data set example is as follows Figure 2 shown.

[0088] The preprocessing mainly uses the MSRCR algorithm in the Retinex algorithm to preprocess the surface state image of the wind turbine blade; Retinex is an algorithm commonly used for image enhancement, and its core idea is to adjust the image contrast and brightness while retaining the image detail information. There are three main different implementations of the Retinex algorithm: single-scale Retinex (SSR), multi-scale Retinex (MSR) and multi-scale adaptive gain Retinex (MSRCR). The SSR algorithm assumes that the illumination changes in the image are mainly concentrated on the low-frequency components, so only one filtering process is required to remove the low-frequency components, and then the obtained medium and high-frequency components are enhanced. The MSR algorithm decomposes the image into images of different scales and enhances the image of each scale to retain the detail information of different scales. The MSRCR algorithm combines the first two algorithms, first decomposing them into images of different scales, then using an adaptive gain function to enhance the image of each scale, and finally reconstructing the image to achieve the enhancement effect. The flow chart of the MSRCR algorithm used in the present invention is as follows: Figure 3 As shown, the following steps are included:

[0089] Step 2.1: Read the original image data and convert the integer to double using the log function according to the formula s(x,y)=log[S(x,y)].

[0090] Step 2.2: According to the formula The logarithmic domain enhancement result r(x,y) is obtained.

[0091] Step 2.3: According to the formula R(x,y)=exp[r(x,y)], the value of r(x,y) is converted from the logarithmic domain to the real number domain R(x,y) through the natural exponential function operation.

[0092] Step 2.4: Introduce adaptive enhancement function C k (x,y), enhance the image.

[0093]

[0094] Step 2.5: According to formula R MSRCR (x,y)=C k (x,y)iR(x,y) to get the final output result.

[0095] In the above formula, S(x,y) represents the original image data, s(x,y) represents the converted image data, and r(x,y) represents the logarithmic domain enhancement result. It means summing the results of taking k from 1 to 3, q k is the weight corresponding to the k value, log represents the logarithmic calculation of the data, exp represents the natural exponential function calculation of the data, R(x,y) represents the real domain enhancement result, C k (x, y) is the adaptive enhancement function, β is the gain constant, α is the nonlinear transformation adjustment coefficient, s k (x, y) represents the image data when k takes the corresponding value, R MSRCR (x,y) is the final output result.

[0096] In step 3, the wind turbine blade status data set is divided into a training set and a test set in a ratio of 8:2, and input into the improved Yolov8-obb ​​network model for training, and finally the weight with the highest accuracy is selected and saved for performing subsequent defect target detection tasks.

[0097] In the wind turbine blade status dataset, there are 2400 training set images and 600 test set images, both with a resolution of 300 × 300 pixels. The network training parameters are as follows: learning rate is 0.001, network model optimizer is Adam, batch size is 16, training cycle is 300 rounds, including 30 rounds of freezing cycle, and other hyperparameters are all default values.

[0098] In step 3, the OBB (Oriented Bounding Box) used by the Yolov8-obb ​​network is a rotation detection box. The detection effect is as follows: Figure 4 As shown. The OBB detection frame principle of Yolov8-obb ​​algorithm is as follows Figure 5 As shown in the figure, the Yolov8-obb ​​algorithm consists of 5 parameters, and its target detection box can be expressed as OBB(x obb ,y obb ,w obb ,h obb ,θ obb ), where (x obb ,y obb ) is the center coordinate of the OBB rotation detection frame, w obb is the long side of the OBB rotation detection box, h obb With the long side w obb The adjacent sides can also be called short sides, θ obb is the positive direction of the x-axis and the long side w obb The angle between θ∈[-π / 2,π / 2];

[0099] The improved Yolov8-obb ​​network model structure is as follows Figure 6 As shown in the figure. By improving the C2f module in the original Yolov8-obb ​​network to the C3-BiFormer module, the network's ability to learn image features can be more effectively enhanced; by improving the SPPF module in the original Yolov8-obb ​​network to the SPPFCSPC module, the network's reasoning speed can be more effectively improved; the original loss function CIoU is improved to SIoU with angle loss, and the improved loss function can better describe the regression of rotated targets.

[0100] In the Yolov8-obb ​​network, the backbone network uses Conv2d and c2f modules for downsampling, which will lead to a certain loss of contextual features, resulting in reduced feature map resolution and loss of spatial information of blade surface defects. In order to reduce the loss of small target information during downsampling, the present invention replaces the C2f module in the original network by fusing the C3 module with the BiFormer attention mechanism to enhance the network's attention to small target features. The C3-BiFoemer module structure is as follows: Figure 7 Compared with the convolutional block attention module and other attention mechanisms, the adaptive attention mechanism BiFormer can dynamically adjust the weights of channels and structures, while reducing the number of calculation parameters and enhancing the feature extraction effect of small targets. It is suitable for the case of small detection targets such as wind turbine blade crack detection.

[0101] The SPPF module in the Yolov8-obb ​​network uses pyramid pooling to pool the input feature map and obtain a series of sub-feature maps corresponding to different pooling sizes. These sub-feature maps are combined to produce a larger feature map. This pyramid pooling operation enables the algorithm to capture contextual information at different scales while minimizing information loss and computation. The present invention replaces the SPPF module in the Yolov8-obb ​​network with the SPPFCSPC module, and introduces multiple convolution and cascade techniques to extract and fuse features of different scales, thereby expanding the model receptive field and improving the model accuracy. It also further improves the model performance while maintaining its efficiency. The schematic diagram of the SPPFCSPC module is shown in the figure. Figure 8 shown.

[0102] The SPP FCSPC module integrates two sub-modules: SPP and fully connected spatial pyramid convolution (FCSPC). As a pooling layer, SPP can process input feature maps of different scales and effectively detect small and large objects. Fully connected spatial pyramid convolution (FCSPC) is an improved convolution layer designed to optimize the representation of feature maps to improve detection performance. By performing multi-scale spatial pyramid pooling on the input feature map, the SPP module captures information about objects and scenes at different scales. Subsequently, the FCSPC module convolves the feature maps of different scales output by the SPP module and divides the input feature maps into blocks. These blocks are then merged and connected, and then convolved, so that the receptive field of the model is enhanced while retaining key feature information, thereby improving the accuracy of the model.

[0103] The SPPFCSPC module is an optimization of the SPPCSPC module based on the SPPF concept. It reduces the computational requirements of the pooling layer output by connecting three independent pooling operations and improves the speed and detection accuracy of dense targets without changing the receptive field. The results produced by using this pooling method are comparable to those obtained using larger pooling kernels, thereby optimizing the training and inference speed of the model. The calculation formula is as follows:

[0104]

[0105] S 4 =S 1 *S 2 *S 3

[0106] Among them, R represents the input feature layer, S 1 (R) represents the pooling layer result of the minimum pooling kernel, p represents the padding value of the pooling operation, k represents the size of the pooling kernel of the pooling operation, Indicates that the input feature layer is pooled with a padding value of 2 and a pooling kernel size of 5. 2 (S 1 ) represents the pooling layer result of a medium-sized pooling kernel, Indicates that the pooling operation with a fill value of 2 and a pooling kernel size of 5 is performed on the pooling layer result of the minimum pooling kernel. 3 (S 2 ) represents the pooling layer result of the maximum pooling kernel, Indicates that the pooling operation with a fill value of 2 and a pooling kernel size of 5 is performed on the pooling layer result of the medium-sized pooling kernel. 4 Indicates the final output result.

[0107] In step 3, during network training, the improved Yolov8-obb ​​network model uses the SIoU loss function. It takes into account the vector angle between the real detection box and the predicted detection box, and redefines the penalty indicator. Compared with other loss functions, the SIoU loss function takes into account the vector angle of the expected regression and has a better effect on rotated target detection. The SIoU loss function is mainly composed of 4 Cost functions: ①, Angle cost; ②, Distance cost; ③, Shapecost; ④, IoU cost. The calculation formula is as follows:

[0108] ①、Angle cost:

[0109]

[0110] In the above formula, Λ represents the calculation result of Anglecost, α represents the angle between the center point of the real detection box and the predicted detection box, x represents the sine function value of the angle α, and c h represents the height difference between the center point of the real detection box and the predicted detection box, σ represents the horizontal distance between the center point of the real detection box and the predicted detection box, Indicates the ordinate of the center point of the real detection box, b cy Represents the ordinate of the center point of the predicted detection box, express With b cy The maximum value between express With b cy The minimum value between Indicates the horizontal coordinate of the center point of the real detection frame; b cx Represents the horizontal coordinate of the center point of the predicted detection box.

[0111] ②、Distance cost:

[0112]

[0113] In the above formula, Δ represents the calculation result of Distance cost. It means to calculate and sum the cases of t=x or t=y respectively, x represents the horizontal direction, y represents the vertical direction, γ represents the calculation parameter, ρ t It means that the case of t = x or t = y is substituted into the calculation, e (-γρt) Represents -γρ t Perform exponential function operation, ρ x Represents the calculation result for the horizontal direction, ρ y Indicates the calculation result for the vertical direction, w cIndicates the width of the minimum bounding rectangle of the real detection box and the predicted detection box, h c Indicates the height of the minimum bounding rectangle of the real detection box and the predicted detection box.

[0114] ③、Shape cost:

[0115]

[0116] In the above formula, Ω represents the calculation result of Shapecost, w represents the width of the predicted detection box, and h represents the height of the predicted detection box. It means to calculate and sum the cases of t=w or t=h, ω t represents the value of ω when t = w or t = h, θ represents the degree of concern for shape loss, which is generally between [2, 6], ω w represents the calculation result in the width direction, ω h Indicates the calculation result in the height direction, w gt Indicates the width of the real detection box; h gt Indicates the height of the real detection box; max(w,w gt ) represents w and w gt The maximum value between max(h,h gt ) represents h and h gt The maximum value between .

[0117] ④, IoU cost:

[0118]

[0119] In the above formula, A represents the real detection box, B represents the predicted detection box, A∩B represents the intersection of A and B, and A∪B represents the union of A and B.

[0120] In summary, the expression of SIoU loss function is:

[0121]

[0122] Repeat the training for multiple rounds, and finally select the weight with the highest accuracy on the test set to save. During the training process, the training parameter setting will generate a weight file that can be used for subsequent detection tasks every ten rounds of training. In order to make the subsequent detection accuracy higher, each generated weight will be tested in the test set to verify the correctness of the detection, and then the weight accuracy p will be obtained. Among them, TP is a correctly detected positive result, and FP is an erroneously detected positive result. Finally, the weight with the highest accuracy is selected for subsequent defect detection work.

[0123] In step 4, during the process of detecting crack defects on the surface of the wind turbine blade, the visual module, that is, the camera carried by the drone, is used to record the surface status of the wind turbine blade, and after detection by the processing module, the detection result is obtained.

[0124] In step 5, according to the detection result obtained in step 4, the surface state image of the wind turbine blade with crack defects is statistically analyzed to obtain a detection report.

[0125] In order to prove the accuracy of the crack defect detection method of the present invention for detecting crack defects on the surface of fan blades, two images of fan blades with defects are detected below.

[0126] The surface state images of the wind turbine blades collected in this embodiment are shown in Figures 9(a) and 9(b), where both Figures 9(a) and 9(b) have crack defects, and there is more than one defect. Figures 10(a) and 10(b) are schematic diagrams of crack defect detection results. It can be seen from Figures 10(a) and 10(b) that the method of the present invention can effectively fit the crack defect position on the surface of the wind turbine blade with high accuracy.

Claims

1. A fan blade crack defect detection system based on improved Yolov8-obb ​​is characterized by: The system includes: A visual module is used to record the surface status of wind turbine blades; The transceiver module is responsible for the communication and information transmission between the visual module and the processing module; The processing module is used to control the UAV and detect whether there are defects on the surface of the wind turbine blades, and save the detection results.

2. The fan blade crack defect detection method based on the improved Yolov8-obb ​​of the fan blade crack defect detection system as claimed in claim 1 is characterized in that The following steps are involved: Step 1: Collecting image information of the surface state of the fan blade; Step 2: preprocessing the collected fan blade status image; Step 3: Train the improved Yolov8-obb ​​network model and generate weights; Step 4: Detect the fan blade status image.

3. The fan blade crack defect detection method based on improved Yolov8-obb ​​according to claim 2 is characterized in that: In the step 1, image information of the surface state of the fan blades is collected, including normal fan blades and fan blades with crack defects, wherein 3000 images of each type are used as the initial data set.

4. The fan blade crack defect detection method based on the improved Yolov8-obb ​​according to claim 2 is characterized in that: In the step 2, the collected fan blade surface state images are preprocessed, and the final fan blade state data set is obtained by preprocessing the fan blade state images in the initial data set.

5. The fan blade crack defect detection method based on improved Yolov8-obb ​​according to claim 4 is characterized in that: The surface state image of the wind turbine blade is preprocessed using the MSRCR algorithm; based on the MSRCR algorithm, it is first decomposed into images of different scales, and then the image of each scale is enhanced using an adaptive gain function, and finally the image is reconstructed to achieve the enhancement effect; the following steps are included: Step 2.1: Read the original image data and convert the integer to double type using the log function according to the formula s(x,y)=log[S(x,y)]; Step 2.2: According to the formula Obtain the logarithmic domain enhancement result r(x,y); Step 2.3: According to the formula R(x,y)=exp[r(x,y)], the value of r(x,y) is converted from the logarithmic domain to the real number domain R(x,y) through the natural exponential function operation; Step 2.4: Introduce adaptive enhancement function C k (x,y), enhance the image: Step 2.5: According to formula R MSRCR (x,y)=C k (x,y)iR(x,y), get the final output result; In the above formula, S(x,y) represents the original image data, s(x,y) represents the converted image data, and r(x,y) represents the logarithmic domain enhancement result. It means summing the results of taking k from 1 to 3, q k is the weight corresponding to the k value, log represents the logarithmic calculation of the data, exp represents the natural exponential function calculation of the data, R(x,y) represents the real domain enhancement result, C k (x, y) is the adaptive enhancement function, β is the gain constant, α is the nonlinear transformation adjustment coefficient, s k (x, y) represents the image data when k takes the corresponding value, R MSRCR (x,y) is the final output result.

6. The fan blade crack defect detection method based on improved Yolov8-obb ​​according to claim 2 is characterized in that: In step 3, the wind turbine blade status data set is divided into a training set and a test set in a ratio of 8:2, and input into the improved Yolov8-obb ​​network model for training, and finally the weight with the highest accuracy is selected and saved; it is used to perform subsequent defect target detection tasks.

7. The fan blade crack defect detection method based on improved Yolov8-obb ​​according to claim 6 is characterized in that: In step 3, the OBB (Oriented Bounding Box) used by the Yolov8-obb ​​network is a rotation detection box. The Yolov8-obb ​​algorithm consists of 5 parameters, and its target detection box is represented by OBB(x obb ,y obb ,w obb ,h obb ,θ obb ), where (x obb ,y obb ) is the center coordinate of the OBB rotation detection frame, w obb is the long side of the OBB rotation detection box, h obb With the long side w obb The adjacent sides can also be called short sides, θ obb is the positive direction of the x-axis and the long side w obb The angle between θ∈[-π / 2,π / 2].

8. The fan blade crack defect detection method based on improved Yolov8-obb ​​according to claim 7 is characterized in that: Improved Yolov8-obb ​​network model structure, improves the C2f module in the original Yolov8-obb ​​network to the C3-BiFormer module; improves the SPPF module in the original Yolov8-obb ​​network to the SPPFCSPC module; improves the original loss function CIoU to SIoU with angle loss; In the Yolov8-obb ​​network, the C3 module is fused with the BiFormer attention mechanism to replace the C2f module in the original network to enhance the network's attention to small target features. The SPPF module in the Yolov8-obb ​​network is replaced by the SPPFCSPC module, and multiple convolution and cascade techniques are introduced to extract and fuse features of different scales, thereby expanding the model receptive field.

9. The fan blade crack defect detection method based on improved Yolov8-obb ​​according to claim 8 is characterized in that: The SPPFCSPC module integrates two sub-modules: SPP and fully connected spatial pyramid convolution (FCSPC); SPP acts as a pooling layer to process input feature maps of different scales; Fully connected spatial pyramid convolution (FCSPC) is an improved convolution layer used to optimize the representation of feature maps; By performing multi-scale spatial pyramid pooling on the input feature map, the SPP module captures information about objects and scenes at different scales; Subsequently, the FCSPC module convolves the feature maps of different scales output by the SPP module and divides the input feature maps into blocks. These blocks are then merged and connected, and then convolved to enhance the model receptive field while retaining key feature information, thereby improving the model accuracy. The calculation formula of SPPFCSPC module is as follows: S4=S1*S2*S3 Among them, R represents the input feature layer, S1(R) represents the pooling layer result of the minimum pooling kernel, p represents the padding value of the pooling operation, and k represents the size of the pooling kernel of the pooling operation. It indicates that the input feature layer is pooled with a fill value of 2 and a pooling kernel size of 5. S2(S1) indicates the pooling layer result of a medium-sized pooling kernel. It means that the pooling layer result of the minimum pooling kernel is filled with a value of 2 and a pooling kernel size of 5. S3(S2) means the pooling layer result of the maximum pooling kernel. It means that the pooling operation with a filling value of 2 and a pooling kernel size of 5 is performed on the pooling layer result of the medium-sized pooling kernel, and S4 represents the final output result.

10. The fan blade crack defect detection method based on improved Yolov8-obb ​​according to claim 1 is characterized in that: In step 3, during network training, the improved Yolov8-obb ​​network model uses the SIoU loss function; it takes into account the vector angle between the real detection box and the predicted detection box, and redefines the penalty index; the SIoU loss function consists of 4 Cost functions: ①, Angle cost; ②, Distance cost; ③, Shape cost; ④, IoU cost; the calculation formula is as follows: ①、Angle cost: In the above formula, Λ represents the calculation result of Anglecost, α represents the angle between the center point of the real detection box and the predicted detection box, x represents the sine function value of the angle α, and c h represents the height difference between the center point of the real detection box and the predicted detection box, σ represents the horizontal distance between the center point of the real detection box and the predicted detection box, Indicates the ordinate of the center point of the real detection box, b cy Represents the ordinate of the center point of the predicted detection box, express With b cy The maximum value between express With b cy The minimum value between Indicates the horizontal coordinate of the center point of the real detection frame; b cx Represents the horizontal coordinate of the center point of the predicted detection box; ②、Distance cost: In the above formula, Δ represents the calculation result of Distance cost. It means to calculate and sum the cases of t=x or t=y respectively, x represents the horizontal direction, y represents the vertical direction, γ represents the calculation parameter, ρ t It means that the case of t=x or t=y is substituted into the calculation, Represents -γρ t Perform exponential function operation, ρ x Represents the calculation result for the horizontal direction, ρ y Indicates the calculation result for the vertical direction, w c Indicates the width of the minimum bounding rectangle of the real detection box and the predicted detection box, h c Indicates the height of the minimum bounding rectangle of the real detection box and the predicted detection box; ③、Shape cost: In the above formula, Ω represents the calculation result of Shapecost, w represents the width of the predicted detection box, and h represents the height of the predicted detection box. It means to calculate and sum the cases of t=w or t=h, ω t represents the value of ω when t = w or t = h, θ represents the degree of concern for shape loss, which is generally between [2, 6], ω w represents the calculation result in the width direction, ω h Indicates the calculation result in the height direction, w gt Indicates the width of the real detection box; h gt Indicates the height of the real detection box; max(w,w gt ) represents w and w gt The maximum value between max(h,h gt ) represents h and h gt The maximum value between ④, IoU cost: In the above formula, A represents the real detection box, B represents the predicted detection box, A∩B represents the intersection of A and B, and A∪B represents the union of A and B; In summary, the expression of SIoU loss function is: Repeat the training for multiple rounds, and finally select the weight with the highest accuracy on the test set to save; during the training process, the training parameter setting will generate a weight file that can be used for subsequent detection tasks every ten rounds of training. In order to make the subsequent detection accuracy higher, each generated weight will be tested in the test set to verify the correctness of the detection, and then the weight accuracy p is obtained. Among them, TP means a positive result is detected correctly, and FP means a positive result is detected incorrectly; finally, the weight with the highest accuracy is selected for subsequent defect detection work.

Images