Fan blade defect detection method based on residual feature fusion network

Abstract

The invention provides a fan blade defect detection method based on a residual feature fusion network, and the method comprises the steps: 1, obtaining a multi-target data set and a multi-classification data set of a blade defect of a wind driven generator, and dividing the multi-target data set and the multi-classification data set into a training set, a verification set and a test set; step 2, building a wind driven generator blade defect detection model based on a residual feature fusion network RFFNet; 3, inputting the wind driven generator blade defect images in the training set and the verification set into a wind driven generator blade defect detection model for training to obtain an optimal parameter model; and step 4, inputting the test set into the optimal parameter model trained in the step 3, and outputting an accurate identification graph of the wind driven generator blade defect image. The network provided by the invention has remarkable advantages in the aspects of detecting the defects of the blades of the wind driven generator, improving false detection and missing detection and the like, effectively solves the problems existing in complex environments and multi-target defect detection, and has relatively high detection speed.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection, and in particular to a fan blade defect detection method based on a residual feature fusion network. Background Art

[0002] As wind power gradually rises in the global energy structure, the research and application of defect detection has also received widespread attention. The necessity of surface defect detection of wind turbine blades is reflected in many aspects, including ensuring safety, improving economic benefits, protecting the environment, complying with regulations and standards, and promoting technological progress. Through regular and effective defect detection, the service life of wind power equipment can be extended, maintenance costs can be reduced, power generation efficiency can be improved, and the reliability and sustainability of wind power generation as a green energy can be ensured.

[0003] In recent years, deep learning has been closely integrated with the field of industrial inspection. Visible light defect imaging detection based on drone platforms has the advantages of low cost and fast detection speed. One of the key technologies for defect detection is to use convolutional neural network (CNN) to identify defects. Mao Yulin used the cascaded R-CNN feature extraction network based on ResNet-101 as the detection model, and introduced the idea of ​​transfer learning when detecting defects in wind turbine blades to enable the model to converge faster. However, after the introduction of the ResNet network, the training was not stable enough in the later stage, which easily caused image distortion. Qiu et al. combined the YOLO and CNN models, applied the inverse convolutional neural network to the high-level features of the feature pyramid, used the middle layer of CNN to enrich the feature expression of small targets, and trained the extracted multi-scale convolution features in the classification model. The detection accuracy of various defects on the surface of the wind turbine reached more than 80%. Although the existing technology has achieved certain results in the detection of defects in wind turbine blades, the detection effect of the defect detection equipment of wind turbine blades is not ideal in the face of complex backgrounds and small target defects. Summary of the invention

[0004] Purpose of the invention: The technical problem to be solved by the present invention is to provide a fan blade defect detection method based on a residual feature fusion network in view of the shortcomings of the prior art. The present invention can well solve the problems of false detection and missed detection, and has a faster detection speed to meet the application requirements in actual detection environments.

[0005] The present invention comprises the following steps: Step 1, obtaining a multi-target dataset and a multi-classification dataset of wind turbine blade defects, wherein the multi-target dataset and the multi-classification dataset comprise an original image and a corresponding label image, pre-processing the multi-target dataset and the multi-classification dataset, and then dividing them into a training set, a verification set and a test set respectively according to a set ratio.

[0006] Step 2, build a wind turbine blade defect detection model based on the residual feature fusion network RFFNet, the model includes an input end, a backbone network, a neck network Neck and a detection head head, wherein the backbone network and the neck network Neck collaboratively process features; the detection head head includes a re-parameterized convolution RepConv and a final detection head Head.

[0007] Step 3, respectively input the wind turbine blade defect images in the training set and validation set obtained by dividing the multi-objective dataset and the multi-classification dataset into the wind turbine blade defect detection model for training, calculate the loss function and perform back propagation, update the network parameters, and obtain the optimal parameter model of the multi-objective dataset and the optimal parameter model of the multi-classification dataset.

[0008] Step 4, respectively input the wind turbine blade defect images in the test set obtained by dividing the multi-objective dataset and the multi-classification dataset into the multi-objective dataset optimal parameter model and the multi-classification dataset optimal parameter model, and output an accurate recognition map of the wind turbine blade defect images.

[0009] Step 1 includes: obtaining a multi-target dataset and a multi-classification dataset of wind turbine blade defects, as well as corresponding defect location labels, and the data labels are in a text file txt format.

[0010] Considering the depth of the image pyramid (Feature PyramidNetworks) required for the wind turbine blade defect detection task, the images contained in the multi-target dataset and the multi-classification dataset are preprocessed to a uniform size of 640×640. In the preprocessing operation, the image size is unified by reshaping the image. On the one hand, the model can process images of different sizes, and on the other hand, it is convenient for the model to perform mosaic data enhancement, stitching multiple different images into a large image, and mapping the label boxes to generate new training samples. The specific workflow is as follows: (1) Randomly select four different images as input. These four images will be spliced ​​into a large image; (2) Randomly select a center point of a large image, and then place the four small images in the four quadrants of the center point of the large image; (3) Adjust the position coordinates of the target box in each small image according to the position of the center point of the large image. That is, map the coordinates of the target box to the large image; (4) The large image is cropped into a fixed size to facilitate the input of training samples into the model constructed by the present invention.

[0011] The multi-target dataset and the multi-classification dataset are randomly divided into three parts using computer random seeds, of which the training set accounts for 70%, the validation set accounts for 20%, and the test set accounts for 10%. The multi-target dataset and the multi-classification dataset are trained and tested respectively. The multi-target dataset used in the present invention is an image of a wind turbine of the Yellow River Wind Power Company taken during an unmanned aerial vehicle inspection. It contains single-category damage (damage) uniformly annotated at different locations such as blades and hubs. The images in this dataset have single-target and multi-target damage defects. In the present invention, it is used to test the single-class multi-target detection capability of the model. The image size is 586×371, with a total of 2191 images and 6480 defect targets. This dataset provides a complex and real dataset for wind turbine blade defect detection in natural environments.

[0012] The multi-classification dataset is composed of wind turbine defect images taken by drones and some outdoor scene drone images provided by the Roboflow website. The Roboflow shooting environment is good and the images have good pixels. It contains 3584 pictures with a size of 1280×1080, which are divided into five categories: oil leakage, dirt, erosion, PU gel coat damage, and paint peeling, with a total of 5371 defects.

[0013] Step 2 includes the following steps: Step 2.1, inputting the training set and the validation set obtained by dividing the multi-target data set and the multi-classification data set into the wind turbine blade defect detection model for training.

[0014] In step 2.2, at the end of the backbone network, feature maps of different scales are obtained by using a pooling pyramid, which helps to detect objects at different scales and extract more expressive features.

[0015] Step 2.3, after step 2.2, there are three output feature maps at the output ends of the neck network Neck corresponding to the inputs of three detection heads of three different sizes, and the final detection heads Head of the three detection heads output feature maps respectively.

[0016] In step 2.4, the minimum parameter distance intersection over union (MPDIoU) is used to calculate the loss.

[0017] Step 2.5, the original input image is input into the backbone network after the preprocessing operation in step 1, so as to realize the large, medium and small scale prediction of fan defects.

[0018] Step 2.1 includes: in a model training process, the images in the training set and the validation set obtained after dividing the multi-target data set are preprocessed as input images and input into the backbone network, the backbone network includes 4 convolution modules CBS, 4 efficient multi-level residual modules (Efficient Channel Attention and Multi level residual Convolutionalmodule, ECAMC) based on the improved efficient channel attention mechanism ECA (Efficient Channel Attention) and 3 downsampling convolution MPConv (MaxpoolingConv) modules, the convolution module CBS includes a convolution layer Conv, normalization and SiLU activation function, the convolution module CBS is used to perform preliminary extraction of image features and size transformation of the image, wherein the convolution kernel size of the first convolution module CBS and the third convolution module CBS are both 3, the step size is 1, the convolution kernel size of the second convolution module CBS and the fourth convolution module CBS is 3, the step size is 2, after the input image passes through the 4 convolution modules CBS, the convolution operation outputs a feature size of 160×160×128. Figure X 1. Then input an efficient multi-level residual module, wherein the efficient multi-level residual module includes efficient channel attention with maximum pooling convolution (ECAM), conventional convolution, feature map concatenation Concat, normalization and SiLU activation function, and the four efficient multi-level residual modules have the same structure.

[0019] A downsampling convolution MPConv module is used to adjust the size of the feature map to facilitate the detection of defect targets of different sizes. The downsampling convolution MPConv module includes a maximum pooling layer, two convolution modules CBS with a convolution kernel size of 1×1 and a convolution step size of 1, and a convolution module CBS with a convolution kernel size of 3×3 and a convolution step size of 2.

[0020] The image to be detected obtains multi-scale features through each efficient multi-level residual module in the backbone network, and then the multi-scale features are input into the neck network Neck.

[0021] The efficient multi-level residual module is composed of a stepped residual structure consisting of maximum pooling efficient channel attention ECAM, conventional convolution, Concat, normalization and SiLU activation function. Each efficient multi-level residual module includes an 8-layer convolution structure and a two-layer splicing structure. The residual structure used inside the efficient multi-level residual module includes a layer of conventional convolution with a convolution kernel size of 1×1 and a layer of maximum pooling efficient channel attention ECAM. The convolution kernel size of all convolutions except the first layer in the efficient multi-level residual module is 3×3.

[0022] The efficient multi-level residual module obtains image features at different levels through level-by-level convolution, and finally combines the image features at different levels within the module by using the residual structure in the splicing layer to obtain a new feature map; the results of the first and third convolution layers in the efficient multi-level residual module are spliced ​​and input into the fourth convolution layer. This structure increases the proportion of original information in the shallow network, fully extracts and fuses the feature outputs of each level, and enriches the output features; in addition, the maximum pooling efficient channel attention ECAM is used in the residual connection part of the efficient multi-level residual module. The maximum pooling efficient channel attention ECAM uses global average pooling and global maximum pooling in parallel to extract features, and uses a one-dimensional convolution kernel of size k to generate channel weights, and uses a 1×1 convolution operation with a Sigmoid activation function to obtain channel attention information, and combines the channel attention information with the feature Figure X The initial feature information contained in 1 is point-multiplied to obtain a specific channel attention feature map; finally, the feature maps of different levels are fused in the Concat layer to obtain the output feature map of the efficient multi-level residual module.

[0023] Step 2.2 includes: at the end of the backbone network, the output feature map of the fourth efficient multi-level residual module is used as the input of the cross-level spatial pyramid pooling module SPPCSPC of the neck network Neck: the cross-level spatial pyramid pooling module SPPCSPC is composed of 5 convolution modules CBS with a convolution kernel size of 1, two convolution modules CBS with a convolution kernel size of 3, a maximum pooling layer with a convolution kernel size of 5, a maximum pooling layer with a convolution kernel size of 9, and a maximum pooling layer with a convolution kernel size of 13. By using the pooling pyramid, feature maps of different scales are obtained, which is helpful to detect objects at different scales and extract more expressive features.

[0024] The neck network Neck includes 1 cross-level spatial pyramid pooling module SPPCSPC, 1 secondary residual feature fusion module SRGS, 2 downsampling convolution MPConv modules, 2 upsamplings, 3 new generation convolution deformation modules CNT, 4 Concat operations and 2 convolution modules CBS. The output feature maps of the second and third efficient multi-level residual modules of the backbone network pass through a convolution module CBS and an upsampling module in the neck network Neck respectively. The convolution module CBS adjusts the number of channels through convolution operations. The upsampling module uses the nearest neighbor difference method to adjust the feature map size, and then inputs it into the neck network Neck layer by layer for multi-scale feature fusion to reduce the loss in the feature transmission process. The upsampling module is UPsample.

[0025] The new generation convolution deformation module CNT includes a ConvNext module and three convolution modules CBS, wherein the ConvNext module includes depth-separable convolution, layer normalization, CBS convolution, GELU activation function and LayerScale layer.

[0026] The SRGS (Secondary residual Ghost Squeeze and convolution block) module is used to further fuse the feature information extracted by the backbone network and suppress irrelevant information. The secondary residual feature fusion module SRGS adopts a secondary residual connection structure. First, a convolution with a convolution kernel size of 1×1 is used to adjust the number of channels before the feature map enters GSConv.

[0027] The secondary residual feature fusion module SRGS includes 4 GSConvs with a convolution kernel size of 3×3, 2 conventional convolutions, and 2 partial convolutions, which are spliced ​​twice along the channel dimension. Both splicing operations adopt the residual connection method, wherein the residual connection of the first splicing operation is the first residual connection part, and the residual connection of the second splicing operation is the second residual connection part; the first splicing part of the secondary residual feature fusion module SRGS is composed of a feature map after a 1×1 convolution and two GSConvs, a feature map after a 1×1 convolution and a 3×3 size partial convolution PConv, and input data after a 1×1 convolution; the second splicing part of the secondary residual feature fusion module SRGS, i.e., the output part, adopts the fourth splicing operation in the secondary residual feature fusion module. The feature maps of GSConv and the third GSConv after 3×3 partial convolution PConv are concatenated; since there is often a high degree of similarity between different channels of the feature map, partial convolution PConv is introduced in the residual connection part of the secondary residual feature fusion module SRGS, and standard convolution is applied only to 1 / 2 of the data in the input channel, while the remaining 1 / 2 channels are not operated, which reduces redundant calculations and memory accesses while more effectively extracting spatial features; applying partial convolution PConv to the second residual connection part of the secondary residual feature fusion module SRGS can help the module learn information at different levels in the context, improve the expressiveness of the model by reusing shallower feature information operations, and help enhance the model's adaptability to new input data.

[0028] The new generation convolution deformation module CNT includes a first calculation stage and a second calculation stage. In the first calculation stage, the output is the convolution after the ConvNext module and the next layer of 1×1. The advantage of this part is that the output result can be used as the input of the next layer of the network, which minimizes the number of parameters and does not affect the overall operation of the network. Since the input data source of ConvNext is the ConCat layer after the feature extraction network and the shallow network are spliced, this layer contains rich feature information, and the information output by ConvNext is not fully integrated. The second calculation stage is the prediction part. After the ConvNext module, the convolution module CBS is used twice to perform feature fusion while adjusting the number of channels, avoiding the problem of increased calculation amount caused by sending the spliced ​​feature map to the re-parameterized convolution RepConv, which is convenient for the final detection head to perform prediction operations.

[0029] The ConvNext module includes four layers of operations: the first layer performs a 7×7 depth-separable convolution on the input feature map, uses a step size of 1, and pads 3 to ensure the size of the feature map. Then, a layer normalization operation is used to stabilize the training process of the model. The second layer uses a 1×1 convolution kernel and a GELU activation function to integrate information and adjust the number of feature map channels to four times the original number. The third layer first uses a 1×1 convolution operation and then uses a scaling layer LayerScale to solve the problems of unstable network depth and excessive computation. The fourth layer uses a random drop layer Drop Path for regularization to prevent overfitting. Finally, the output layer feature map size is adjusted back to h×w×c; h represents the height of the image and w represents the width of the image.

[0030] Step 2.3 includes: after step 2.2, there are three output feature maps in the neck network Neck corresponding to the inputs of three detection heads of three different sizes respectively, the detection heads include reparameterized convolution RepConv and final head detection Head, when the images in the training set and validation set obtained after dividing the multi-target data set are processed by the backbone network and the neck network Neck, the output feature maps will be input into the detection head for prediction, the reparameterized convolution RepConv includes convolution and normalization; the final detection heads Head of the three detection heads output feature maps of sizes 20×20×255, 40×40×255, and 80×80×255 respectively.

[0031] Step 2.4 includes: using the minimum parameter distance intersection-over-union ratio MPDIoU to calculate the loss, the formula is: (1) (2) (3) (4) (5) (6) (7) (8) (9) (10)

[0032] in , Respectively represent the horizontal and vertical coordinates of the upper left corner of the target to be detected in the image, , Respectively represent the horizontal and vertical coordinates of the lower right corner of the target to be detected in the image; , Respectively represent the coordinates of the predicted target in the upper left corner of the image; , They represent the coordinates of the predicted target in the lower right corner of the image respectively; d1 represents the distance between the target to be detected and the upper left corner coordinates of the predicted target, and d2 represents the distance between the target to be detected and the lower right corner coordinates of the predicted target; represents the area of ​​the real box, represents the area of ​​the real box, Represents the larger horizontal coordinate of the upper left corner of the detection frame and the upper left corner of the real frame. Represents the larger ordinate between the upper left corner of the detection frame and the upper left corner of the real frame. Represents the smaller horizontal coordinate of the lower right corner of the detection frame and the lower right corner of the real frame. The smaller ordinate represents the ordinate of the lower right corner of the detection frame and the lower right corner of the real frame. The detection frame is a square frame with an upper left corner, an upper right corner, a lower left corner, and a lower right corner. The same is true for the prediction frame. The larger ordinate of the upper left corner of the detection frame and the real frame refers to which one is larger. x represents the abscissa and y represents the ordinate. For example, if the detection frame coordinates are (2,5) and the real frame coordinates are (3,4), then x is 3 and y is 5.

[0033] represents the overlapping area, Represents the total area occupied by the detection frame and the real frame including the overlapping part. Iou represents the result of dividing the overlapping part of the two regions by the combined part of the two regions. The loss represented by the minimum parameter distance intersection over union (IoU) ratio used in the present invention for calculation is that in the wind turbine defect detection task, different types of defects often have irregular shapes. When the aspect ratios of the bounding boxes of defective targets differ greatly, the traditional IoU loss may be difficult to effectively deal with this situation. The minimum parameter distance intersection over union (MPDIoU) uses the distance between the upper left and lower right corners of the actual annotated bounding box and the model-predicted bounding box to measure similarity, which simplifies the comparison process between bounding boxes and makes the model more sensitive to the responsiveness of the bounding box shape adjustment, especially when dealing with highly overlapping or partially overlapping bounding boxes. It is more effective.

[0034] Step 2.5 includes: inputting the original input image into the backbone network after the preprocessing operation of step 1, and after passing through four convolution modules CBS, the first efficient multi-level residual module is called the first stage, the second efficient multi-level residual module is called the second stage, the third efficient multi-level residual module is called the third stage, and the fourth efficient multi-level residual module is called the fourth stage, wherein the output of the fourth stage is used as the input of the cross-level spatial pyramid pooling module SPPCSPC of the neck network Neck, and after passing through the cross-level spatial pyramid pooling module SPPCSPC, the image size is transformed through the convolution module CBS and upsampling operation, and feature extraction is performed, and the output of the third stage is subjected to a convolution with a convolution kernel size of 1×1 and then outputs the feature, which is fused, which is called the first fusion, and then passes through the secondary residual feature fusion module SRGS of the neck network Neck, and then passes through two convolution modules CBS with a convolution kernel size of 1×1 and the feature map after upsampling operation, and the output feature map of the second stage is fused through the convolution module CBS with a convolution kernel size of 1×1, which is called For the second fusion, the feature map output by the first new generation convolution deformation module CNT is used to realize the prediction of large-scale targets of fan defects after re-parameterized convolution RepConv. The feature map output by the first new generation convolution deformation module CNT and an MP module is fused with the feature map output by a secondary residual feature fusion module SRGS and a convolution module CBS to obtain a new feature map and input it into the second new generation convolution deformation module CNT. The feature map output by the second new generation convolution deformation module CNT is used to realize the medium-scale prediction of fan defects after re-parameterized convolution RepConv. At the same time, the feature map output by the second new generation convolution deformation module CNT is fused with the feature map output by the second MP module and SPPCSPC module and enters the third new generation convolution deformation module CNT. The feature map output by the third new generation convolution deformation module CNT is used to realize the small- and medium-scale prediction of fan defects after re-parameterized convolution RepConv. The accuracy and loss are calculated for each round of training.

[0035] The present invention also provides an electronic device, comprising a processor and a memory, wherein the memory stores program code, and when the program code is executed by the processor, the processor executes the steps of the described method.

[0036] The present invention also provides a storage medium storing a computer program or instruction. When the computer program or instruction is run on a computer, the steps of the method described are executed.

[0037] Beneficial effects: The present invention obtains image features at different levels through step-by-step convolution, constructs an efficient convolution module ECAMC as a backbone feature extraction module, introduces the ECAM attention mechanism, and adjusts the residual connection mode so that the network can fuse features at different levels and obtain information at different scales. The backbone network and the neck network Neck connection part use a secondary residual feature fusion module SRGS to further fuse the feature information extracted by SPPCSP and suppress irrelevant information. A new generation of convolution deformation module CNT is introduced in the output part of the neck layer. The ConvNext module used inside the module adopts a variety of feature fusion, deep separable convolution and other technologies to make the model more flexible and further improve the detection efficiency. The proposed network has significant advantages in detecting defects in wind turbine blades and improving false detections and missed detections. It can effectively deal with problems existing in complex environments and multi-target defect detection, and has a faster detection speed, which can be used to meet application requirements in actual detection environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments, and the above and / or other advantages of the present invention will become more clear.

[0039] Figure 1 A schematic diagram of a flow chart provided for an embodiment.

[0040] Figure 2 The overall network structure diagram of the residual feature fusion network provided in the embodiment.

[0041] Figure 3 A structural diagram of an efficient multi-stage residual module provided in an embodiment.

[0042] Figure 4 Improved efficient channel attention structure diagram provided for embodiments.

[0043] Figure 5 A structural diagram of a secondary residual feature fusion module provided in an embodiment.

[0044] Figure 6 A structural diagram of a new generation of convolution provided for an embodiment.

[0045] Figure 7 A structural diagram of a new generation convolution deformation module provided for an embodiment.

[0046] Figure 8 A schematic diagram of calculating the minimum parameter distance intersection-over-union ratio MPDIoU provided in the embodiment.

[0047] Fig. 9 , Fig.10 , Fig.11 , Fig.12 , Fig.13 , Fig.14 This is a graph showing comparative experimental results of the detection method provided in the embodiment and other mainstream methods.

[0048] Fig.15 , Fig.17 , Fig.19 , Fig.21 , Fig.23 This is a diagram of the detection results of the detection method provided in the embodiment.

[0049] Fig.16 , Fig.18 , Fig. 20 , Fig. 22 , Fig.24 This is the detection result diagram of the YOLOv7 detection method. DETAILED DESCRIPTION

[0050] An embodiment of the present invention provides a method for detecting defects in wind turbine blades based on a residual feature fusion network, comprising the following steps: Step 1, obtaining a multi-target dataset and a multi-classification dataset of defects in wind turbine blades, wherein the multi-target dataset and the multi-classification dataset include original images and corresponding label images, pre-processing the multi-target dataset and the multi-classification dataset, and then dividing them into a training set, a validation set, and a test set according to set ratios.

[0051] Step 2, build a wind turbine blade defect detection model based on the residual feature fusion network RFFNet, the model includes an input end, a backbone network, a neck network Neck and a detection head head, wherein the backbone network and the neck network Neck collaboratively process features; the detection head head includes a re-parameterized convolution RepConv and a final detection head Head.

[0052] Figure 2 It is the overall structure diagram of the residual feature fusion network of the present invention, where C1, C2, C3, C4 are ECAMC modules, MP is a downsampling module, M1 and M2 represent the feature maps output by the second and third ECAMC modules respectively, and the network includes a SPPCSP module, 3 new generation convolution deformation modules CNT, a secondary residual feature fusion module SRGS, 3 re-parameterized convolution RepConv modules, and 2 Upsample upsampling modules.

[0053] Step 3, respectively input the wind turbine blade defect images in the training set and validation set obtained by dividing the multi-objective dataset and the multi-classification dataset into the wind turbine blade defect detection model for training, calculate the loss function and perform back propagation, update the network parameters, and obtain the optimal parameter model of the multi-objective dataset and the optimal parameter model of the multi-classification dataset.

[0054] Step 4, respectively input the wind turbine blade defect images in the test set obtained by dividing the multi-objective dataset and the multi-classification dataset into the multi-objective dataset optimal parameter model and the multi-classification dataset optimal parameter model, and output an accurate recognition map of the wind turbine blade defect images.

[0055] Step 1 includes: obtaining a multi-target dataset and a multi-classification dataset of wind turbine blade defects, as well as corresponding defect location labels, and the data labels are in a text file txt format.

[0056] Considering the depth of the image pyramid (Feature PyramidNetworks) required for the wind turbine blade defect detection task, the images contained in the multi-target dataset and the multi-classification dataset are preprocessed to a uniform size of 640×640. In the preprocessing operation, the image size is unified by reshaping the image. On the one hand, the model can process images of different sizes, and on the other hand, it is convenient for the model to perform mosaic data enhancement, stitching multiple different images into a large image, and mapping the label boxes to generate new training samples. The specific workflow is as follows: (1) Randomly select four different images as input. These four images will be spliced ​​into a large image; (2) Randomly select a center point of a large image, and then place the four small images in the four quadrants of the center point of the large image; (3) Adjust the position coordinates of the target box in each small image according to the position of the center point of the large image. That is, map the coordinates of the target box to the large image; (4) The large image is cropped into a fixed size to facilitate the input of training samples into the model constructed by the present invention.

[0057] The multi-target dataset and the multi-classification dataset are randomly divided into three parts using computer random seeds, of which the training set accounts for 70%, the validation set accounts for 20%, and the test set accounts for 10%. The multi-target dataset and the multi-classification dataset are trained and tested respectively. The multi-target dataset used in the present invention is an image of a wind turbine of the Yellow River Wind Power Company taken during an unmanned aerial vehicle inspection. It contains single-category damage (damage) uniformly annotated at different locations such as blades and hubs. The images in this dataset have single-target and multi-target damage defects. In the present invention, it is used to test the single-class multi-target detection capability of the model. The image size is 586×371, with a total of 2191 images and 6480 defect targets. This dataset provides a complex and real dataset for wind turbine blade defect detection in natural environments.

[0058] Step 2 includes: Step 2.1: In a model training process, the images in the training set and the validation set obtained after dividing the multi-target data set are preprocessed as input images and input into the backbone network, wherein the backbone network includes the initial 4 convolution modules CBS and 4 efficient multi-level residual modules (Efficient ChannelAttention and Multi level residual Convolutional module, ECAMC) based on the improved ECA attention mechanism and 3 downsampling convolution MPConv (MaxpoolingConv) modules.

[0059] The convolution module CBS includes a convolution layer Conv, normalization and SiLU activation function, which is used for the preliminary extraction of image features and the size transformation of the image. The convolution kernel size of the first convolution module CBS and the fourth convolution module CBS is 3, and the step size is 1. The convolution kernel size of the second convolution module CBS and the fourth convolution module CBS is 3, and the step size is 2. At this time, the feature map size becomes 160×160×128, and then a high-efficiency multi-level residual module, namely the ECAMC module: the ECAMC module is a stepped residual structure composed of ECAM attention, conventional convolution, Concat, normalization and SiLU activation function, wherein the convolution kernel size of the first layer of conventional convolution of each branch is one, and the convolution kernel size of other convolutions is 3. The MP module performs a Concat operation on the output through a parallel structure consisting of a maximum pooling layer, two convolution modules CBS with a convolution kernel size of 1 and a convolution module CBS with a convolution kernel size of 3. The input feature map is divided, and the maximum value of each divided area is selected as the output. In the backbone network, the original image undergoes the above operation so that each ECAMC module of the backbone network obtains multi-scale features as the input features of the Neck part.

[0060] Figure 3 The efficient multi-level residual module ECAMC designed for the present invention, Input represents the feature map input to the module, Output represents the feature map output by the module, Conv represents convolution, BN represents layer normalization, and SiLU is a new activation function. In the module, S1 represents the first feature map splicing node, S2 represents the feature information extracted from the attention residual part, and S3 represents the feature information output by the second convolution block.

[0061] Figure 4 is the ECAM attention map, H represents the feature map height, W represents the feature map width, C represents the number of feature map channels, GAP represents global average pooling, and GMP represents global maximum pooling.

[0062] Step 2.2: At the end of the backbone network, the output feature map of the fourth efficient multi-level residual module is used as the input of the cross-level spatial pyramid pooling module SPPCSPC of the neck network Neck: SPPCSPC is composed of 5 convolution modules CBS with a convolution kernel size of 1, two convolution modules CBS with a convolution kernel size of 3, a maximum pooling layer with a convolution kernel size of 5, a maximum pooling layer with a convolution kernel size of 9, and a maximum pooling layer with a convolution kernel size of 13. By using the pooling pyramid, feature maps of different scales can be obtained, which is helpful for detecting objects at different scales and extracting more expressive features.

[0063] The neck network Neck includes 1 cross-level spatial pyramid pooling module SPPCSPC, 1 secondary residual feature fusion module SRGS, 2 downsampling convolution MPConv modules, 2 upsamplings, 3 new generation convolution deformation modules CNT, 4 Concat operations and 2 convolution modules CBS. The output feature maps of the second and third efficient multi-level residual modules of the backbone network pass through a convolution module CBS and an upsampling module in the neck network Neck respectively. The convolution module CBS adjusts the number of channels through convolution operations. The upsampling module uses the nearest neighbor difference method to adjust the feature map size, and then inputs it into the neck network Neck layer by layer for multi-scale feature fusion to reduce the loss in the feature transmission process. The upsampling module is UPsample.

[0064] The secondary residual feature fusion module SRGS consists of 4 GSConv, 2 PConv, and 2 convolution modules CBS. The new generation convolution deformation module CNT consists of a ConvNext module and 3 convolution modules CBS. The ConvNext module consists of depth separable convolution, layer normalization, CBS convolution, GELU activation function, and LayerScale layer.

[0065] Figure 5 It is a two-level residual feature fusion module SRGS, where S1 represents the feature information obtained by adjusting the number of channels of the backbone through conventional convolution and one GSConv, S2 represents the information after the first-level residual connection feature fusion, and S3 represents the feature information output by S2 after one GSConv.

[0066] Figure 7 It is a new generation of convolution deformation module CNT, where Output1 represents the feature information extracted by one convolution module CBS, C1 represents the feature information extracted by two convolution modules CBS, and the output result of Output2 will be input to the head layer for prediction.

[0067] Step 2.3: After step 2.2, there are three output feature maps in the neck network Neck, which correspond to the inputs of three detection heads of three different sizes. The detection heads include reparameterized convolution RepConv and final head detection Head. When the images in the training set and validation set obtained after dividing the multi-target data set are processed by the backbone network and the neck network Neck, the output feature maps will be input into the detection head for prediction. The reparameterized convolution RepConv includes convolution and normalization; the final detection heads Head of the three detection heads output feature maps of sizes 20×20×255, 40×40×255, and 80×80×255, respectively.

[0068] Step 2.4, use the minimum parameter distance intersection and union ratio MPDIoU to calculate the loss, the formula is: (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) in , Respectively represent the horizontal and vertical coordinates of the upper left corner of the target to be detected in the image, , Respectively represent the horizontal and vertical coordinates of the lower right corner of the target to be detected in the image; , Respectively represent the coordinates of the predicted target in the upper left corner of the image; , They represent the coordinates of the predicted target in the lower right corner of the image respectively; d1 represents the distance between the target to be detected and the upper left corner coordinates of the predicted target, and d2 represents the distance between the target to be detected and the lower right corner coordinates of the predicted target; represents the area of ​​the real box, represents the area of ​​the real box, Represents the larger horizontal coordinate of the upper left corner of the detection frame and the upper left corner of the real frame. Represents the larger ordinate between the upper left corner of the detection frame and the upper left corner of the real frame. Represents the smaller horizontal coordinate of the lower right corner of the detection frame and the lower right corner of the real frame. The smaller ordinate represents the ordinate of the lower right corner of the detection frame and the lower right corner of the real frame. The detection frame is a square frame with an upper left corner, an upper right corner, a lower left corner, and a lower right corner. The same is true for the prediction frame. The larger ordinate of the upper left corner of the detection frame and the real frame refers to which one is larger. x represents the abscissa and y represents the ordinate. For example, if the detection frame coordinates are (2,5) and the real frame coordinates are (3,4), then x is 3 and y is 5.

[0069] represents the overlapping area, Represents the total area occupied by the detection frame and the real frame including the overlapping part. Iou represents the result of dividing the overlapping part of the two regions by the combined part of the two regions. The loss represented by the minimum parameter distance intersection over union (IoU) ratio used in the present invention for calculation is that in the wind turbine defect detection task, different types of defects often have irregular shapes. When the aspect ratios of the bounding boxes of defective targets differ greatly, the traditional IoU loss may be difficult to effectively deal with this situation. The minimum parameter distance intersection over union (MPDIoU) uses the distance between the upper left and lower right corners of the actual annotated bounding box and the model-predicted bounding box to measure similarity, which simplifies the comparison process between bounding boxes and makes the model more sensitive to the responsiveness of the bounding box shape adjustment, especially when dealing with highly overlapping or partially overlapping bounding boxes. It is more effective.

[0070] Step 2.5: After deploying steps 2.1, 2.2, 2.3, and 2.4, the original input image is preprocessed in step 1 and input into the backbone network. After passing through four convolution modules CBS, the first ECAMC is called the first stage, the second ECAMC is called the second stage, the third ECAMC is called the third stage, and the fourth ECAMC is called the fourth stage. The output of the fourth stage is used as the input of the SPPCSPC of the neck network Neck. After the pooling pyramid SPPCSPC, its output is transformed into image size and feature extraction through the convolution module CBS and upsampling operation, and the output of the third stage is output after a convolution with a convolution kernel size of 1×1. The features are fused, which is called the first fusion. After passing through the secondary residual feature fusion module SRGS of the neck network Neck, it passes through two convolution modules CBS with a convolution kernel size of 1×1 and upsampling operations, and then is fused with the output of the second stage through the convolution module CBS output with a convolution kernel size of 1, which is called the second fusion. After passing through the first new generation convolution deformation module CNT prediction part and re-parameterized convolution RepConv, large-scale prediction is performed. At the same time, the trunk of the first CNT passes through the MP module and is fused with the convolution module CBS output after the secondary residual feature fusion module SRGS. The output feature map enters the second new generation convolution deformation module CNT, and the prediction part of the second CNT passes through the re-parameterized convolution RepConv for medium-scale prediction. At the same time, the second CNT trunk passes through the second MP module and is fused with the output features of SPPCSPC before entering the third new generation convolution deformation module CNT. The prediction part output of the third new generation convolution deformation module CNT passes through the re-parameterized convolution RepConv for small-scale prediction. After all the above processes, the accuracy and loss of each round of training are calculated.

[0071] The experimental indicators in step 2.5 are accuracy, recall, and mean average precision (mAP). Accuracy refers to the probability of correct detection among all detected defect targets, which is used to measure the accuracy of the model. Recall refers to the proportion of correct predictions made by the model among all defect targets. mAP is a comprehensive consideration of accuracy and recall, which is used to comprehensively evaluate the effectiveness and reliability of the model. When mAP is 0.5, it means that the network model sets the IoU threshold for judging positive and negative samples to 0.5.

[0072] The calculation formula of accuracy P is as follows: , The recall rate R formula is as follows: , The accuracy is expressed as , the calculation formula is: , mAP is expressed as , the calculation formula is: , in, Represents a true positive sample (True Positive), Represents a false positive sample (FalsePositive), Indicates a positive sample that has not been detected (False Negative).

[0073] In the target detection task, IOU is used to measure the overlap between the predicted bounding box (or detection box) and the true bounding box. It measures the close correlation between the detection result and the target by calculating the ratio of the intersection area of ​​two bounding boxes to their union area. In the present invention, the minimum parameter distance intersection-over-union ratio MPDIOU is used to replace the traditional bounding box regression loss, which can further improve the recall rate of the model for detecting defects in wind turbine blades.

[0074] In order to prove the effectiveness of the wind turbine blade defect detection method based on residual feature fusion network provided in this embodiment, the model is trained, verified and tested using multi-target dataset 1 and multi-classification dataset 2.

[0075] from Fig.15 , Fig.17 , Fig.19 , Fig.21 , Fig.23 The detection result diagram of the detection method provided in this embodiment is Fig.16 , Fig.18 , Fig. 20 , Fig. 22 , Fig.24 The YOLOv7 detection method is divided into five categories: oil leakage, dirt, erosion, PU tape, and paint. The detection advantages of this embodiment can be seen from the comparison of the detection effect diagram. Fig. 9 , Fig.10 , Fig.11 , Fig.12 , Fig.13 , Fig.14 It is a mainstream detection model used in defect detection. According to Table 1, each evaluation index of this embodiment is higher than that of the existing detection network and can meet the application requirements in the actual detection environment.

[0076] Table 1

[0077] The present invention provides a method for detecting defects in fan blades based on a residual feature fusion network. There are many methods and approaches to implement the technical solution. The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention. All components not specified in this embodiment can be implemented using existing technologies.

Claims

1. A fan blade defect detection method based on residual feature fusion network, characterized in that: The following steps are involved: Step 1, obtaining a multi-target dataset and a multi-classification dataset of wind turbine blade defects, wherein the multi-target dataset and the multi-classification dataset include original images and corresponding label images, preprocessing the multi-target dataset and the multi-classification dataset, and then dividing them into a training set, a validation set, and a test set according to proportions; Step 2, building a wind turbine blade defect detection model based on the residual feature fusion network RFFNet, the model includes an input end, a backbone network, a neck network Neck and a detection head head, wherein the backbone network and the neck network Neck collaboratively process features; The detection head includes the re-parameterized convolution RepConv and the final detection head Head; Step 3, respectively inputting the wind turbine blade defect images in the training set and the validation set obtained by dividing the multi-objective data set and the multi-classification data set into the wind turbine blade defect detection model for training, calculating the loss function and performing back propagation, updating the network parameters, and obtaining the optimal parameter model of the multi-objective data set and the optimal parameter model of the multi-classification data set; Step 4, respectively input the wind turbine blade defect images in the test set obtained by dividing the multi-objective dataset and the multi-classification dataset into the multi-objective dataset optimal parameter model and the multi-classification dataset optimal parameter model, and output an accurate recognition map of the wind turbine blade defect images.

2. The method according to claim 1, characterized in that Step 1 includes: obtaining a multi-target dataset and a multi-classification dataset of wind turbine blade defects, as well as corresponding defect location labels, where the data labels are in a text file txt format; The images contained in the multi-target dataset and the multi-classification dataset are preprocessed to a uniform size of 640×640. In the preprocessing operation, the image size is unified through the reshape operation, different images are spliced ​​into a large image, and the label boxes are mapped to each other to generate new training samples.

3. The method according to claim 2, characterized in that Step 2 includes the following steps: Step 2.1, input the training set and validation set obtained by dividing the multi-target data set and the multi-classification data set into the wind turbine blade defect detection model for training; Step 2.2, at the end of the backbone network, by using the pooling pyramid, feature maps of different scales are obtained, which helps to detect objects at different scales and extract more expressive features; Step 2.3, after step 2.2, there are three output feature maps in the neck network Neck, which correspond to the inputs of three detection heads of three different sizes, and the final detection heads Head of the three detection heads output feature maps respectively; Step 2.4, use the minimum parameter distance intersection over union (MPDIoU) to calculate the loss; Step 2.5, the original input image is input into the backbone network after the preprocessing operation in step 1, so as to realize the large, medium and small scale prediction of fan defects.

4. The method according to claim 3, characterized in that Step 2.1 includes: in a model training process, the images in the training set and the validation set obtained after dividing the multi-target data set are preprocessed as input images and input into the backbone network, the backbone network includes 4 convolution modules CBS, 4 efficient multi-level residual modules ECAMC based on the improved efficient channel attention mechanism ECA and 3 downsampling convolution MPConv modules, the convolution module CBS includes a convolution layer Conv, normalization and SiLU activation function, the convolution module CBS is used to perform preliminary extraction of image features and size transformation of the image, wherein the convolution kernel size of the first convolution module CBS and the third convolution module CBS are both 3, the step size is 1, the convolution kernel size of the second convolution module CBS and the fourth convolution module CBS is 3, the step size is 2 After the input image passes through four convolution modules CBS, a feature map X1 of size 160×160×128 is output after convolution operation, and then input into an efficient multi-level residual module. The efficient multi-level residual module includes maximum pooling efficient channel attention ECAM, conventional convolution, feature map concatenation Concat, normalization and SiLU activation function. The four efficient multi-level residual modules have the same structure; A downsampling convolution MPConv module is used to adjust the size of the feature map. The downsampling convolution MPConv module includes a maximum pooling layer, two convolution modules CBS with a convolution kernel size of 1×1 and a convolution step size of 1, and one convolution module CBS with a convolution kernel size of 3×3 and a convolution step size of 2; The image to be detected obtains multi-scale features through each efficient multi-level residual module in the backbone network, and then the multi-scale features are input into the neck network Neck; The efficient multi-level residual module is composed of a stepped residual structure consisting of maximum pooling efficient channel attention ECAM, conventional convolution, Concat, normalization and SiLU activation function. Each efficient multi-level residual module includes an 8-layer convolution structure and a two-layer splicing structure. The residual structure used inside the efficient multi-level residual module includes a layer of conventional convolution with a convolution kernel size of 1×1 and a layer of maximum pooling efficient channel attention ECAM. The convolution kernel size of all convolutions except the first layer in the efficient multi-level residual module is 3×3. The efficient multi-level residual module obtains image features of different levels through level-by-level convolution, and finally combines the image features of different levels within the module by using the residual structure in the splicing layer to obtain a new feature map; the results of the first and third convolution layers in the efficient multi-level residual module are spliced ​​and input into the fourth convolution layer. In addition, the maximum pooling efficient channel attention ECAM is used in the residual connection part of the efficient multi-level residual module. The maximum pooling efficient channel attention ECAM uses global average pooling and global maximum pooling in parallel to extract features, and uses a one-dimensional convolution kernel of size k to generate channel weights, and uses a 1×1 convolution operation with a Sigmoid activation function to obtain channel attention information, and performs point multiplication of the channel attention information with the initial feature information contained in the feature map X1 to obtain a specific channel attention feature map; finally, the feature maps of different levels are feature fused in the Concat layer to obtain the output feature map of the efficient multi-level residual module.

5. The method according to claim 4, characterized in that Step 2.2 includes: at the end of the backbone network, the output feature map of the fourth efficient multi-level residual module is used as the input of the cross-level spatial pyramid pooling module SPPCSPC of the neck network Neck: the cross-level spatial pyramid pooling module SPPCSPC is composed of 5 convolution modules CBS with a convolution kernel size of 1, two convolution modules CBS with a convolution kernel size of 3, a maximum pooling layer with a convolution kernel size of 5, a maximum pooling layer with a convolution kernel size of 9, and a maximum pooling layer with a convolution kernel size of 13. By using the pooling pyramid, feature maps of different scales are obtained; The neck network Neck includes 1 cross-level spatial pyramid pooling module SPPCSPC, 1 secondary residual feature fusion module SRGS, 2 downsampling convolution MPConv modules, 2 upsamplings, 3 new generation convolution deformation modules CNT, 4 Concat operations and 2 convolution modules CBS. The output feature maps of the second and third efficient multi-level residual modules in the backbone network pass through a convolution module CBS and an upsampling module in the neck network Neck respectively. The convolution module CBS adjusts the number of channels through the convolution operation, and the upsampling module uses the nearest neighbor difference method to adjust the feature map size, and then inputs them into the neck network Neck layer by layer for multi-scale feature fusion to reduce the loss in the feature transmission process. The new generation convolution deformation module CNT includes a ConvNext module and three convolution modules CBS, wherein the ConvNext module includes depthwise separable convolution, layer normalization, CBS convolution, GELU activation function and LayerScale layer; The secondary residual feature fusion module SRGS is used to further fuse the feature information extracted by the backbone network and suppress irrelevant information. The secondary residual feature fusion module SRGS adopts a secondary residual connection structure. First, before the feature map enters GSConv, a convolution with a convolution kernel size of 1×1 is used to adjust the number of channels; The secondary residual feature fusion module SRGS includes 4 GSConvs with a convolution kernel size of 3×3, 2 regular convolutions, and 2 partial convolutions, which are spliced ​​twice along the channel dimension. Both splicing operations use residual connections, where the residual connection of the first splicing operation is the first residual connection part, and the residual connection of the second splicing operation is the second residual connection part; the first splicing of the secondary residual feature fusion module SRGS is composed of a feature map after a 1×1 convolution and two GSConvs, which is a 1×1 convolution and a The feature map after the 3×3 partial convolution PConv and the input data after the 1×1 convolution are composed; the second splicing part of the secondary residual feature fusion module SRGS, that is, the output part, is composed of the feature map after the 3×3 partial convolution PConv of the fourth GSConv and the third GSConv in the secondary residual feature fusion module; partial convolution PConv is introduced in the residual connection part of the secondary residual feature fusion module SRGS, and standard convolution is applied only to 1 / 2 of the data in the input channel, and no operation is performed on the remaining 1 / 2 channels; The new generation convolution deformation module CNT includes a first calculation stage and a second calculation stage. In the first calculation stage, the convolution after the ConvNext module and the next layer of 1×1 is used as the output. In the second calculation stage, i.e. the prediction part, the convolution module CBS is used twice after the ConvNext module to perform feature fusion and adjust the number of channels at the same time. The ConvNext module includes four layers of operations: the first layer performs a 7×7 depth-separable convolution on the input feature map, uses a step size of 1, and pads 3 to ensure the size of the feature map, and then uses a layer normalization operation; the second layer uses a 1×1 convolution kernel and a GELU activation function to integrate information, and adjusts the number of feature map channels to four times the original number; the third layer first uses a 1×1 convolution operation, and then uses a scaling layer LayerScale; the fourth layer uses a random drop layer Drop Path for regularization, and finally adjusts the output layer feature map size back to h×w×c; h represents the height of the image, and w represents the width of the image.

6. The method according to claim 5, characterized in that Step 2.3 includes: after step 2.2, there are three output feature maps in the neck network Neck corresponding to the inputs of three detection heads of three different sizes respectively, the detection heads include reparameterized convolution RepConv and final head detection Head, when the images in the training set and validation set obtained after dividing the multi-target data set are processed by the backbone network and the neck network Neck, the output feature maps will be input into the detection head for prediction, the reparameterized convolution RepConv includes convolution and normalization; the final detection heads Head of the three detection heads output feature maps of sizes 20×20×255, 40×40×255, and 80×80×255 respectively.

7. The method according to claim 6, characterized in that Step 2.4 includes: using the minimum parameter distance intersection-over-union ratio MPDIoU to calculate the loss, the formula is: (1)， (2)， (3)， (4)， (5)， (6)， (7)， (8)， (9)， (10)， in , Respectively represent the horizontal and vertical coordinates of the upper left corner of the target to be detected in the image, , Respectively represent the horizontal and vertical coordinates of the lower right corner of the target to be detected in the image; , Respectively represent the coordinates of the upper left corner of the predicted target in the image; , They represent the coordinates of the predicted target in the lower right corner of the image respectively; d1 represents the distance between the target to be detected and the upper left corner coordinates of the predicted target, and d2 represents the distance between the target to be detected and the lower right corner coordinates of the predicted target; represents the area of ​​the real box, represents the area of ​​the real box, Represents the larger horizontal coordinate of the upper left corner of the detection frame and the upper left corner of the real frame. Represents the larger ordinate between the upper left corner of the detection frame and the upper left corner of the real frame. Represents the smaller horizontal coordinate of the lower right corner of the detection frame and the lower right corner of the real frame. Represents the smaller ordinate between the lower right corner of the detection frame and the lower right corner of the real frame. represents the overlapping area, Represents the total area occupied by the detection frame and the real frame including the overlapping part. Iou represents the result of dividing the overlapping part of the two regions by the combined part of the two regions. Represents the loss calculated using the minimum parameter distance intersection MPDIou.

8. The method according to claim 7, characterized in that Step 2.5 includes: inputting the original input image into the backbone network after the preprocessing operation of step 1, and after passing through four convolution modules CBS, the first efficient multi-level residual module is called the first stage, the second efficient multi-level residual module is called the second stage, the third efficient multi-level residual module is called the third stage, and the fourth efficient multi-level residual module is called the fourth stage, wherein the output of the fourth stage is used as the input of the cross-level spatial pyramid pooling module SPPCSPC of the neck network Neck, and after passing through the cross-level spatial pyramid pooling module SPPCSPC, the image size is transformed through the convolution module CBS and upsampling operation, and feature extraction is performed, and the output of the third stage is subjected to a convolution with a convolution kernel size of 1×1 and then outputs the feature, which is fused, which is called the first fusion, and then passes through the secondary residual feature fusion module SRGS of the neck network Neck, and then passes through two convolution modules CBS with a convolution kernel size of 1×1 and the feature map after upsampling operation, and the output feature map of the second stage is fused through the convolution module CBS with a convolution kernel size of 1×1, which is called For the second fusion, the feature map output by the first new generation convolution deformation module CNT is used to realize the prediction of large-scale targets of fan defects after re-parameterized convolution RepConv. The feature map output by the first new generation convolution deformation module CNT and an MP module is fused with the feature map output by a secondary residual feature fusion module SRGS and a convolution module CBS to obtain a new feature map and input it into the second new generation convolution deformation module CNT. The feature map output by the second new generation convolution deformation module CNT is used to realize the medium-scale prediction of fan defects after re-parameterized convolution RepConv. At the same time, the feature map output by the second new generation convolution deformation module CNT is fused with the feature map output by the second MP module and SPPCSPC module and enters the third new generation convolution deformation module CNT. The feature map output by the third new generation convolution deformation module CNT is used to realize the small- and medium-scale prediction of fan defects after re-parameterized convolution RepConv. The accuracy and loss are calculated for each round of training.

9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores program codes, and when the program codes are executed by the processor, the processor executes the steps of the method according to any one of claims 1 to 8.

10. A storage medium, characterized in that: A computer program or instruction is stored, and when the computer program or instruction is run on a computer, the steps of the method according to any one of claims 1 to 8 are executed.

Images