Aircraft engine blade defect detection method based on windmill convolution attention and multi-order information dynamic aggregation

Abstract

The invention provides an aero-engine blade defect detection method based on windmill convolution attention and multi-order information dynamic aggregation, and belongs to the field of aero-engine blade defect detection. The method comprises the following steps: acquiring a real surface defect image of an aero-engine blade, marking different defects in the image by utilizing expert knowledge, and dividing the marked defect image into a training set, a verification set and a test set; constructing a defect detection network MSDNet containing windmill convolution attention and a multi-order information dynamic aggregation module, and training the multi-scale detection model through the training set; mining small-scale defect features in the surface defects of the aero-engine blade through a windmill convolution attention module to obtain defect features; the deep features and the shallow features from the image are fused through a multi-order information dynamic aggregation module, and fused defect features are obtained; and utilizing the trained defect detection network MSDNet to carry out surface defect detection on the aero-engine blade, inputting a to-be-detected image of the blade, and outputting a defect detection result corresponding to the blade image.

Description

TECHNICAL FIELD

[0001] The application relates to the field of aero-engine blade defect detection, and in particular to an aero-engine blade defect detection method based on windmill convolution attention and multi-order information dynamic aggregation. BACKGROUND

[0002] In the field of aerospace, the quality of engine blades is crucial to flight safety. Surface defects of the blades can cause the performance of the aero-engine to decline, leading to mechanical failure or even accidents. With the development of deep learning and computer vision technology, industrial defect detection based on deep neural networks has made great progress and is widely used in various fields. However, the surface defects of aero-engine blades often have large scale variations. For small-scale defects, general detection methods are difficult to directly identify. By deepening the network layers, the recognition of small-scale defects can be enhanced, but the global information is lost, resulting in reduced detection ability for large-scale defects. Therefore, there is an urgent need for a method for detecting surface defects of aero-engine blades. SUMMARY

[0003] According to the fact that the surface defects of aero-engine blades are small and have large scale variations, the application provides an aero-engine blade defect detection method based on windmill convolution attention and multi-order information dynamic aggregation. The method comprises the following steps:

[0004] Step 1: Obtain the real surface defect image of the aero-engine blade through an industrial camera, and label different defects in the image using expert knowledge. Divide the multiple labeled defect images into a training set, a validation set and a test set;

[0005] Step 2: Construct a defect detection network MSDNet containing windmill convolution attention and multi-order information dynamic aggregation, and train the MSDNet through the training set in step 1;

[0006] Step 3: Extract small-scale defect features in the surface defects of the aero-engine blade through windmill convolution attention to obtain defect features;

[0007] Step 4: Perform adaptive fusion of deep features and shallow features from the image through multi-order information dynamic aggregation to obtain fused defect features;

[0008] Step 5: Use the trained MSDNet to detect the surface defects of the aero-engine blade, input the blade image to be detected, output the defect detection result image corresponding to the blade image, and classify the defect detection result.

[0009] In step 1, the original picture of the aero-engine blade is obtained by an industrial camera, the surface defects of the blade are labeled and classified by expert knowledge, and the labeled images are divided into a training set, a validation set and a test set according to a ratio of 6:2:2.

[0010] In step 2, the MSDNet defect detection network model constructed by the windmill convolution attention and the multi-order information dynamic aggregation has the following characteristics:

[0011] The MSDNet defect detection network containing the windmill convolution attention and the multi-order information dynamic aggregation is composed of an extraction network, a fusion network and a prediction network. The extraction network is composed of 4 CBS modules (3x3 size convolution + normalization + SiLU activation function), 4 jump connection modules (C3k2) containing 2 cores and a feature pyramid pooling module (SPPF). The fusion network is composed of 4 C3k2 modules, 4 multi-order information dynamic aggregation modules (MIDA), 4 windmill convolution attention modules (PCAM), 2 CBS modules and 2 up-sampling modules (Upsample), wherein the three PCAM modules connected with the prediction network are provided with residual connection. The prediction network predicts the position information and the category information of the defects through a decoupling head.

[0012] In step 3, the feature map input into the PCAM is divided into two parts, one part is used for jump connection, and the other part is used for calculating the windmill convolution attention. The part for calculating the windmill convolution attention first passes through normalization and convolution with a kernel size of 1x1 in turn, and then is divided into three parts to calculate Q, K and V through windmill convolution (PConv) with a convolution kernel size of 6x6, wherein Q and K are multiplied to obtain the attention weight through Softmax normalization, and finally V is multiplied and passed through convolution with a kernel size of 1x1.

[0013] Specifically, the calculation formula of this process is:

[0014] x out =x in +Conv 1×1 (Softmax(Q×K)×V);

[0015] Q=K=V=PConv 6×6 (Norm(Conv 1×1 (x in )))

[0016] Wherein, x in represents the feature map input into the PCAM; Conv 1×1 (·) represents convolution with a kernel size of 1x1; Softmax(·) represents the Softmax normalization operation; x outdenotes the feature map of the output PCAM; Norm(·) denotes the Norm normalization operation; PConv 6×6 denotes a windmill convolution with a kernel size of 6x6.

[0017] In step 4 above, the feature maps F1 and F2 of the input MIDA are added to obtain the fusion feature map F Fuse . Fuse After four branches, three sets of pooling deformation and one set of identity mapping are obtained. In the three sets of pooling deformation branches, the fusion feature map F Fuse is respectively subjected to AvgPool4(·), AvgPool2(·) and AvgPool1(·) pooling operations to obtain f c1 ∈R C×4×4 , f c2 ∈R C×2×2 and f c3 ∈R C×1×1 . In order to be compatible with the identity mapping f s ∈R C×H×W , we reshape it back to R C×H×W , and add the four branches to obtain the weight parameter a e R C×H×W . Multiply the weight a(x) by the feature map F1, and multiply the weight 1-a(x) by the feature map F2 to obtain the weighted feature maps F1 and F2, respectively, and finally add them to output F out .

[0018] Specifically, the calculation formula of this process is:

[0019] F Fuse =F1+F2

[0020] F out =a(x)xF1+(1-a(x))xF2

[0021]

[0022] Where F1 and F2 represent the input feature maps; F Fuse represents the fused feature map; a represents the weight parameter; AvgPool i (·) represents the mean pooling operation, i represents the kernel size in mean pooling; UnPool(·) represents the operation of reshaping the feature map back to the original size; F out represents the output feature map.

[0023] In step 5 above, the trained model parameters are saved, the aero-engine blade defect images to be detected are placed in the same folder, the folder path is input into the network model, the model identifies and classifies the defects in the images, and finally outputs.

[0024] The present invention has the following beneficial effects: This method utilizes windmill convolution attention and multi-level information dynamic aggregation target detection methods to address the problem of low detection accuracy for aircraft engine blade surface defects due to large scale variations. For small-scale defects, traditional deep learning detection methods can only improve accuracy by increasing the number of network layers, but this also loses global information in the image, resulting in an inability to detect large-scale defects. This method effectively solves these problems and significantly improves the detection accuracy of defects with varying scales in aircraft engine blade surface defects. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 Schematic diagram of the process of this application.

[0026] Figure 2 This is a schematic diagram of the structure of the aircraft engine blade defect detection network MSDNet.

[0027] Figure 3 Schematic diagram of the pinwheel convolution attention structure.

[0028] Figure 4 Schematic diagram of the dynamic aggregation process of multi-level information.

[0029] Figure 5 This is the result of surface defect detection on aircraft engine blades. DETAILED DESCRIPTION

[0030] In order to present the technical solution of the method of this application more clearly, the following will describe in detail the implementation methods, technical features and advantages of this application in combination with illustrated cases, through corresponding descriptions of specific embodiments and drawings.

[0031] like Figure 1 As shown in FIG, a flow chart of the method of the present application is provided. For the defect detection of aircraft engine blades, a defect detection method based on windmill convolution attention and multi-order information dynamic aggregation is performed, which includes the following steps:

[0032] Step 1: Use an industrial camera to capture images of real aircraft engine blade surface defects. Under the guidance of professionals, the defects on the blades are labeled and classified to construct an aircraft engine blade surface defect detection dataset. The constructed dataset contains 1,120 images of four types of defects at a resolution of 3024×3024. The labeled images are divided into training, validation, and test sets in a 6:2:2 ratio.

[0033] Step 2: Construct an aircraft engine defect detection network MSDNet that includes windmill convolution attention and multi-level information dynamic aggregation. Figure 2 As shown, the network model is trained using the training set.

[0034] MSDNet consists of three parts: extraction network, fusion network and prediction network. Among them, the extraction network consists of 4 CBS modules (3x3 size convolution + normalization + SiLU activation function), 4 jump connection modules containing 2 cores (C3k2) and a feature pyramid pooling module (SPPF); the neck network consists of 4 C3k2 modules, 4 multi-order information dynamic aggregation modules (MIDA), 4 pinwheel convolution attention modules (PCAM), 2 CBS modules and 2 upsample modules (Upsample), wherein the 3 PCAM modules connected with the prediction network are provided with residual connection; the prediction network predicts the position information and category information of defects through the decoupling head.

[0035] In the training of MSDNet, transfer learning and network fine-tuning are used for training. At the beginning, the proposed network model is initialized by pre-trained weights. All models are pre-trained for 100 epochs, divided into two stages. The first stage (0-40 epochs): freeze the backbone network, learn the network parameters of the remaining part. Since the number of training parameters in the first stage is small, the batch size is set to 8. The second stage (40-100 epochs): open the backbone network, train all weight parameters of the network. Since the number of weight parameters to be trained at this time is large, the batch size is set to 4. At the same time, in order to improve the stability of the training process, the preheating strategy is used in the first three training epochs, and the cosine learning rate is used in the last three training epochs. The learning rate of the whole process is set to between 0.0001 and 0.001.

[0036] Step 3: Extract small-scale defect features in the surface defects of the aero-engine blade through pinwheel convolution attention, and obtain defect features, as shown in Figure 3 .

[0037] The pinwheel convolution attention module generates query (Q), key (K) and value (V) vectors through pinwheel convolution PConv, calculates the similarity between Q and K to generate attention weight (this process is realized through Softmax), and applies it to the V vector to obtain the final attention weighted feature. The feature map input into the pinwheel convolution attention module (PCAM) is divided into two parts, one part is used for jump connection, and the other part is used to calculate convolution attention. The part used to calculate attention first passes through normalization and convolution with a kernel size of 1x1 in turn, and then is divided into 3 parts to calculate Q, K and V through PConv with a convolution kernel size of 6x6. Q and K are multiplied to obtain the attention weight through Softmax normalization, and then multiplied with V and passed through a convolution with a kernel size of 1x1.

[0038] Specifically, the calculation formula of this process is:

[0039] x out =x in +Conv 1×1 (Softmax(Q×K)×V);

[0040] Q=K=V=PConv 6×6 (Norm(Conv 1×1 (x in )))

[0041] where x in denotes the feature map of the input PCAM; Conv 1×1 (·) denotes the convolution with the kernel size of 1x1; Softmax(·) denotes the Softmax normalization operation; x out denotes the feature map of the output PCAM; Norm(·) denotes the Norm normalization operation; PConv 6×6 (·) denotes the pinwheel convolution with the kernel size of 6x6.

[0042] The pinwheel convolution structure is shown in Figure 3 . Unlike ordinary convolution, the pinwheel convolution uses asymmetric padding to create horizontal and vertical convolution kernels for different regions of the image. Utilize H, W and C to represent the height, width and channel number of the input tensor X respectively. After each convolution, apply normalization and activation function to improve training stability and speed. The calculation formula of this process is:

[0043]

[0044]

[0045] where X1, X2, X3 and X4 denote the convolution of the four colors in Figure Three respectively; W (3,1,c) denotes the convolution kernel with the kernel size of 3x1; the padding parameter P(1, 0, 0, 3) represents the number of padding pixels on the left, right, top and bottom respectively.

[0046] After obtaining the four kinds of padded convolution, use two convolution kernels with the kernel size of 1x3 and two convolution kernels with the kernel size of 3x1 to perform convolution operation with a step of 1 to obtain the final result.

[0047] Step 4: fuse the deep and shallow features from the image through the multi-order information dynamic aggregation module to obtain the fused defect feature, as shown in Figure 4 .

[0048] The multi-order information dynamic aggregation module (MIDA) can selectively learn different information from deep and shallow networks, and efficiently learn useful information using an attention mechanism. The feature maps F1 and F2 input into the MIDA are subjected to linear addition to obtain a fused feature map F Fuse . Fuse There are four branches corresponding to three sets of pooling deformations and one set of identity mappings. In the three sets of pooling deformation branches, the fused feature map F Fuse is subjected to AvgPool4(·), AvgPool2(·) and AvgPool1(·) pooling operations, respectively, to obtain f c1 ∈R C×4×4 , f c2 ∈R C×2×2 and f c3 ∈R C×1×1 . In order to be compatible with the identity mapping f s ∈R C×H×W , we reshape it back to R C×H×W , and add the four branches and pass them through a Sigmoid function to obtain the weight parameter a e R C×H×W . Multiply the weight a(x) by the feature map F1, and multiply the weight 1-a(x) by the feature map F2 to obtain the weighted feature maps F1 and F2, respectively, and finally add them to output F out .

[0049] Specifically, the calculation formula of this process is:

[0050] F Fuse =F1+F2

[0051] F out =a(x)xF1+(1-a(x))xF2

[0052]

[0053] where F1 and F2 represent the input feature maps; F Fuse represents the fused feature map; a represents the weight parameter, and a e [0, 1]; AvgPool i (·) represents the mean pooling operation, and i represents the kernel size in the mean pooling; UnPool(·) represents the operation of reshaping the feature map back to the original size; F out represents the output feature map.

[0054] Step 5: Use the trained defect detection model to detect defects on the surface of an aero-engine blade, input the image of the blade to be detected, and output the defect detection result image corresponding to the blade image and classify it, as shown in Figure 5 .

[0055] The test set data in the aero-engine blade surface defect detection data set constructed in step 1 is used for testing, the trained model parameters are saved, the aero-engine blade defect image to be detected is placed in the same folder, the folder path is input into the network model, the model identifies and classifies the defects in the image, and finally outputs.

[0056] The specific embodiments of the patent application are described in detail above, but they are only examples, and the patent application is not equivalent to the specific embodiments described above, and the patent application can realize aero-engine blade surface defect detection. Any equivalent modification and substitution of the patent application by those skilled in the art is also within the scope of the patent application. Therefore, any equivalent transformation and modification made without departing from the spirit and scope of the patent application should be covered within the scope of the patent application.

Claims

1. A method for detecting defects in aircraft engine blades based on windmill convolution attention and multi-order information dynamic aggregation, characterized by The steps include: Step 1: Obtain real surface defect images of aircraft engine blades, annotate the different defects in the images, and divide the multiple annotated defect images into training, validation, and test sets; Step 2: Construct a defect detection network MSDNet including a pinwheel convolution attention module and a multi-order information dynamic aggregation module, and train MSDNet using the training set; Step 3: Use windmill convolution attention to mine small-scale defect features in the surface defects of aircraft engine blades and obtain defect features; Step 4: The deep features and shallow features from the image are fused through the multi-level information dynamic aggregation module to obtain the fused defect features; Step 5: Use the trained defect detection network MSDNet to detect surface defects in aircraft engine blades. Input the image of the blade to be detected, output the defect detection result image corresponding to the blade image, and classify the defect detection results.

2. The method for detecting defects in aircraft engine blades based on windmill convolution attention and multi-order information dynamic aggregation according to claim 1 is characterized in that: In step 1, an industrial camera is used to obtain a real aircraft engine blade defect image, and the surface defects of the blade are marked and classified based on expert knowledge.

3. The method for detecting defects in aircraft engine blades based on windmill convolution attention and multi-order information dynamic aggregation according to claim 1 is characterized in that: In step 2, the defect detection network MSDNet constructed including the windmill convolution attention module and the multi-order information dynamic aggregation module has the following characteristics: The MSDNet defect detection network, which incorporates pinwheel convolutional attention and multi-level dynamic information aggregation, consists of three parts: an extraction network, a fusion network, and a prediction network. The extraction network consists of four CBS modules (3×3 convolution + normalization + SiLU activation function), four skip connection modules (C3k2) with two cores, and a feature pyramid pooling module (SPPF). The fusion network consists of four C3k2 modules, four multi-level dynamic information aggregation modules (MIDA), four pinwheel convolutional attention modules (PCAM), two CBS modules, and two upsampling modules (Upsample). The three PCAM modules connected to the prediction network use residual connections. The prediction network uses a decoupling head to predict the location and category of defects.

4. The method for detecting defects in aircraft engine blades based on windmill convolution attention and multi-order information dynamic aggregation according to claim 1, characterized in that: In step 3, the feature map input to the Pinwheel Convolution Attention Module (PCAM) is divided into two parts: one for skip connections and the other for calculating the Pinwheel Convolution Attention. The part used for calculating the Pinwheel Convolution Attention is first normalized and convolved with a kernel size of 1×1. It is then divided into three parts and each undergoes a Pinwheel Convolution (PConv) with a kernel size of 6×6 to calculate Q, K, and V. Q is multiplied by K and then normalized by Softmax to obtain the attention weight. Finally, it is multiplied by V and convolved with a kernel size of 1×1. Specifically, the calculation formula for this process is: x out =x in +Conv 1×1 (Softmax(Q×K)×V); Q=K=V=PWConv 6×6 (Norm(Conv 1×1 (x in ))) Among them, x in Represents the feature map of the input PCAM; Conv 1×1 (·) represents a convolution with a kernel size of 1×1; Softmax(·) represents a Softmax normalization operation; x out Represents the feature map of the output PCAM; Norm(·) represents the Norm normalization operation; PConv 6×6 (·) represents a pinwheel convolution with a kernel size of 6×6. Pinwheel convolution differs from ordinary convolution in that it uses asymmetric padding to create horizontal and vertical convolution kernels for different regions of the image. H, W, and C represent the height, width, and number of channels of the input tensor X, respectively. Normalization and activation functions are applied after each convolution to improve training stability and speed. The calculation formula for this process is: X1=SiLU(BN(X P(1,0,0,3) ×W1 (1,3,c) )) X2=SiLU(BN(X P(0,3,0,1) ×W2 (3,1,c) )) X3=SiLU(BN(X P(0,1,3,0) ×W3 (1,3,c) )) X4=SiLU(BN(X P(3,0,1,0) ×W4 (3,1,c) )) Among them, X1, X2, X3 and X4 represent the convolution of the four colors in Figure 3 respectively; W (3,1,c) represents a convolution kernel with a kernel size of 3×1; the padding parameter P(1,0,0,3) represents the number of padding pixels on the left, right, top, and bottom, respectively. After obtaining the four padded convolutions, we use two convolution kernels with a kernel size of 1×3 and two convolution kernels with a kernel size of 3×1 to perform a convolution operation with a step size of 1 to obtain the final result.

5. The method for detecting defects in aircraft engine blades based on windmill convolution attention and multi-order information dynamic aggregation according to claim 1, characterized in that: In step 4, the feature maps F1 and F2 input to the multi-scale information adaptive fusion module (MIAF) are added to obtain a fused feature map F Fuse . F Fuse After four branches, there are three groups of pooling deformations and one group of identity mapping. In the three groups of pooling deformation branches, the fusion feature map F Fuse Perform AvgPool4(·), AvgPool2(·) and AvgPool1(·) pooling operations respectively to obtain f c1 ∈R C×4×4 、f c2 ∈R C×2×2 and f c3 ∈R C×1×1 In order to match the identity map f s ∈R C×H×W compatible, we reshape it back to R C×H×W And add the four branches to get the weight parameter α∈R C×H×W Multiply the weight α(x) by the feature map F1, and multiply the weight 1-α(x) by the feature map F2 to obtain the weighted feature maps F1 and F2 respectively, and finally add them together to output F out . Specifically, the calculation formula for this process is: F Fuse =F1+F2 F out =α(x)×F1+(1-α(x))×F2 Among them, F1 and F2 represent the input feature maps; F Fuse Represents the fused feature map; α represents the weight parameter; AvgPool i (·) represents the mean pooling operation, i represents the kernel size in the mean pooling; UnPool(·) represents the operation of reshaping the feature map back to its original size; F out Represents the output feature map.

Images