Aero-engine blade defect detection method based on multi-scale detr

Abstract

The application discloses an aero-engine blade defect detection method based on a multi-scale DETR, first, X-ray image data sets of defective blades are collected, and data enhancement is performed on the unbalanced data sets; then, feature maps extracted in a feature extraction network are input into a multi-scale fusion attention mechanism module to obtain weighted feature maps; subsequently, the weighted feature maps are input into a multi-scale feature fusion module, and final feature maps are output after cross-scale feature fusion; the feature maps added with position encoding and target query sequences are interacted with a Transformer, and R-Focal loss is used to calculate loss and optimize the model through back propagation; finally, defect pictures obtained in an actual industrial scene are input into the network for detection, and final detection results are obtained; through model training and industrial application test, the method can improve blade defect detection precision, improve detection speed, and has high practical value.

Description

Technical Field

[0001] This invention relates to the field of defect detection using computer vision knowledge, and particularly to a method for detecting defects in aero-engine blades based on multi-scale DETR. Background Technology

[0002] The proper functioning of aero engines is crucial for the safe takeoff and operation of aircraft. As a core component of the engine, turbine blades need to operate under extreme high temperature, high pressure, and complex load conditions. This necessitates rigorous quality inspection before leaving the factory to ensure the absence of casting defects or other damage, thus guaranteeing the safety of the engine and aircraft. Defect detection in blades is a key step in engine production quality control. During the blade casting process, defects such as porosity and sand inclusions are prone to occur, which can seriously threaten the structural stability of the engine and even endanger the lives of the crew. Currently, the main detection methods include X-ray inspection, liquid penetrant testing, eddy current testing, and magnetic particle testing. However, these traditional techniques are relatively inefficient and heavily rely on the experience of the inspectors. Prolonged work can lead to visual fatigue, resulting in false positives or missed negatives.

[0003] Therefore, developing an intelligent and efficient blade defect detection method is of great significance. In recent years, with the increase in data volume and the successful application of deep learning models, represented by convolutional neural networks, in the field of computer vision, deep learning-based image recognition methods have been widely used in scenarios such as face recognition, fault detection, and flaw detection. Mainstream image recognition networks include DETR, SSD, and YOLOv2 / v3 / v4. Applying image recognition technology to blade defect detection can effectively solve problems such as reliance on manual inspection, low detection accuracy, and slow response speed. Currently, research on blade defect detection using image recognition technology is relatively limited, which seriously restricts the automated development of detection technology. Therefore, research on blade defect and damage detection based on image recognition networks is of great significance. It can break through the limitations of traditional detection technologies, achieve automation and intelligence, effectively improve detection efficiency and accuracy, and provide technical assurance for the safety of aero-engines. Summary of the Invention

[0004] Purpose of the invention: The purpose of this invention is to provide a method for detecting defects in aero-engine blades based on multi-scale DETR, which can improve the problems of inaccurate location identification, poor classification effect, poor detection effect of small targets and slow detection speed in the process of detecting defects in aero-engine blades, and realize accurate and efficient real-time detection of defects in aero-engine blades.

[0005] Technical solution: The multi-scale DETR-based defect detection method for aero-engine blades described in this invention specifically includes the following steps:

[0006] S1: Acquire X-ray image datasets of defective blades and perform data augmentation on the imbalanced dataset;

[0007] S2: Input the data augmented in S1 into the feature extraction network ResNet50, and extract the feature maps of four layers of ResNet50: block2, block3, block4 and block5 respectively. Block2 and block3 extract low-dimensional features to represent the geometric structure information such as the contour and texture of the target; block4 and block5 extract high-dimensional features to express the semantic information of the image.

[0008] S3: Input the feature map extracted in S2 into the multi-scale fusion attention mechanism module based on the improved convolutional channel attention mechanism, and use the multi-scale self-attention mechanism to perform weighted fusion of the feature map to obtain a weighted feature map.

[0009] S4: Input the weighted feature map from S3 into the multi-scale feature fusion module, and output the final feature map after cross-scale feature fusion.

[0010] S5: Add positional encoding to the final feature map output from S4 and input it into the Transformer encoder along with the target query sequence;

[0011] S6: The feature map with added position encoding and the target query sequence obtained in S5 are interacted with the Transformer, and the loss is calculated using R-Focal loss and backpropagation is performed to optimize the model, thus completing the entire target detection process. The target detection process is implemented through the complete DETR, which includes the feature extraction network in S1-S4 and the Transformer in S5-S6. The Transformer architecture is used instead of CNN to enable the target detection network to better acquire global information.

[0012] S7: Input defect images obtained from actual industrial scenarios into the network for detection to obtain the final detection results.

[0013] Preferably, the X-ray image dataset in S1 is obtained by capturing images of the blades with a dedicated X-ray camera and processing them by a computer. The data augmentation methods are Mixup, Mosaic, and Mix-Mos methods, which alleviate data imbalance and prevent data overfitting. Specifically:

[0014] The Mixup method generates images with new structures and probability distributions by linearly interpolating or stitching together different images, increasing the diversity of the dataset. Assuming two training images {x1, x2} and their corresponding labels {y1, y2}, the new image x generated by Mixup... mand tag y m The calculation is as follows:

[0015] x m =λx1+(1-λ)x2 (1)

[0016] y m =λy1+(1-λ)y2 (2)

[0017] λ is a random value between [0, 1] that controls the mixing ratio of the two original images. When λ is close to 0.5, the new sample contains richer information from the original sample, maximizing the diversity of the data and improving the generalization ability of the model.

[0018] The Mosaic method constructs entirely new training images by stitching together images while maintaining the complete label information of the original images, thereby maximizing the diversity of data, providing the model with richer semantic information, and improving detection and recognition performance.

[0019] The Mix-Mos method first uses the Mixup method for data augmentation, and then applies the Mosaic method. The Mixup method obtains the continuity of image content, and the Mosaic method obtains the richness of image information.

[0020] Preferably, the multi-scale fusion attention mechanism in S3 is an improvement on the convolutional channel attention mechanism, specifically including the following steps:

[0021] S3.1: Expand the feature map dimension to the same dimension (C3, H3, W3) as the block3 feature map by 1×1 convolution and upsampling, and then concatenate the four feature maps along the channel direction to obtain the feature map X0, with dimensions denoted as (C, H, W).

[0022] S3.2: Pass the feature map X0 through two 3×3 convolutional layers, denoted as C1 and C2 respectively, with padding set to 1, to obtain feature maps X1(C,H,W) and X2(C,H,W);

[0023] S3.3: Obtain the fused feature layers Y1 and Y2 according to formulas (1) and (2):

[0024] Y1=C1(X0)+X0 (3)

[0025] Y2=C2(X1)+Y1 (4)

[0026] S3.4: Dimensionality reduction of Y1 and Y2 is performed by mean pooling, and the resulting weights W1 and W2 have dimensions of (C,1,1);

[0027] S3.5: Input W1 and W2 into the FC layer, which consists of two fully connected layers and a last sigmoid activation function layer, to obtain the final fusion weight W. F (C,1,1);

[0028] S3.6: Merge weight W F Multiplying the original feature map X0 by the original feature map X0 yields a weighted feature map X(C,H,W) after a multi-scale fusion attention mechanism.

[0029] Preferably, the multi-scale feature fusion module in S4 is based on multiple pooling methods and an improved spatial attention mechanism, and specifically includes the following calculation steps:

[0030] S4.1: Perform global mean pooling and global max pooling on the weighted feature map X(C,H,W) obtained in S3 to obtain two feature maps with the same dimension P1(2,H,W) after dimensionality reduction, and then concatenate the other channel dimensions to obtain feature map P2(2,H,W).

[0031] S4.2: Perform deconvolution on the obtained feature map to obtain a feature map with dimension P3(2,2H,2W), with a kernel size of 7×7, a stride of 2, and padding = 3;

[0032] S4.3: Perform two convolutions on the feature maps to obtain feature maps P4 and P5 with dimensions (1,H,W). The size of the convolution kernel is 7×7, the padding is 3, the stride of the two convolutions is 1 and 2 respectively, and the number of output channels is 2 and 1 respectively.

[0033] S4.4: After linearly activating the fused feature map P5 with the Sigmoid function, multiply it with X(C,H,W) to obtain the final fused feature X′(C,H,W).

[0034] Preferably, the location encoding in step S5 is added using sine and cosine functions to allow the data to carry accurate location information, as shown in the following formula:

[0035]

[0036] Where pos represents the index of the feature point in the feature map, and d model It represents the model dimension, where i indicates the dimension index for the position encoding.

[0037] Preferably, in S5, the target query sequence is a set of vectors input into the Transformer decoder. The target query vector provides a unique query vector for each predicted box, enabling the decoder to predict the category and regression result of the corresponding box based on each query vector, thereby achieving the correspondence and matching between the box and the prediction result. The dimension of the query vector is equal to the feature dimension output by the encoder, enabling the query vector to interact and match with the global features of the encoder, generating a prediction output with the corresponding category and coordinates for each predicted box.

[0038] Preferably, the Transformer part of the DETR consists of an encoder, a decoder, and a prediction head;

[0039] The encoder consists of multiple stacked encoder layers, each connected sequentially to two sub-layers: a multi-head attention mechanism (MHA) and a position feedforward network (PWFFN). The sub-layers use residual jumpers and layer normalization to ensure sufficient training of sub-layer parameters, accelerating convergence. The MHA addresses the limitation of self-attention mechanisms, which tend to overemphasize their own position when encoding current location information. By using h different attention heads to perform different linear projections on the data, different queries Q, keys K, and values ​​V are learned. The h groups of different Q, K, and V values ​​are merged in parallel, and finally, the pooled outputs of the h groups are concatenated and transformed through another learnable linear projection to produce the final output. The calculation formula is shown below:

[0040] MH(Q,K,V)=Concat(head1,head2...head h W O

[0041]

[0042] The learnable parameters include and

[0043] The decoder consists of N stacked decoder layers. Each encoder has three sequentially connected sub-layers: a masked multi-head attention layer (masked MHA), MHA, and PWFFN. The masked multi-head attention layer is the same as the MHA in the encoder block except that a mask is added. The mask indicates that certain values ​​are masked so that they do not take effect when parameters are updated.

[0044] The prediction head consists of two fully connected layers to generate detection output. The first fully connected layer is used for box category prediction, outputting the category of the box. The second layer is used for box coordinate regression prediction, outputting the center x-coordinate, y-coordinate, height, and width of the box. Each layer of the decoder's attention mechanism module outputs a feature vector, which represents the relevant image region corresponding to the query vector. The vector is concatenated with the corresponding query vector and input into the two fully connected layers to obtain the corresponding category and coordinates. R-Focal loss and SmoothL1 loss are used for loss calculation.

[0045] The R-Focal loss is an improvement upon Focal loss. The part of the R-Focal loss that controls the weights of positive and negative samples is as follows:

[0046]

[0047] The value of α ranges from 0 to 1. By controlling the value of α, the contribution of positive and negative samples to the cross-entropy loss can be controlled.

[0048] The part of the R-Focal loss that controls the weights of samples classified as easy or difficult is:

[0049]

[0050] The value of γ is greater than 0;

[0051] The complete formula for calculating R-Focal loss is as follows:

[0052]

[0053] The R-Focal loss uses a random search method to select the hyperparameter settings of the Focal loss, resulting in better generalization.

[0054] Preferably, the R-Focal loss employs a random search method to select the hyperparameters of the Focal loss, addressing the difficulty in choosing α and γ. Specifically, it includes the following steps:

[0055] S6.1: Let the ranges for random searches of points α and γ be α = [0.1, 0.2, 0.3, 0.4, 0.5] and γ = [1, 2, 3, 4, 5], respectively;

[0056] S6.2: During each batch of training, randomly select a set of α and γ for loss calculation, and record the proportion of loss reduction caused by each set of hyperparameters. The larger the proportion, the greater the contribution of this set of hyperparameters to the loss.

[0057] S6.3: Every 20-30 epochs, the worst-performing hyperparameters are removed, and the best hyperparameter combinations are retained;

[0058] S6.4: Repeat S6.3 until the optimal combination of hyperparameters is selected for training.

[0059] Preferably, after the data is processed through two fully connected layers to obtain the type and coordinates of the predicted bounding boxes, the DETR uses the Hungarian algorithm to find the best global match between the predicted bounding boxes and the ground truth bounding boxes. Specifically:

[0060] A: Treat the predicted bounding boxes as one set of nodes in a bipartite graph, and the ground truth bounding boxes as another set of nodes;

[0061] B: Calculate the IoU between each predicted bounding box and all ground truth bounding boxes as its matching degree;

[0062] C: Select the ground truth box index with the largest IoU for each predicted box; select the predicted box index with the largest IoU for each ground truth box, generate index matching relationships based on this, and form the maximum matching of the bipartite graph;

[0063] D: Only calculate the loss between the predicted box and the ground truth box in the maximum match, including class loss and regression loss;

[0064] E: Summing the loss between all matching boxes;

[0065] F: Repeat D and E to update the DETR model.

[0066] Preferably, the test image in S7 is no longer subjected to data augmentation; the original image is simply input into the network for testing.

[0067] Beneficial effects:

[0068] (1) This application proposes a novel data augmentation method. By combining the Mixup and Mosaic methods, the Mix-Mos method is proposed, which can effectively solve the problem of uneven sample distribution in industrial datasets, making the model optimization process smoother. Experimental verification shows that it improves the detection accuracy by 1.1%.

[0069] (2) This application proposes a multi-scale fusion attention module based on the convolutional channel attention mechanism, which can adaptively learn the statistical features of the data and perform attention weighting on feature maps of multiple scales, so that the network has a stronger generalization ability and improves the detection accuracy by about 2.3%.

[0070] (3) This application proposes a multi-scale feature fusion module, which realizes the fusion of high-dimensional semantic features and low-dimensional representation features, better captures the structural information of multi-layer feature maps, and improves the detection accuracy of multi-scale targets by about 3.2%.

[0071] (4) This application proposes an improved R-Focal loss, which mainly uses a random search method to optimize the hyperparameters of the Focal loss, further accelerating model convergence and improving training speed by about 10% and detection speed by about 40%. Attached Figure Description

[0072] Figure 1 This is the overall flowchart of this application;

[0073] Figure 2 This is a flowchart of the multi-scale fusion attention mechanism module computation in this application;

[0074] Figure 3 This is a flowchart of the multi-scale feature fusion module calculation in this application;

[0075] Figure 4 This is a comparison chart of the prediction accuracy of different models;

[0076] Figure 5 It is a visualization of the attention mechanism;

[0077] Figure 6 This is a comparison chart showing the contribution of different modules to prediction accuracy. Detailed Implementation

[0078] To better understand the above-described objectives, features, and advantages of the present invention, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough understanding of the invention and to fully convey the scope of the invention to those skilled in the art.

[0079] like Figure 1 The diagram shown is the overall flowchart of this application, which specifically includes the following steps:

[0080] S1: Acquire X-ray image datasets of defective blades and perform data augmentation on the imbalanced dataset;

[0081] Defective engine blades were collected, and X-ray cameras were used to photograph the internal and surface defects of the blades, obtaining the contours and features of different defects. Since blades with casting defects are rare in industrial environments, it is often impossible to collect a sufficient amount of blade data for training, which significantly affects the training effect of data-driven neural networks. Therefore, we used some common data augmentation methods, such as random cropping, scaling, brightness enhancement, and contrast adjustment. In addition, this application used Mixup and Mosaic data augmentation methods, as well as the Mix-Mos fusion method, on the original image dataset. These three methods can generate new samples through simple operations, effectively expanding the image dataset and increasing the model's generalization ability and robustness. Specifically:

[0082] The Mixup method generates images with new structures and probability distributions by linearly interpolating or stitching together different images, increasing the diversity of the dataset. Assuming two training images {x1, x2] and their corresponding labels {y1, y2}, the new image x generated by Mixup... m and tag y m The calculation is as follows:

[0083] x m =λx1+(1-λ)x2 (1)

[0084] y m =λy1+(1-λ)y2 (2)

[0085] λ is a random value between [0, 1] that controls the mixing ratio of the two original images. When λ is close to 0.5, the new sample contains richer information from the original sample, maximizing the diversity of the data and improving the generalization ability of the model.

[0086] The Mosaic method constructs entirely new training images by stitching together images while maintaining the complete label information of the original images. This maximizes the diversity of the data, provides the model with richer semantic information, and improves detection and recognition performance.

[0087] The Mix-Mos method first uses the Mixup method for data augmentation, and then applies the Mosaic method. The Mixup method obtains the continuity of image content, and the Mosaic method obtains the richness of image information.

[0088] S2: The data augmented in S1 is input into the ResNet50 feature extraction network. The ResNet50 network effectively alleviates the training difficulties of deep networks by introducing residual connections, making it easier for gradients to propagate backward, thus achieving effective training of the super-powerful network. The ResNet50 network has 50 layers, and its huge number of parameters gives it a more powerful feature learning and extraction capability. It is a widely used feature extraction network in object detection tasks. In this application, the feature maps of four layers of ResNet50, namely blocks 2, 3, 4 and 5, are extracted. Blocks 2 and 3 extract low-dimensional features to represent the geometric structure information such as the contour and texture of the target; blocks 4 and 5 extract high-dimensional features to express the semantic information of the image.

[0089] S3: Input the feature map extracted in S2 into the multi-scale fusion attention mechanism module based on the improved convolutional channel attention mechanism, and use the multi-scale self-attention mechanism to perform weighted fusion of the feature map to obtain a weighted feature map.

[0090] Attention mechanisms are a powerful tool in deep learning. They can learn to assign weights to different locations or features, enabling dynamic selection and concentration of input information. Specifically, this includes the following steps:

[0091] S3.1: Expand the feature map dimension to the same dimension (C3, H3, W3) as the block3 feature map by 1×1 convolution and upsampling, and then concatenate the four feature maps along the channel direction to obtain the feature map X0, with dimensions denoted as (C, H, W).

[0092] S3.2: Pass the feature map X0 through two 3×3 convolutional layers, denoted as C1 and C2 respectively, with padding set to 1, to obtain feature maps X1(C,H,W) and X2(C,H,W);

[0093] S3.3: Obtain the fused feature layers Y1 and Y2 according to formulas (1) and (2):

[0094] Y1=C1(X0)+X0 (3)

[0095] Y2=C2(X1)+Y1 (4)

[0096] S3.4: Dimensionality reduction of Y1 and Y2 is performed by mean pooling, and the resulting weights W1 and W2 have dimensions of (C,1,1);

[0097] S3.5: Input W1 and W2 into the FC layer, which consists of two fully connected layers and a last sigmoid activation function layer, to obtain the final fusion weight W. F(C,1,1);

[0098] S3.6: Merge weight W F Multiplying the original feature map X0 by the original feature map X0 yields a weighted feature map X(C,H,W) obtained through a multi-scale fusion attention mechanism, with fusion weights W. F The attention allocation weights learned through neural networks can adaptively adjust the distribution of attention, enabling dynamic selection of input information. This allows for better weighting of channels in multi-level feature maps, retaining important features and eliminating redundant features.

[0099] S4: The weighted feature map from S3 is input into the multi-scale feature fusion module. After cross-scale feature fusion, the final feature map is output. Since the feature extraction network ResNet50 used in this application is a traditional CNN structure, it has certain disadvantages in combining low-dimensional and high-dimensional features. In order to enable the extracted feature map to combine features of different spatial scales and allow the model to have different perspectives for better multi-object detection, the multi-scale feature fusion module proposed in this application can fuse features at different semantic levels, realizing feature representation from shallow to deep. The network structure of the proposed feature fusion module is as follows: Figure 3 As shown, the calculation steps are as follows:

[0100] S4.1: Perform global mean pooling and global max pooling on the weighted feature map X(C,H,W) obtained in S3 to obtain two feature maps with the same dimension P1(2,H,W) after dimensionality reduction, and then concatenate the other channel dimensions to obtain feature map P2(2,H,W).

[0101] S4.2: Perform deconvolution on the obtained feature map to obtain a feature map with dimension P3(2,2H,2W), with a kernel size of 7×7, a stride of 2, and padding = 3;

[0102] S4.3: Perform two convolutions on the feature maps to obtain feature maps P4 and P5 with dimensions (1,H,W). The size of the convolution kernel is 7×7, the padding is 3, the stride of the two convolutions is 1 and 2 respectively, and the number of output channels is 2 and 1 respectively.

[0103] S4.4: After linearly activating the fused feature map P5 with the Sigmoid function, multiply it with X(C,H,W) to obtain the final fused feature X′(C,H,W).

[0104] S5: Add positional encoding to the final feature map output from S4, and input it along with the target query sequence into the Transformer encoder. The positional encoding is added using sine and cosine functions to ensure the data carries accurate positional information. The formula is as follows:

[0105]

[0106] Where pos represents the index of the feature point in the feature map, and d model This is the model dimension, where i represents the dimension index for position encoding;

[0107] The target query sequence is a set of vectors input into the Transformer decoder, typically with a length of 100, representing the maximum number of targets to be detected per image. The target query vector provides a unique query vector for each predicted bounding box, enabling the decoder to predict the category and regression result of the corresponding box based on each query vector, thus achieving a correspondence between the bounding box and the prediction result. The dimension of the query vector is equal to the feature dimension of the encoder output, allowing the query vector to interact and match with the global features of the encoder, generating a prediction output with the corresponding category and coordinates for each predicted bounding box.

[0108] S6: The feature map with added position encoding and the target query sequence obtained in S5 are interacted with the Transformer, and the loss is calculated using R-Focal loss and backpropagation is performed to optimize the model, completing the entire target detection process. The target detection process is implemented through the complete DETR, which includes the feature extraction network in S1-S4 and the Transformer in S5-S6. The Transformer architecture is used to replace CNN so that the target detection network can better obtain global information.

[0109] In this application, the object detection network is an improvement upon the DETR network, replacing the traditional cross-entropy loss with R-Facol loss. DETR is a Transformer-based object detection model proposed in recent years, mainly composed of a feature extraction network and a Transformer. The feature extraction network used in this application is ResNet50. The main innovation of DETR is that it treats object detection as a set prediction problem. Specifically: ① DETR predicts all object boxes in the image as a whole, instead of generating multiple anchor boxes for prediction one by one as in existing methods. This allows it to utilize the dependencies between predicted boxes for joint optimization, achieving higher accuracy. ② DETR uses the object query sequence as input to the decoder, with each sequence corresponding to a predicted box, and uses the Hungarian algorithm to calculate the bipartite graph matching loss. Ultimately, each object in the image will only get one predicted box, allowing the model to focus on correct prediction and matching, thereby improving the efficiency of model training. Overall, the Transformer part of DETR consists of an encoder, a decoder, and a prediction head, as detailed below;

[0110] The encoder consists of multiple stacked encoder layers, each connected sequentially to two sub-layers: a multi-head attention mechanism (MHA) and a position feedforward network (PWFFN). The sub-layers employ residual jumpers and layer normalization to ensure sufficient training of sub-layer parameters, accelerating convergence. The MHA addresses the limitation of self-attention mechanisms, which tend to overemphasize their own position when encoding current location information. By using h different attention heads to perform different linear projections on the data, different queries Q, keys K, and values ​​V are learned. The h groups of different Q, K, and V values ​​are then merged in parallel. Finally, the pooled outputs of the h groups are concatenated and transformed through another learnable linear projection to produce the final output. The calculation formula is shown below:

[0111] MH(Q,K,V)=Concat(head1,head2...head h W O

[0112]

[0113] The learnable parameters include and

[0114] The decoder consists of multiple stacked decoder layers, each with three sequentially connected sub-layers: a masked multi-head attention layer (masked MHA), MHA, and PWFFN. The masked MHA is identical to the MHA in the encoder block except for the addition of a mask; the mask indicates that certain values ​​are masked so that they do not take effect during parameter updates. In time series prediction, for a time step t, the decoder's output only needs to depend on the outputs before time t. Therefore, applying the masked MHA to the decoder's input yields previously predicted output information, essentially recording information between the current input and the current time step.

[0115] It should be noted that, unlike the traditional Transformer, the query sequence input to DETR's Transformer decoder is the target query sequence. Its purpose is to provide a unique query vector for each predicted box. This allows the decoder to generate the predicted output of the corresponding box based on each query vector, thus achieving a correspondence between the box and the output result.

[0116] The prediction head consists of two fully connected layers for generating detection output. The first fully connected layer is used for box category prediction, outputting the category of the box. The second layer is used for box coordinate regression prediction, outputting the center x-coordinate, y-coordinate, height, and width of the box. Each attention mechanism module of the decoder outputs a feature vector, which represents the relevant image region corresponding to the query vector. The vector is concatenated with the corresponding query vector and input into the two fully connected layers to obtain the corresponding category and coordinates. R-Focal loss and SmoothL1 loss are used for loss calculation.

[0117] R-Focalloss is an improvement on Focal loss. It can effectively solve the problems of class imbalance in industrial datasets and insufficient sample size for a small number of classes. Its main ideas are reflected in two aspects: ① controlling the weights of positive and negative samples; ② controlling the weights of easy-to-classify and hard-to-classify samples.

[0118] The part of R-Focal loss that controls the weights of positive and negative samples is:

[0119]

[0120] The value of α ranges from 0 to 1. By controlling the value of α, the contribution of positive and negative samples to the cross-entropy loss is controlled. α is generally set below 0.5 to balance the values ​​of positive and negative samples when calculating the loss, so as to avoid the loss weight being too large when there are many negative samples, which would affect the recognition of positive samples.

[0121] The part of R-Focal loss that controls the weights of samples that are easy or difficult to classify is:

[0122]

[0123] The value of γ is greater than 0. When γ is 0, R-Focalloss is the traditional cross-entropy loss. The larger the value of γ, the more the model will focus on samples that are difficult to classify. However, if the value of γ is too large, it will also cause the model to ignore samples that are easy to classify, thereby reducing the recall value. Therefore, it is very important to choose an appropriate value of γ.

[0124] The complete formula for calculating R-Focal loss is as follows:

[0125]

[0126] To address the difficulty in selecting α and γ in traditional Focal loss, R-Focal loss employs a random search method to select the hyperparameters of the Focal loss, resulting in better generalization. The specific steps include:

[0127] S6.1: Let the ranges for random searches of points α and γ be α = [0.1, 0.2, 0.3, 0.4, 0.5] and γ = [1, 2, 3, 4, 5], respectively;

[0128] S6.2: During each batch of training, randomly select a set of α and γ for loss calculation, and record the proportion of loss reduction caused by each set of hyperparameters. The larger the proportion, the greater the contribution of this set of hyperparameters to the loss.

[0129] S6.3: Every 20-30 epochs, the worst-performing hyperparameters are removed, and the best hyperparameter combinations are retained;

[0130] S6.4: Repeat S6.3 until the optimal combination of hyperparameters is selected for training.

[0131] After the data is processed through two fully connected layers to obtain the types and coordinates of the predicted boxes, the DETR uses the Hungarian algorithm to find the best global match between the predicted boxes and the ground truth boxes. Instead of matching each predicted box to a ground truth box and then performing post-processing to remove redundant boxes, it only considers the loss between the ground truth box and the matched predicted box. Specifically:

[0132] A: Treat the predicted bounding boxes as one set of nodes in a bipartite graph, and the ground truth bounding boxes as another set of nodes;

[0133] B: Calculate the IoU between each predicted bounding box and all ground truth bounding boxes as its matching degree;

[0134] C: Select the ground truth box index with the largest IoU for each predicted box; select the predicted box index with the largest IoU for each ground truth box, generate index matching relationships based on this, and form the maximum matching of the bipartite graph;

[0135] D: Only calculate the loss between the predicted box and the ground truth box in the maximum match, including class loss and regression loss;

[0136] E: Summing the loss between all matching boxes;

[0137] F: Repeat D and E to update the DETR model.

[0138] S7: Input defect images obtained from actual industrial scenarios into the network for detection to obtain the final detection results.

[0139] To validate this application, extensive experiments were conducted on an aero-engine X-ray image dataset to detect various defects in engine blades.

[0140] This application compares the detection accuracy (AP50) with a large number of existing models, such as... Figure 4As shown, the AP50 of the present invention is the highest (94.2) among all model comparisons, surpassing the second-place YOLOX (87.3) by 6.9 points, demonstrating the superiority of the present invention.

[0141] Figure 5 The paper employs a visualization of the attention mechanism to demonstrate the image locations that the model focuses on during image recognition, showcasing the DETR model's adaptability to complex scenes and its advantages in object detection by leveraging the global modeling capabilities of the Transformer.

[0142] Figure 6 This paper demonstrates the improvements in model accuracy and recall brought about by the proposed data augmentation method, multi-scale fusion attention mechanism module, feature fusion module, and R-Focal loss. It can be seen that, regarding model accuracy, the feature fusion module improves the AP50 by 3.7, effectively enhancing the model's detection accuracy. For recall, R-Focal loss shows a more significant improvement, primarily because it better matches hyperparameters, allowing for more targeted and effective tuning of positive / negative samples and samples with varying difficulty levels. In summary, this invention effectively improves the detection accuracy and efficiency in the field of aero-engine blade defect detection, achieving end-to-end detection and making it more suitable for industrial deployment.

[0143] Although embodiments of the present invention have been described above in conjunction with the accompanying drawings, the present invention is not limited to the specific embodiments and application fields described above. The specific embodiments described above are merely illustrative and instructive, and not restrictive. Those skilled in the art can make many other modifications based on the guidance of this specification and without departing from the scope of the claims of the present invention, and all of these modifications are within the scope of protection of the present invention.

Claims

1. A method for detecting defects in aero-engine blades based on multi-scale DETR, characterized in that: Specifically, the following steps are included: S1: Acquire X-ray image datasets of defective blades and perform data augmentation on the imbalanced dataset; The X-ray image dataset was obtained by capturing images of the blades with a dedicated X-ray camera and then processing them by computer. The data augmentation methods used were Mixup, Mosaic, and Mix-Mos methods to alleviate data imbalance and prevent overfitting. Specifically: The Mixup method generates images with new structures and probability distributions by linear interpolation or stitching together different images, increasing the diversity of the dataset. Assuming two training images... and corresponding tags New image generated by Mixup and tags The calculation is as follows: It is in The random value between the two original images controls the blending ratio. When the value approaches 0.5, the new sample contains richer information from the original sample, maximizing the diversity of the data and improving the model's generalization ability. The Mosaic method constructs entirely new training images by stitching together images while maintaining the complete label information of the original images, thereby maximizing the diversity of data, providing the model with richer semantic information, and improving detection and recognition performance. The Mix-Mos method first uses the Mixup method for data augmentation, and then applies the Mosaic method. The continuity of image content is obtained through Mixup, and the richness of image information is obtained through Mosaic. S2: Input the data augmented in S1 into the feature extraction network ResNet50, and extract the feature maps of four layers of ResNet50: block2, block3, block4 and block5 respectively. Block2 and block3 extract low-dimensional features to represent the geometric structure information such as the contour and texture of the target; block4 and block5 extract high-dimensional features to express the semantic information of the image. S3: Input the feature map extracted in S2 into the multi-scale fusion attention mechanism module, which is an improvement on the convolutional channel attention mechanism. The feature map is weighted and fused using a multi-scale self-attention mechanism to obtain a weighted feature map. The multi-scale fusion attention mechanism is an improvement on the convolutional channel attention mechanism and includes the following steps: S3.1: Through Convolution and upsampling expand the feature map dimension to the same dimension as the block3 feature map. The four feature maps are then stitched together along the channel direction to obtain the feature map. Dimension is denoted as ; S3.2: Transfer the feature map After two floors The convolutional layers are denoted as follows: and Setting padding to 1 yields the feature map. and ; S3.3: Obtain the fused feature layer according to formulas (1) and (2). and : S3.4: To and Dimensionality reduction is performed using mean pooling, resulting in weights. and The dimension is ; S3.5: Will and The input is an FC layer, which consists of two fully connected layers and a final sigmoid activation function layer, yielding the final fused weights. ; S3.6: Merge weights Compared with the original feature map Multiplication yields a weighted feature map obtained through a multi-scale fusion attention mechanism. ; S4: Input the weighted feature map from S3 into the multi-scale feature fusion module, and output the final feature map after cross-scale feature fusion. S5: Add positional encoding to the final feature map output from S4 and input it into the Transformer encoder along with the target query sequence; S6: The feature map with added position encoding and the target query sequence obtained in S5 are interacted with the Transformer, and the loss is calculated using R-Focal loss and backpropagation is performed to optimize the model, thus completing the entire target detection process. The target detection process is implemented through the complete DETR, which includes the feature extraction network in S1-S4 and the Transformer in S5-S6. The Transformer architecture is used instead of CNN to enable the target detection network to better acquire global information. S7: Input defect images obtained from actual industrial scenarios into the network for detection to obtain the final detection results.

2. The method for detecting defects in aero-engine blades based on multi-scale DETR according to claim 1, characterized in that: The multi-scale feature fusion module in S4 is based on multiple pooling methods and an improved spatial attention mechanism, and specifically includes the following calculation steps: S4.1: The weighted feature map obtained in S3 Performing global mean pooling and global max pooling respectively yields two dimensionality-reduced results with the same dimension. The feature map is obtained by concatenating the feature maps of other channels. ; S4.2: Perform deconvolution on the obtained feature map to obtain a dimension of Feature map, convolution kernel size is The sliding step size is 2, and the padding is 3. S4.3: Perform two convolutions on the feature map to obtain a dimension of... Feature map and The size of the convolution kernel is padding=3, the stride of the two convolutions is 1 and 2 respectively, and the number of output channels is 2 and 1; S4.4: Merge the feature maps After linear activation via the Sigmoid function and... Multiplying them together yields the final fusion feature. .

3. The method for detecting defects in aero-engine blades based on multi-scale DETR according to claim 1, characterized in that: In step S5, the location encoding is added using sine and cosine functions to allow the data to carry accurate location information. The formula is as follows: Where pos represents the index of the feature point in the feature map. It represents the model dimension, where i indicates the dimension index for the position encoding.

4. The method for detecting defects in aero-engine blades based on multi-scale DETR according to claim 1, characterized in that: In S5, the target query sequence is a set of vectors input into the Transformer decoder. The target query vector provides a unique query vector for each predicted box, enabling the decoder to predict the category and regression result of the corresponding box based on each query vector, thereby achieving the correspondence and matching between the box and the prediction result. The dimension of the query vector is equal to the feature dimension output by the encoder, enabling the query vector to interact and match with the global features of the encoder, generating a prediction output with the corresponding category and coordinates for each predicted box.

5. The method for detecting defects in aero-engine blades based on multi-scale DETR according to claim 1, characterized in that: The Transformer part of the DETR consists of an encoder, a decoder, and a prediction head; The encoder consists of multiple stacked encoder layers, each connected sequentially to two sub-layers: a multi-head attention mechanism (MHA) and a position feedforward network (PWFFN). The sub-layers employ residual jumpers and layer normalization to ensure sufficient training of sub-layer parameters, accelerating convergence. The MHA addresses the limitation of self-attention mechanisms, which tend to overemphasize their own position when encoding current location information. It learns different queries Q, keys K, and values ​​V by using h different attention heads to perform different linear projections on the data. The h groups of different Q, K, and V values ​​are then merged in parallel, and finally, the pooled outputs of the h groups are concatenated and connected through another learnable linear projection. The changes are dimensional. The final output is generated using the following formula: Learnable parameters include , , and ; The decoder consists of multiple stacked decoder layers, each of which has three sequentially connected sub-layers: a masked multi-head attention layer (masked MHA), MHA, and PWFFN. The masked multi-head attention layer is the same as the MHA in the encoder block except that a mask is added. The mask indicates that certain values ​​are masked so that they do not take effect when parameters are updated. The prediction head consists of two fully connected layers to generate detection output. The first fully connected layer is used for box category prediction, outputting the category of the box. The second layer is used for box coordinate regression prediction, outputting the center x-coordinate, y-coordinate, height, and width of the box. Each layer of the decoder's attention mechanism module outputs a feature vector, which represents the relevant image region corresponding to the query vector. The vector is concatenated with the corresponding query vector and input into the two fully connected layers to obtain the corresponding category and coordinates. R-Focal loss and SmoothL1 loss are used for loss calculation. The R-Focal loss is an improvement upon Focal loss. The part of the R-Focal loss that controls the weights of positive and negative samples is as follows: in The value range is 0-1, and it is controlled by... The value controls the contribution of positive and negative samples to the cross-entropy loss; The part of the R-Focal loss that controls the weights of samples classified as easy or difficult is: in The range of values ​​is greater than 0; The complete formula for calculating R-Focal loss is as follows: The R-Focal loss uses a random search method to select the hyperparameter settings of the Focal loss, resulting in better generalization.

6. The method for detecting defects in aero-engine blades based on multi-scale DETR according to claim 5, characterized in that: The R-Focal loss employs a random search method to select the hyperparameters of the Focal loss, in order to solve... and The difficulty in making a choice includes the following steps: S6.1: Setting points and The ranges for random search are respectively and ; S6.2: Randomly select a group during each batch of training. and Perform loss calculation and record the percentage reduction in loss caused by each set of hyperparameters. The larger the percentage, the greater the contribution of this set of hyperparameters to the loss. S6.3: Every 20-30 epochs, the worst-performing hyperparameters are removed, and the best hyperparameter combinations are retained; S6.4: Repeat S6.3 until the optimal combination of hyperparameters is selected for training.

7. The method for detecting defects in aero-engine blades based on multi-scale DETR according to claim 5, characterized in that: After the data is processed through two fully connected layers to obtain the types and coordinates of the predicted bounding boxes, the DETR uses the Hungarian algorithm to find the best global match between the predicted bounding boxes and the ground truth bounding boxes. Specifically: A: Treat the predicted bounding boxes as one set of nodes in a bipartite graph, and the ground truth bounding boxes as another set of nodes; B: Calculate the IoU between each predicted bounding box and all ground truth bounding boxes as its matching degree; C: Select the index of the ground truth box with the largest IoU for each predicted box; For each ground truth bounding box, select the index of the predicted bounding box with the largest IoU, and generate index matching relationships based on this to form the maximum matching of the bipartite graph; D: Only calculate the loss between the predicted box and the ground truth box in the maximum match, including class loss and regression loss; E: Summing the loss between all matching boxes; F: Repeat D and E to update the DETR model.

8. The method for detecting defects in aero-engine blades based on multi-scale DETR according to claim 1, characterized in that: The test images in S7 are no longer data augmented; the original images are simply input into the network for testing.

Images