Aero-engine surface defect detection method and electronic equipment

Abstract

The invention relates to an aero-engine surface defect detection method and electronic equipment. The method comprises the following steps: acquiring an aero-engine surface image; the marked surface defect data set is put into an MBE-YOLOv5 surface defect detection model for training; inputting a to-be-detected aero-engine surface image into the trained MBE-YOLOv5 surface defect detection model, and outputting an aero-engine surface defect detection result; according to the MBE-YOLOv5 surface defect detection model, a YOLOv5 model is improved, the MBE-YOLOv5 surface defect detection model takes MobileNetV3Marge as a backbone network, a neck part is optimized to be BiFPN, and an ECA attention mechanism is added in front of a detection head. Compared with the prior art, the method has the advantages of being high in detection precision, high in detection efficiency, easy to deploy on resource-limited equipment and platforms, wide in applicability and the like.

Description

Technical Field

[0001] The invention relates to the field of aircraft engine surface defect detection, and in particular to an aircraft engine surface defect detection method and electronic equipment. Background Art

[0002] With the rapid development of the aviation industry, aircraft surface defect detection has become an important part of ensuring flight safety. Traditional aircraft engine surface inspection methods mainly rely on manual inspection and some traditional non-destructive testing technologies, such as borescope inspection, eddy current inspection and ultrasonic inspection. Although these methods can detect defects to a certain extent, they have problems such as low efficiency, high cost and susceptibility to human factors.

[0003] In recent years, deep learning-based object detection technology has developed rapidly and has been widely used in the field of defect detection. At present, the existing technical solutions for aircraft engine surface defect detection mainly include the following:

[0004] Detection methods based on traditional image processing: These methods usually achieve defect detection through steps such as image preprocessing, feature extraction, and classifiers. Although these methods have achieved certain results in some applications, they have poor detection accuracy and robustness when faced with complex backgrounds and diverse defects, and require large computing resources.

[0005] Detection methods based on deep learning: such as Faster R-CNN, SSD and YOLO series algorithms. These methods automatically extract image features through convolutional neural networks and perform target detection. As an efficient target detection algorithm, YOLOv5 has attracted widespread attention due to its fast detection speed and high accuracy. The original YOLOv5 results are as follows: Figure 1 , it still has certain limitations when it comes to defect detection in small defect targets and complex backgrounds.

[0006] After searching, Chinese invention patent publication number CN111833328B discloses a method for detecting surface defects of aircraft engine blades based on deep learning, including: collecting images of aircraft engine blades, manually annotating images containing surface defects, and constructing a surface defect image dataset; preprocessing the surface defect image dataset; constructing a deep convolution defect detection network for detecting defect labels and defect categories, and using the preprocessed surface defect image dataset to train the deep convolution defect detection network; wherein the deep convolution defect detection network uses a parallel ResNet network and a DenseNet network to extract features from the input blade image at the same time, superimposes the two extracted features at the corresponding positions for feature fusion, and then combines the fused features for defect classification; imports the image to be detected into the trained deep convolution model, and outputs the defect label and defect category. This existing patent has the problem of low detection accuracy and efficiency.

[0007] How to achieve accurate and efficient detection of aircraft engine surface defects has become a technical problem that needs to be solved. Summary of the invention

[0008] The purpose of the present invention is to provide an aircraft engine surface defect detection method and electronic equipment in order to overcome the defects of the above-mentioned prior art.

[0009] The purpose of the present invention can be achieved by the following technical solutions:

[0010] According to one aspect of the present invention, a method for detecting surface defects of an aircraft engine is provided, the method comprising the following steps:

[0011] Step S1, acquiring an aircraft engine surface image;

[0012] Step S2: Put the labeled surface defect dataset into the MBE-YOLOv5 surface defect detection model for training.

[0013] Step S3, inputting the surface image of the aircraft engine to be detected into the trained MBE-YOLOv5 surface defect detection model, and outputting the aircraft engine surface defect detection result;

[0014] The MBE-YOLOv5 surface defect detection model is an improvement on the YOLOv5 model. The MBE-YOLOv5 surface defect detection model uses MobileNetV3 Large as the backbone network, optimizes the neck to BiFPN, and adds an ECA attention mechanism before the detection head.

[0015] Preferably, the backbone network of the MBE-YOLOv5 surface defect detection model includes a CBS module, a MobileNetV3 network module and a SPPF module;

[0016] The backbone network first performs a 1×1 convolution operation on the input feature map to adjust the dimension of the feature map, then the feature map passes through a series of inverted residual structures, and finally processes the feature map through the SPPF module to output feature maps of different scales to the neck.

[0017] More preferably, the 4th layer MobileNetV3 and the 6th layer MobileNetV3 of the backbone network are spliced ​​with the 12th layer BiFPN and the 16th layer BiFPN of the neck respectively to obtain the first feature map and the second feature map respectively, and the 6th layer MobileNetV3 is then spliced ​​with the 14th layer to generate the third feature map, which is output to the detection head.

[0018] Preferably, the neck of the MBE-YOLOv5 surface defect detection model uses a BiFPN structure to perform bidirectional information fusion on feature maps of different scales, and a weighted feature fusion mechanism is used in the feature fusion process of each scale.

[0019] More preferably, the method comprises: before training the MBE-YOLOv5 surface defect detection model, using a K-means clustering algorithm, obtaining optimized anchor points adapted to the surface defect data set according to a genetic algorithm, and putting them into the detection model for training.

[0020] More preferably, the process of obtaining an optimized anchor point adapted to the surface defect data set according to a genetic algorithm comprises:

[0021] Step S3-1, read the width and height of each image in the training set, as well as the width and height of all bounding boxes;

[0022] Step S3-2, scaling the maximum width and height of each image to a specified size in proportion;

[0023] Step S3-3, converting the bounding box from relative coordinates to absolute coordinates;

[0024] Step S3-4, filtering bounding boxes, and retaining bounding boxes whose width and height are both greater than or equal to two pixels;

[0025] Step S3-5, using K-means clustering algorithm to obtain k anchor points;

[0026] Step S3-6, using a genetic algorithm to randomly mutate the width and height of the anchor point, if the effect becomes better after the mutation, assign the result after the mutation to the anchor point; if the effect becomes worse after the mutation, skip it;

[0027] Step S3-7, sort the anchor points finally obtained by mutation by area and return them.

[0028] Preferably, the method comprises: preprocessing the aircraft engine surface image to obtain a surface defect image, wherein the preprocessing comprises performing data enhancement by inversion, translation, brightness change, and noise enhancement to increase the number of data samples.

[0029] More preferably, the method further comprises: annotating the defects in the surface defect image to obtain an annotation file, and the surface defect image and the corresponding annotation file constitute a surface defect dataset.

[0030] More preferably, the defects in the surface defect image are annotated to form an annotation file for each image, and the data format in the annotation file is set according to the YOLO format;

[0031] The annotation file includes defect category information and defect location information.

[0032] According to another aspect of the present invention, an electronic device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the method described above is implemented when the processor executes the program.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] 1) The present invention optimizes the network structure of the original YOLOv5. Aiming at the complex and diverse backgrounds and lightweight processing requirements of aircraft engine surface images, the original backbone network is replaced with MobileNetV3Large, aiming to utilize the high efficiency and lightweight characteristics of MobileNetV3, which is suitable for deployment on mobile terminals to reduce computing overhead and speed up detection while maintaining good performance; in view of the diversity and subtlety of aircraft engine surface defects, BiFPN is added to the neck, which can more effectively fuse multi-scale features and improve the model's detection accuracy for targets of different sizes, thereby improving the overall detection performance; in view of the environmental interference on the surface of the aircraft engine, the ECA attention mechanism is added to the detection head to enhance the focus on specific features and improve the overall accuracy of the model; thus, the present invention can provide more accurate detection for targets of different sizes and improve the model's detection accuracy and efficiency for small and large targets.

[0035] 2) The present invention optimizes the network structure of YOLOv5, and BiFPN adopts a bidirectional feature fusion mechanism and a weighted feature fusion strategy. The weighted fusion method adopts fast normalization fusion, which makes the network training speed very fast. The design principle of MobileNetV3 is combined with deep separable convolution and hardware-friendly optimization, which significantly reduces the computational overhead while maintaining high performance and improves detection efficiency.

[0036] 3) The ECA attention mechanism of the present invention can enhance the model's attention to key feature areas, involve fewer parameters and calculations, and improve the detection accuracy and efficiency of minor defects, especially in the presence of environmental interference.

[0037] 4) The lightweight detection network of the present invention is easier to deploy on resource-constrained devices and platforms, so that the model has higher performance and wider applicability in practical applications.

[0038] 5) By replacing the initial target box loss function IoU with GIoU, the model can better adjust the predicted box for the data set to make it closer to the real box, thereby improving the convergence speed and detection accuracy of the model; redundant detection boxes are removed through NMS non-maximum suppression, the final detection results are retained, and false detections and missed detections are reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a schematic diagram of the structure of the YOLOv5 network;

[0040] Figure 2 It is a schematic diagram of the structure of MobileNetV3 Large in the present invention;

[0041] Figure 3 Schematic diagram of the channel separation convolution process in the present invention;

[0042] Figure 4 It is a schematic diagram of the structure of the SE attention mechanism of MobileNetV3 Large in the present invention;

[0043] Figure 5 Schematic diagram of the ECA attention mechanism structure of the detection head in the present invention;

[0044] Figure 6 Schematic diagram of the BiFPN structure of the neck in the present invention;

[0045] Figure 7 It is a schematic diagram of the structure of the MBE-YOLOv5 surface defect detection model in the present invention. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0047] Aircraft engine surface images have their own characteristics, including high reflectivity materials, complex geometries, diverse defect types, high resolution requirements, and environmental interference.

[0048] a) High reflectivity materials: The surface of aircraft engines is usually made of metal materials, such as titanium alloy and aluminum alloy, which have high reflectivity. Under lighting conditions, the surface is prone to strong reflections or shadows, affecting the image quality and the detection accuracy of the algorithm.

[0049] b) Complex geometry: The surface of an aircraft engine has complex geometry, including curved and uneven surfaces, which may cause defects to appear differently at different viewing angles. This complex shape increases the difficulty of defect identification.

[0050] c) Various defect types: There may be many types of defects on the surface of aircraft engines, such as scratches, corrosion, cracks, dents, etc. These defects vary in shape, size and color, requiring the detection algorithm to have high resolution and flexibility.

[0051] d) High resolution requirement: In order to detect tiny defects, aircraft engine surface images usually require high resolution. This means that the detection algorithm needs to be able to process large-size images while ensuring computational efficiency while maintaining detection accuracy.

[0052] e) Environmental interference: During the actual inspection process, there may be environmental interference such as oil, dust, etc. on the surface of the aircraft engine. These factors will have a negative impact on the image quality and increase the difficulty of defect detection.

[0053] This embodiment relates to an aircraft engine surface defect detection method, which is an aircraft engine surface defect detection method based on MBE-YOLOv5 (YOLOv5 improved based on MobileNetV3-BiFPN-ECA). The method uses an improved YOLOv5 network model to perform defect detection based on the characteristics of aircraft engine surface images, and improves the YOLOv5 network structure, including:

[0054] 1) The backbone network is replaced with MobileNetV3 Large, such as Figure 2 .

[0055] Due to the complex and diverse background and lightweight processing requirements, the YOLOv5 backbone network is optimized and replaced with modules suitable for deployment on mobile terminals, making the model lightweight and enabling the model to run efficiently in embedded hardware or real-time detection systems.

[0056] MobileNetV3 Large is used as the backbone network. Its high efficiency and lightweight characteristics reduce the number of model parameters. It is targeted at defect detection on the surface of aircraft engines and facilitates mobile deployment. The use of a new backbone network improves computing efficiency and speeds up reasoning.

[0057] The main features of MobileNetV3 include: channel separable convolution, SE module, h-swish and h-sigmoid and bneck.

[0058] (1) Channel separation convolution is the main feature of the MobileNet series and the main factor that makes it lightweight. Figure 3 , the channel separable convolution is divided into two processes:

[0059] ① Channel direction channel separable convolution;

[0060] ② Normal 1X1 convolution outputs the specified number of channels (such as Figure 3 ).

[0061] (2) SE module (such as Figure 4 )

[0062] SE attention mechanism (Squeeze-and-Excitation Networks) is a module used to enhance the representation ability of convolutional neural networks. It improves the representation ability of the network by explicitly modeling the interdependencies between convolutional feature channels. The SE module mainly consists of three steps: Squeeze, Excitation, and Scale.

[0063] (3) h-swish and h-sigmoid use approximate operations to simulate swish and relu. The ReLU6 function is chosen because the optimized implementation of ReLU6 can be used on almost all software and hardware frameworks; at the same time, it can eliminate the potential loss of numerical accuracy caused by different implementations of approximate sigmoid in specific modes. The formula is as follows

[0064]

[0065] Among them, h-swish is a variant of Swish, x is the input value of the activation function, and ReLU6 is a variant of the ReLU (Rectified Linear Unit) function.

[0066] (4) Bneck is the core module of MobileNetV3 and the basic module of the network. It mainly implements channel-separable convolution + SE channel attention mechanism + residual connection.

[0067] The design principle of MobileNetV3 combines depth-wise separable convolutions and hardware-friendly optimizations, which significantly reduces computational overhead while maintaining high performance.

[0068] 2) Add ECA attention mechanism, the structure is as follows Figure 5 .

[0069] The ECA (Efficient Channel Attention) attention mechanism is a mechanism designed to improve the performance of deep convolutional neural networks (CNNs) by focusing on the most important features in the data. Its main features are as follows:

[0070] Avoid dimensionality reduction: Unlike traditional attention mechanisms, ECA maintains the original dimensions of the data, which helps preserve important information.

[0071] Local cross-channel interaction: ECA uses 1D convolution to interact between different channels without reducing their dimensionality. This approach ensures that the model is efficient while still capturing important features.

[0072] Adaptive kernel size: The kernel size of 1D convolution is adaptively selected according to the channel dimension, allowing effective local interactions.

[0073] Efficient performance: Compared with other attention mechanisms, ECA involves fewer parameters and computations, making it suitable for real-time applications.

[0074] The ECA attention mechanism can enhance the model’s attention to key feature areas and improve the ability to detect tiny defects, especially in the presence of environmental interference.

[0075] 3) Add BiFPN to the neck:

[0076] Taking into account the diversity and subtlety of aircraft engine surface defects, the feature extraction module of YOLOv5 is improved, and Bi-directional Feature Pyramid Network (BiFPN) is added to the neck structure of YOLOv5 to better capture defect features of different scales.

[0077] BiFPN enhances the model's ability to detect objects of different sizes by bidirectionally fusing multi-scale features. Its bidirectional feature fusion method improves the representation ability of the feature pyramid and significantly improves the detection accuracy. BiFPN is an improved FPN network structure, mainly used for target detection tasks. The structure is weighted and bidirectionally connected, that is, a top-down and bottom-up structure. It realizes cross-scale connection by constructing a bidirectional channel, and directly fuses the features in the feature extraction network with the relative size features in the bottom-up path, retaining shallower semantic information without losing too much deep semantic information.

[0078] In traditional feature fusion, feature maps of different scales are weighted with the same weight. However, when the resolution of the input feature maps is different, the output feature maps are not equal if the same weight is used. Therefore, BiFPN sets different weights according to the importance of different input features, and repeatedly uses this structure to strengthen feature fusion.

[0079] The weighted fusion method in the BiFPN structure uses fast normalized fusion, which is proposed for slow training speed. The weights are scaled to the range of 0 to 1. Since the Softmax method is not used, the training speed is very fast. The cross-scale connection is achieved by adding a skip connection and a bidirectional path, thus realizing weighted fusion and bidirectional cross-scale connection. The BiFPN structure diagram is shown in the attached figure. Figure 6 shown.

[0080] 4) The target box function is replaced by GIoU:

[0081] The GIoU loss function is an improvement on the IoU loss function in the original YOLOv5. IoU only focuses on the overlapping area between the predicted box and the true box, while GIoU not only focuses on the overlapping area, but also considers the distance between the two boxes and the non-overlapping area. This makes GIoU more effective when dealing with non-overlapping or partially overlapping boxes. The calculation formula is as follows:

[0082]

[0083] Among them, A and B are the predicted box and the true box respectively, and C is the minimum closure area containing A and B. By maximizing GIoU, the model is able to better adjust the predicted box to make it closer to the true box.

[0084] The improved MBE-YOLOv5 overall model structure is as follows Figure 7 shown.

[0085] The method comprises the following steps:

[0086] Step S1, obtain the surface image of the aircraft engine, and pre-process the surface image to obtain the surface defect image, perform data enhancement through operations such as inversion, translation, brightness change, and noise enhancement, increase the number of data samples, and make the model more robust through data enhancement, so that it can better adapt to various lighting conditions and viewing angles in practical applications. For problems with high reflectivity and complex geometric shapes, design image enhancement methods such as lighting changes, reflection simulation, mirror rotation, etc. In the image preprocessing stage, add adaptive contrast enhancement and noise suppression algorithms to improve the detectability of the image.

[0087] Step S2, label the defects in the image to obtain a labeling file, and use the surface defect image and the corresponding labeling file as the surface defect data set; label the defects in the image through LabelImg to form a labeling file txt for each image, and the data format in the labeling file txt is set according to the YOLO format: category, marking box center coordinates (x, y), marking box width w, height h display, that is, including defect category information and defect location information. The surface defect image and the corresponding labeling file constitute a surface defect data set.

[0088] Step S3, optimize the anchor points according to the surface defect data set to obtain the optimized anchor points adapted to the surface defect data set: use the K-means clustering algorithm and the genetic algorithm to obtain the optimized anchor points adapted to the surface defect data set. The main steps are as follows:

[0089] Step S3-1, read the width and height of each image in the training set, as well as the width and height of all bounding boxes (BoundingBox).

[0090] Step S3-2, scale the maximum width and height of each image to the specified size (img_size) in proportion. Since the bounding box read is a relative coordinate, it does not need to be modified.

[0091] Step S3-3, convert the bounding box from relative coordinates to absolute coordinates (multiplied by the scaled width and height).

[0092] Step S3-4, screening bounding boxes, retaining bounding boxes whose width and height are both greater than or equal to two pixels (width pixels and height pixels).

[0093] Step S3-5, using K-means clustering algorithm to obtain k anchors.

[0094] Step S3-6, use genetic algorithm to randomly mutate the width and height of the anchor point. If the effect becomes better after mutation, assign the result after mutation to the anchor point; if the effect becomes worse after mutation, skip it. By default, 1000 mutations are performed.

[0095] Step S3-7, sort the anchor points finally obtained by mutation by area and return them.

[0096] Step S4, according to the optimized anchor point, put it into the YOLOv5 network model for training. This network structure mainly uses MobileNetV3 Large as the backbone network, the neck is optimized to BiFPN, and the ECA attention mechanism is added before the detection head to enhance the focus on the target features. The network structure diagram is shown in Figure 7 .

[0097] Step S5, dividing the defect data set into a training set and a validation set, inputting the training set into the surface defect detection model for training, and obtaining a trained surface defect detection model.

[0098] The specific training steps are as follows:

[0099] Step S5-1, construct a surface defect detection model based on the MBE-YOLO algorithm. The preprocessed feature map is input into the backbone layer (Backbone) to extract category features. The backbone layer includes a CBS module, a MobileNetV3 network module, and a SPPF module.

[0100] The CBS module contains a 1×1 convolution kernel, a batch normalization (BN) layer, and an activation function SILU layer. The input image is passed through a convolution layer to adjust the dimension of the feature map, and then the feature map is divided into different levels and processed separately. In each level, the feature map is extracted through a combination of multiple convolution layers, batch normalization layers, and activation function layers.

[0101] The backbone network is mainly responsible for extracting the basic features of the image and generating feature maps of different scales through a series of convolution operations. The MobileNetV3 module replaces the backbone network CSP module. First, the input of the module is passed through a 1×1 convolution kernel to adjust the dimension of the feature map. Then, the feature map passes through a series of inverted residual structures, where the inverted residual structure includes the following steps:

[0102] First, a 1×1 convolution operation is performed on the input feature map, and the number of channels of the feature map is adjusted through Batch Normalization (BN) and ReLU activation functions.

[0103] Then, the adjusted feature map is subjected to a depthwise convolution operation. The depthwise convolution operation is divided into two steps: first, a 3×3 convolution operation is performed on each channel, and then a 1×1 pointwise convolution is performed on the convolved feature map to achieve cross-channel information fusion. This operation is also processed through Batch Normalization and ReLU activation function.

[0104] Next, the output of the depthwise separable convolution is added to the original input feature map at the element level. This part is the residual connection.

[0105] The last part of the backbone layer processes the feature map through the SPPF (Spatial Pyramid Pooling-Fast) module, which is a spatial pyramid pooling layer. It is downsampled through three different maximum pooling layers and then spliced ​​and fused.

[0106] Step S5-2: The feature map processed by the backbone network is input to the neck. Three scale feature maps of 20×20, 40×40 and 80×80 are obtained according to the output of the backbone layer, and are put into the neck network. After convolution and upsampling operations, each layer is sent to the head for prediction. The BiFPN (Bi-directional Feature Pyramid Network) structure is used to fuse information on feature maps of different scales.

[0107] BiFPN is an improved feature pyramid network designed to more efficiently fuse features from different scales to improve the performance of object detection. BiFPN achieves higher detection accuracy through a bidirectional feature fusion mechanism and a weighted feature fusion strategy. On the basis of the traditional feature pyramid network (FPN), an upsampling path from high-level features to low-level features is added, while the downsampling path from low-level features to high-level features is retained. This forms a bidirectional feature fusion, allowing information to flow and share more fully between different scales. In BiFPN, in order to better fuse features from different layers, a weighted feature fusion mechanism is introduced. Specifically, the feature fusion at each scale is weighted, that is, the contribution of different feature maps to the final fusion result is determined by learning weights. In this way, the way of feature fusion can be dynamically adjusted to adapt to different detection tasks and scenarios.

[0108] The structure of BiFPN usually includes several units, each of which performs a bidirectional feature fusion. Figure 6 , the specific structure and calculation formula are as follows:

[0109] a) Initialization of feature map:

[0110] The three scale feature maps output from the backbone network are P3, P4 and P5, corresponding to 80×80, 40×40 and 20×20 respectively.

[0111] b) Top-Down Pathway Fusion:

[0112] The feature map of the previous level is upsampled, weighted and fused with the current feature map before convolution.

[0113]

[0114] c) Bottom-Up Pathway Fusion:

[0115] The next level feature map is downsampled, weighted fused with the current feature map and the feature map after fusion of the previous level, and then convolved.

[0116]

[0117] Among them, ω i is a learnable weight, ∈ is a small value (such as 10 -6 ) is used to prevent division by zero errors. Conv represents the convolution operation, UpSample and DownSample represent upsampling and downsampling operations, respectively.

[0118] d) Weighted Feature Fusion:

[0119] At each scale, feature maps are fused by weighted summation:

[0120] P′4=w1·P4+w2·UpSample(P5)

[0121] P3=w3·P3+w4·U P Sample(P′4)

[0122] P′A=w5·P′4+w6·DownSample(P3)

[0123]

[0124] Among them, ω1, ω2, ...ω8 are learned weighted parameters, which are dynamically adjusted through the training process.

[0125] Specifically, the 4th layer MobileNetV3_1 and the 6th layer MobileNetV3_2 of the backbone network are spliced ​​with the 12th layer BiFPN_Concat2_1 and the 16th layer BiFPN_Concat2_2 of the neck to obtain new feature images, namely the first feature map and the second feature map. The 6th layer MobileNetV3_2 is then spliced ​​with the 14th layer to generate the third feature map, which is output to the detection head. Multi-level feature fusion integrates the overall information and improves the expression ability of the model.

[0126] In step S5-3, after the neck is processed, feature maps of different sizes are input to the detection head. Before the detection head, the ECA attention mechanism is introduced to increase the representation ability of features and global context information. The convolution kernel size of the one-dimensional convolution is self-adapted through a function, so that layers with a larger number of channels can have more cross-channel interactions to improve the representation ability of features and the accuracy of target detection.

[0127] The calculation formula for the adaptive convolution kernel size is:

[0128]

[0129] Here, k represents the kernel size of the one-dimensional convolution, C is the number of channels of the input feature, and γ and b are hyperparameters. By taking the absolute value and rounding it down to the nearest odd number, we ensure that the kernel size is always an odd number. Once k is determined, the ECA module performs a one-dimensional convolution on the input features, aiming to capture the importance of each channel relative to other channels. The formula is as follows:

[0130] [out = Conv1D k (in)]

[0131] This formula represents the conversion of input feature in into output feature out through a one-dimensional convolution operation (kernel size is k). Here, ConvlDk refers to a one-dimensional convolution operation with a kernel size of k.

[0132] The detection head of YOLOv5 adopts the Anchor-based mechanism, that is, a series of predefined anchor boxes are set on the feature map, and these anchor boxes are used to predict the category and border position of the target. Specifically, the detection head performs classification prediction and border regression for each anchor box, and outputs the category, border position and confidence score of the target, which is used to predict the defect type of 4 categories. Finally, the redundant detection boxes are removed by non-maximum suppression (NMS) to retain the final detection results and reduce false detection and missed detection. In target detection, the loss function is a key tool for evaluating model performance. It guides the update of model parameters by calculating the difference between the predicted results and the true labels to gradually improve the detection accuracy of the model. Common target detection loss functions include classification loss and regression loss, where classification loss is used to evaluate the model's ability to predict the target category, while regression loss is used to evaluate the model's prediction accuracy for the target bounding box position.

[0133] IoU (Intersection over Union) is a commonly used evaluation indicator and regression loss function in target detection. The calculation formula of IoU is:

[0134]

[0135] Where A and B represent the predicted box and the true box, respectively. The IoU value is between 0 and 1. The larger the value, the more overlap the predicted box and the true box, and the better the model performance. The role of IoU is to directly measure the degree of overlap between the predicted bounding box and the true bounding box, which is intuitive and easy to understand. However, IoU also has some disadvantages. First, when the predicted box and the true box are completely disjoint, the IoU value is 0, but it cannot reflect the distance between the predicted box and the true box. Secondly, IoU has discontinuity in the optimization process, that is, when the predicted box and the true box do not overlap, the gradient of the loss function is zero, which makes it difficult for the model to update the parameters.

[0136] In order to overcome the shortcomings of IoU, GIoU (Generalized Intersection over Union) was introduced. GIoU not only considers the overlapping area of ​​the predicted box and the true box, but also introduces the concept of the minimum enclosing rectangle to reflect the relative position relationship between the predicted box and the true box. GIoU adjusts IoU by subtracting the proportion of the uncovered area, so that the loss function can still provide effective gradient information when the predicted box and the true box do not intersect, avoiding the discontinuity problem of IoU. In addition, GIoU can also push the predicted box to approach the true box faster during the optimization process, improving the convergence speed and detection accuracy of the model. The final detection head output size is 256×80×80, 512×40×40, and 1024×20×20.

[0137] Step S6, inputting the aircraft engine surface image to be inspected into the trained MBE-YOLOv5 surface defect detection model, and outputting the aircraft engine surface defect detection result.

[0138] Example 2

[0139] This embodiment also relates to a verification test of an aircraft engine surface defect detection method.

[0140] This method is tested on a dataset of surface defect images of aircraft engine turbine blades, and repeated parameter adjustment training is performed. A total of 330 defective images are included, with a total of 4 types of defects. The defect categories include scratches, dots, creases, and damage. These images are divided into a training set and a validation set in a ratio of 8:2 for input training.

[0141] Mosaic data enhancement is also enabled during the image input process of the model. Four images are spliced ​​together by random cropping and scaling. The number of images actually involved in the training is larger and the samples are richer.

[0142] In order to evaluate the quality of the model, the present invention uses a variety of evaluation indicators for comparison, among which the mean average precision (mAP) and the number of frames per second (FPS) are the most commonly used evaluation indicators in the field of target detection. The mAP is calculated by the precision P (Precision) and the recall R (Recall):

[0143]

[0144] Among them, TP, FP, and FN are the number of actual positive samples predicted as positive samples, the number of actual negative samples predicted as positive samples, and the number of actual positive samples predicted as negative samples, respectively, and c is the number of defect types.

[0145] The operating environment of the algorithm of this method is Intel(R) Xeon(R) CPU@2.00GHz, GPU is NVIDIA TeslaP100 PCIe 16GB, Batchsize is 64, epochs training round number is 300, optimizer uses Adam, learning rate is 0.01. The training parameter results of the YOLOv5 model and the method of the present invention on the surface of the engine turbine blade are shown in Table 1, unit: %. By comparing the result data, it is found that compared with the original YOLOv5 model, the optimization model proposed by this method has a relatively large improvement in the detection accuracy of most defect types, and has greatly improved scratches and creases. The accuracy of each category reaches more than 90%, and the overall accuracy, recall rate and average accuracy all reach more than 90% and have been improved. At the same time, GFLOPs (floating point operations per second) has been reduced from the original 15.8 to 14.9, and the number of parameters (Params) has been reduced from the original 7.02*10 6 Reduced to 6.57*10 6 , the reduction of GFLOPs means that the model designed by the present invention requires less computing resources and runs faster; the reduction in the number of parameters means that the complexity of the model is reduced and the storage space occupied is also reduced. Compared with the original YOLOv5, the model FPS (representing the time required to process each image) of this method has increased from 91.74 to 94.34, and the real-time performance is enhanced. It is faster when processing images and can process more frames in a shorter time. This is particularly important for real-time applications of visual inspection in the aerospace field.

[0146] The optimization results improve operational efficiency and reduce resource consumption while maintaining model performance. That is, this method has a high application value in aircraft engine surface defect detection.

[0147] Table 1

[0148]

[0149] Example 3

[0150] The electronic device of the present invention includes a central processing unit (CPU), which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) or loaded from a storage unit into a random access memory (RAM). In the RAM, various programs and data required for device operation can also be stored. The CPU, ROM and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.

[0151] Multiple components in the device are connected to the I / O interface, including: input units, such as keyboards, mice, etc.; output units, such as various types of displays, speakers, etc.; storage units, such as disks, optical disks, etc.; and communication units, such as network cards, modems, wireless communication transceivers, etc. The communication unit allows the device to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunication networks.

[0152] The processing unit performs the various methods and processes described above, such as methods S1 to S6. For example, in some embodiments, methods S1 to S6 may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit. In some embodiments, part or all of the computer program may be loaded and / or installed on the device via a ROM and / or a communication unit. When the computer program is loaded into the RAM and executed by the CPU, one or more steps of methods S1 to S6 described above may be performed. Alternatively, in other embodiments, the CPU may be configured to execute methods S1 to S6 in any other appropriate manner (e.g., by means of firmware).

[0153] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0154] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0155] In the context of the present invention, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0156] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for detecting surface defects of an aircraft engine, characterized in that: The method comprises the following steps: Step S1, acquiring an aircraft engine surface image; Step S2, putting the labeled surface defect data set into the MBE-YOLOv5 surface defect detection model for training; Step S3, inputting the surface image of the aircraft engine to be detected into the trained MBE-YOLOv5 surface defect detection model, and outputting the aircraft engine surface defect detection result; The MBE-YOLOv5 surface defect detection model is an improvement on the YOLOv5 model. The MBE-YOLOv5 surface defect detection model uses MobileNetV3 Large as the backbone network, optimizes the neck to BiFPN, and adds an ECA attention mechanism before the detection head.

2. The method for detecting surface defects of an aircraft engine according to claim 1, characterized in that: The backbone network of the MBE-YOLOv5 surface defect detection model includes a CBS module, a MobileNetV3 network module and an SPPF module; The backbone network first performs a 1×1 convolution operation on the input feature map to adjust the dimension of the feature map, then the feature map passes through a series of inverted residual structures, and finally processes the feature map through the SPPF module to output feature maps of different scales to the neck.

3. The method for detecting surface defects of an aircraft engine according to claim 2, characterized in that: The 4th and 6th MobileNetV3 layers of the backbone network are spliced ​​with the 12th and 16th BiFPN layers of the neck to obtain the first feature map and the second feature map, respectively. The 6th MobileNetV3 layer is then spliced ​​with the 14th layer to generate a third feature map, which is output to the detection head.

4. The method for detecting surface defects of an aircraft engine according to claim 1, characterized in that: The neck of the MBE-YOLOv5 surface defect detection model uses a BiFPN structure to perform bidirectional information fusion on feature maps of different scales, and a weighted feature fusion mechanism is used in the feature fusion process of each scale.

5. The method for detecting surface defects of an aircraft engine according to claim 3, characterized in that: The method comprises: before training the MBE-YOLOv5 surface defect detection model, using a K-means clustering algorithm, obtaining optimized anchor points adapted to the surface defect data set according to a genetic algorithm, and putting the optimized anchor points into the detection model for training.

6. The method for detecting surface defects of an aircraft engine according to claim 5, characterized in that: The process of obtaining an optimized anchor point adapted to the surface defect data set according to a genetic algorithm comprises: Step S3-1, read the width and height of each image in the training set, as well as the width and height of all bounding boxes; Step S3-2, scaling the maximum width and height of each image to a specified size in proportion; Step S3-3, converting the bounding box from relative coordinates to absolute coordinates; Step S3-4, filtering bounding boxes, and retaining bounding boxes whose width and height are both greater than or equal to two pixels; Step S3-5, using K-means clustering algorithm to obtain k anchor points; Step S3-6, using a genetic algorithm to randomly mutate the width and height of the anchor point, if the effect becomes better after the mutation, assign the result after the mutation to the anchor point; if the effect becomes worse after the mutation, skip it; Step S3-7, sort the anchor points finally obtained by mutation by area and return them.

7. The method for detecting surface defects of an aircraft engine according to claim 1, characterized in that: The method comprises: preprocessing the aircraft engine surface image to obtain a surface defect image, wherein the preprocessing comprises data enhancement by inversion, translation, brightness change, and noise enhancement to increase the number of data samples.

8. The method for detecting surface defects of an aircraft engine according to claim 7, characterized in that: The method further comprises: annotating the defects in the surface defect image to obtain an annotation file, wherein the surface defect image and the corresponding annotation file constitute a surface defect data set.

9. The method for detecting surface defects of an aircraft engine according to claim 7, characterized in that: Annotating the defects in the surface defect image to form an annotation file for each image, wherein the data format in the annotation file is set according to the YOLO format; The annotation file includes defect category information and defect location information.

10. An electronic device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

Images