Method and electronic equipment for detecting surface defects of aircraft engines

Abstract

The present invention relates to a method and electronic device for detecting surface defects in aircraft engines. The method comprises the following steps: acquiring an aircraft engine surface image; training a labeled surface defect dataset using an MBE-YOLOv5 surface defect detection model; inputting the aircraft engine surface image to be inspected into the trained MBE-YOLOv5 surface defect detection model, and outputting an aircraft engine surface defect detection result. The MBE-YOLOv5 surface defect detection model is an improvement on the YOLOv5 model, using MobileNetV3Large as the backbone network, optimizing the neck to BiFPN, and incorporating an ECA attention mechanism before the detection head. Compared with existing technologies, the present invention has the advantages of high detection accuracy, high detection efficiency, ease of deployment on resource-constrained devices and platforms, and wide applicability.

Description

Technical Field

[0001] The present invention relates to the field of aircraft engine surface defect detection, and in particular to an aircraft engine surface defect detection method and electronic equipment. Background Art

[0002] With the rapid development of the aviation industry, aircraft surface defect detection has become a crucial component in ensuring flight safety. Traditional aircraft engine surface inspection methods primarily rely on manual inspection and traditional non-destructive testing techniques, such as borescope testing, eddy current testing, and ultrasonic testing. While these methods can detect defects to a certain extent, they suffer from low efficiency, high costs, and susceptibility to human error.

[0003] In recent years, deep learning-based object detection technology has developed rapidly and has been widely used in the field of defect detection. Currently, the existing technical solutions for aircraft engine surface defect detection mainly include the following:

[0004] Traditional image processing-based detection methods: These methods typically achieve defect detection through steps such as image preprocessing, feature extraction, and classifiers. While these methods have achieved some success in certain applications, they suffer from poor accuracy and robustness when faced with complex backgrounds and diverse defects, and require significant computational resources.

[0005] Detection methods based on deep learning: such as Faster R-CNN, SSD and YOLO series algorithms. These methods automatically extract image features through convolutional neural networks and perform target detection. As an efficient target detection algorithm, YOLOv5 has attracted widespread attention due to its fast detection speed and high accuracy. The original YOLOv5 results are as follows: Figure 1 , it still has certain limitations when it comes to defect detection in small defect targets and complex backgrounds.

[0006] After searching, Chinese invention patent publication number CN111833328B discloses a deep learning-based surface defect detection method for aircraft engine blades. The method includes: acquiring images of aircraft engine blades, manually annotating images containing surface defects, and constructing a surface defect image dataset; preprocessing the surface defect image dataset; constructing a deep convolutional defect detection network for detecting defect labels and defect categories, and training the deep convolutional defect detection network using the preprocessed surface defect image dataset. The deep convolutional defect detection network uses a parallel ResNet network and a DenseNet network to simultaneously extract features from the input blade image, superimposes the two extracted features at corresponding positions for feature fusion, and then combines the fused features for defect classification; the image to be inspected is imported into the trained deep convolutional model, and the defect label and defect category are output. This existing patent suffers from low detection accuracy and efficiency.

[0007] How to achieve accurate and efficient detection of aircraft engine surface defects has become a technical problem that needs to be solved. Summary of the Invention

[0008] The purpose of the present invention is to provide an aircraft engine surface defect detection method and electronic equipment in order to overcome the defects of the above-mentioned prior art.

[0009] The purpose of the present invention can be achieved by the following technical solutions:

[0010] According to one aspect of the present invention, a method for detecting surface defects of an aircraft engine is provided, the method comprising the following steps:

[0011] Step S1, acquiring an aircraft engine surface image;

[0012] Step S2: Put the labeled surface defect dataset into the MBE-YOLOv5 surface defect detection model for training.

[0013] Step S3: inputting the surface image of the aircraft engine to be inspected into the trained MBE-YOLOv5 surface defect detection model, and outputting the aircraft engine surface defect detection result;

[0014] The MBE-YOLOv5 surface defect detection model is an improvement of the YOLOv5 model. The MBE-YOLOv5 surface defect detection model uses MobileNetV3 Large as the backbone network, optimizes the neck to BiFPN, and adds an ECA attention mechanism before the detection head.

[0015] Preferably, the backbone network of the MBE-YOLOv5 surface defect detection model includes a CBS module, a MobileNetV3 network module and an SPPF module;

[0016] The backbone network first performs a 1×1 convolution operation on the input feature map to adjust the dimension of the feature map. The feature map then passes through a series of inverted residual structures and is finally processed by the SPPF module to output feature maps of different scales to the neck.

[0017] More preferably, the 4th layer MobileNetV3 and the 6th layer MobileNetV3 of the backbone network are spliced ​​with the 12th layer BiFPN and the 16th layer BiFPN of the neck respectively to obtain the first feature map and the second feature map respectively, and the 6th layer MobileNetV3 is then spliced ​​with the 14th layer to generate the third feature map, which is output to the detection head.

[0018] Preferably, the neck of the MBE-YOLOv5 surface defect detection model uses a BiFPN structure to perform bidirectional information fusion on feature maps of different scales, and a weighted feature fusion mechanism is used in the feature fusion process of each scale.

[0019] More preferably, the method includes: before training the MBE-YOLOv5 surface defect detection model, using the K-means clustering algorithm, obtaining optimized anchor points adapted to the surface defect data set according to the genetic algorithm, and putting them into the detection model for training.

[0020] More preferably, the process of obtaining an optimized anchor point adapted to the surface defect dataset according to a genetic algorithm includes:

[0021] Step S3-1, read the width and height of each image in the training set, as well as the width and height of all bounding boxes;

[0022] Step S3-2, scaling the maximum width and height of each image to a specified size;

[0023] Step S3-3, converting the bounding box from relative coordinates to absolute coordinates;

[0024] Step S3-4, filtering bounding boxes and retaining bounding boxes whose width and height are both greater than or equal to two pixels;

[0025] Step S3-5, using K-means clustering algorithm to obtain k anchor points;

[0026] Step S3-6: Use a genetic algorithm to randomly mutate the width and height of the anchor point. If the effect after mutation is better, assign the mutated result to the anchor point; if the effect after mutation is worse, skip it;

[0027] In step S3-7, the anchor points finally obtained by mutation are sorted by area and returned.

[0028] Preferably, the method includes: preprocessing the aircraft engine surface image to obtain a surface defect image, wherein the preprocessing includes data enhancement by inversion, translation, brightness change, and noise enhancement to increase the number of data samples.

[0029] More preferably, the method further comprises: annotating defects in the surface defect image to obtain an annotation file, and the surface defect image and the corresponding annotation file constitute a surface defect dataset.

[0030] More preferably, the defects in the surface defect image are annotated to form an annotation file for each image, and the data format in the annotation file is set according to the YOLO format;

[0031] The annotation file includes defect category information and defect location information.

[0032] According to another aspect of the present invention, an electronic device is provided, comprising a memory and a processor, wherein a computer program is stored in the memory, and the method described above is implemented when the processor executes the program.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] 1) The present invention optimizes the network structure of the original YOLOv5. In response to the complex and diverse backgrounds and lightweight processing requirements of aircraft engine surface images, the original backbone network is replaced with MobileNetV3Large. This aims to utilize the high efficiency and lightweight characteristics of MobileNetV3, which is suitable for deployment on mobile terminals to reduce computing overhead and speed up detection while maintaining good performance. In response to the diversity and subtlety of aircraft engine surface defects, BiFPN is added to the neck, which can more effectively fuse multi-scale features and improve the model's detection accuracy for targets of different sizes, thereby improving the overall detection performance. In response to environmental interference on the surface of aircraft engines, the ECA attention mechanism is added to the detection head to enhance attention to specific features and improve the overall accuracy of the model. Therefore, the present invention can provide more accurate detection for targets of different sizes and improve the model's detection accuracy and efficiency for small and large targets.

[0035] 2) The present invention optimizes the network structure of YOLOv5, and BiFPN adopts a bidirectional feature fusion mechanism and a weighted feature fusion strategy. The weighted fusion method adopts fast normalization fusion, which makes the network training speed very fast. The design principle of MobileNetV3 is combined with deep separable convolution and hardware-friendly optimization, which significantly reduces the computational overhead while maintaining high performance and improves detection efficiency.

[0036] 3) The ECA attention mechanism of the present invention can enhance the model's attention to key feature areas, involve fewer parameters and calculations, and improve the detection accuracy and efficiency of minor defects, especially in the presence of environmental interference.

[0037] 4) The lightweight detection network of the present invention is easier to deploy on resource-constrained devices and platforms, making the model have higher performance and wider applicability in practical applications.

[0038] 5) By replacing the initial target box loss function IoU with GIoU, the model can better adjust the predicted box for the dataset, making it closer to the real box, improving the convergence speed and detection accuracy of the model; through NMS non-maximum suppression, redundant detection boxes are removed, the final detection results are retained, and false detections and missed detections are reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is a schematic diagram of the structure of the YOLOv5 network;

[0040] Figure 2 This is a schematic diagram of the structure of MobileNetV3 Large in the present invention;

[0041] Figure 3 Schematic diagram of the channel separation convolution process in the present invention;

[0042] Figure 4 This is a schematic diagram of the structure of the SE attention mechanism of MobileNetV3 Large in the present invention;

[0043] Figure 5 Schematic diagram of the ECA attention mechanism structure of the detection head in the present invention;

[0044] Figure 6 Schematic diagram of the BiFPN structure of the neck in the present invention;

[0045] Figure 7 Schematic diagram of the structure of the MBE-YOLOv5 surface defect detection model in the present invention. DETAILED DESCRIPTION

[0046] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0047] Aircraft engine surface images have their own characteristics, including high reflectivity materials, complex geometries, diverse defect types, high resolution requirements, and environmental interference.

[0048] a) Highly reflective materials: Aircraft engine surfaces are typically made of metal materials, such as titanium and aluminum alloys, which have high reflectivity. Under varying lighting conditions, these surfaces can easily produce strong reflections or shadows, affecting image quality and the algorithm's detection accuracy.

[0049] b) Complex geometry: Aircraft engine surfaces have complex geometries, including curved and uneven surfaces. This can cause defects to appear differently from different viewing angles. This complex geometry increases the difficulty of defect identification.

[0050] c) Diverse Defect Types: Aircraft engine surfaces may contain a variety of defects, such as scratches, corrosion, cracks, and dents. These defects vary in shape, size, and color, requiring detection algorithms to be high-resolution and flexible.

[0051] d) High Resolution Requirements: To detect tiny defects, aircraft engine surface images typically require high resolution. This means that the inspection algorithm needs to be able to process large images while ensuring computational efficiency while maintaining inspection accuracy.

[0052] e) Environmental interference: During the actual inspection process, there may be environmental interference such as oil, dust, etc. on the surface of the aircraft engine. These factors will have a negative impact on image quality and increase the difficulty of defect detection.

[0053] This embodiment relates to a method for detecting surface defects in aircraft engines. The method is based on MBE-YOLOv5 (YOLOv5 improved based on MobileNetV3-BiFPN-ECA). This method uses an improved YOLOv5 network model for defect detection based on the characteristics of aircraft engine surface images. The method improves the YOLOv5 network structure, including the following steps:

[0054] 1) The backbone network is replaced with MobileNetV3 Large, such as Figure 2 .

[0055] Due to the complex and diverse backgrounds and lightweight processing requirements, the YOLOv5 backbone network is optimized and replaced with modules suitable for deployment on mobile terminals, making the model lightweight and enabling the model to run efficiently in embedded hardware or real-time detection systems.

[0056] MobileNetV3 Large is used as the backbone network. Its high efficiency and lightweight features reduce the number of model parameters. It is targeted at defect detection on the surface of aircraft engines and facilitates mobile deployment. The use of a new backbone network improves computing efficiency and accelerates inference speed.

[0057] The main features of MobileNetV3 include: channel-separable convolution, SE module, h-swish and h-sigmoid and bneck.

[0058] (1) Channel separation convolution is the main feature of the MobileNet series and the main factor that makes it play a lightweight role. Figure 3 , the channel separable convolution is divided into two processes:

[0059] ① Channel-wise separable convolution;

[0060] ②Normal 1X1 convolution outputs the specified number of channels (such as Figure 3 ).

[0061] (2) SE module (such as Figure 4 )

[0062] The SE attention mechanism (Squeeze-and-Excitation Networks) is a module used to enhance the representational power of convolutional neural networks. It improves the network's representational power by explicitly modeling the interdependencies between convolutional feature channels. The SE module primarily consists of three steps: Squeeze, Excitation, and Scale.

[0063] (3) h-swish and h-sigmoid use approximate operations to simulate swish and relu. The ReLU6 function is chosen because its optimized implementation can be used on almost all software and hardware frameworks; at the same time, it can eliminate the potential loss of numerical accuracy caused by different implementations of the approximate sigmoid in specific modes. The formula is as follows

[0064]

[0065] Among them, h-swish is a variant of Swish, x is the input value of the activation function, and ReLU6 is a variant of the ReLU (Rectified Linear Unit) function.

[0066] (4) Bneck is the core module of MobileNetV3 and the basic module of the network. It mainly implements channel-separable convolution + SE channel attention mechanism + residual connection.

[0067] The design principle of MobileNetV3 combines depth-wise separable convolution and hardware-friendly optimization, which significantly reduces computational overhead while maintaining high performance.

[0068] 2) Add ECA attention mechanism, the structure is as follows Figure 5 .

[0069] The Efficient Channel Attention (ECA) mechanism is a mechanism designed to improve the performance of deep convolutional neural networks (CNNs) by focusing on the most important features in the data. Its main features are as follows:

[0070] Avoid dimensionality reduction: Unlike traditional attention mechanisms, ECA maintains the original dimensions of the data, which helps retain important information.

[0071] Local cross-channel interaction: ECA uses 1D convolution to interact between different channels without reducing their dimensionality. This approach ensures that the model is efficient while still capturing important features.

[0072] Adaptive convolution kernel size: The convolution kernel size of 1D convolution is adaptively selected according to the channel dimension, allowing effective local interaction.

[0073] Efficient performance: Compared with other attention mechanisms, ECA involves fewer parameters and computations, making it suitable for real-time applications.

[0074] The ECA attention mechanism can enhance the model's attention to key feature areas and improve the ability to detect minor defects, especially in the presence of environmental interference.

[0075] 3) Add BiFPN to the neck:

[0076] Considering the diversity and subtlety of aircraft engine surface defects, the feature extraction module of YOLOv5 was improved, and Bi-directional Feature Pyramid Network (BiFPN) was added to the neck structure of YOLOv5 to better capture defect features of different scales.

[0077] BiFPN enhances the model's detection capabilities for objects of varying sizes by bidirectionally fusing multi-scale features. Its bidirectional feature fusion method improves the representational power of the feature pyramid, significantly enhancing detection accuracy. BiFPN is an improved FPN network structure primarily used for object detection tasks. This weighted and bidirectionally connected structure, combining top-down and bottom-up connections, achieves cross-scale connectivity by constructing bidirectional channels. Features from the feature extraction network are directly fused with relative size features from the bottom-up path, preserving shallower semantic information without losing too much deeper semantic information.

[0078] Traditional feature fusion combines feature maps of different scales with the same weight. However, when the input feature maps have different resolutions, this same weighting produces unequal results for the output feature maps. Therefore, BiFPN sets different weights based on the importance of different input features and repeatedly employs this structure to enhance feature fusion.

[0079] The weighted fusion method in the BiFPN structure uses fast normalized fusion. This fusion method is proposed to address the slow training speed. The weights are scaled to the range of 0 to 1. Since the Softmax method is not used, the training speed is very fast. Cross-scale connection is achieved by adding a skip connection and a bidirectional path, thus achieving weighted fusion and bidirectional cross-scale connection. The BiFPN structure is shown in the attached figure. Figure 6 shown.

[0080] 4) The target box function is replaced by GIoU:

[0081] The GIoU loss function is an improvement on the IoU loss function in the original YOLOv5. IoU only focuses on the overlapping area between the predicted box and the true box, while GIoU not only focuses on the overlapping area, but also considers the distance between the two boxes and the non-overlapping area. This makes GIoU more effective when dealing with non-overlapping or partially overlapping boxes. The calculation formula is as follows:

[0082]

[0083] Among them, A and B are the predicted box and the true box respectively, and C is the minimum closure area containing A and B. By maximizing GIoU, the model is able to better adjust the predicted box to make it closer to the true box.

[0084] The improved MBE-YOLOv5 overall model structure is as follows Figure 7 shown.

[0085] The method comprises the following steps:

[0086] In step S1, an image of the aircraft engine surface is acquired and preprocessed to produce a surface defect image. Data enhancement is performed through operations such as inversion, translation, brightness change, and noise enhancement to increase the number of data samples. This data enhancement makes the model more robust and better adaptable to various lighting conditions and viewing angles in practical applications. Image enhancement methods such as illumination variation, reflection simulation, and mirror rotation are designed to address issues with high reflectivity and complex geometric shapes. During the image preprocessing stage, adaptive contrast enhancement and noise suppression algorithms are added to improve image detectability.

[0087] Step S2: Label the defects in the image to obtain a labeling file. The surface defect image and the corresponding labeling file are used as the surface defect dataset. The defects in the image are labeled using LabelImg to form a labeling file (txt) for each image. The data format in the labeling file (txt) is set according to the YOLO format: category, marker box center coordinates (x, y), marker box width w, and height h, that is, the labeling file includes defect category information and defect location information. The surface defect image and the corresponding labeling file constitute the surface defect dataset.

[0088] Step S3: Optimize the anchor points based on the surface defect dataset to obtain optimized anchor points that are suitable for the surface defect dataset: Use the K-means clustering algorithm and the genetic algorithm to obtain optimized anchor points that are suitable for the surface defect dataset. The main steps are as follows:

[0089] Step S3-1, read the width and height of each image in the training set, as well as the width and height of all bounding boxes (BoundingBox).

[0090] In step S3-2, the maximum width and height of each image are proportionally scaled to the specified size (img_size). Since the bounding box read is in relative coordinates, no modification is required.

[0091] Step S3-3, convert the bounding box from relative coordinates to absolute coordinates (multiplied by the scaled width and height).

[0092] Step S3-4, screening bounding boxes, retaining bounding boxes whose width and height are both greater than or equal to two pixels (width pixels and height pixels).

[0093] Step S3-5: Use K-means clustering algorithm to obtain k anchors.

[0094] Step S3-6: Use a genetic algorithm to randomly mutate the width and height of the anchor point. If the mutation improves the effect, the result is assigned to the anchor point. If the mutation deteriorates the effect, the algorithm is skipped. By default, 1000 mutations are performed.

[0095] In step S3-7, the anchor points finally obtained by mutation are sorted by area and returned.

[0096] Step S4: According to the optimized anchor points, the YOLOv5 network model is put into training. The network structure mainly uses MobileNetV3 Large as the backbone network, the neck is optimized to BiFPN, and the ECA attention mechanism is added before the detection head to enhance the focus on the target features. The network structure diagram is shown in the figure. Figure 7 .

[0097] In step S5, the defect data set is divided into a training set and a validation set, and the training set is input into the surface defect detection model for training to obtain a trained surface defect detection model.

[0098] The specific training steps are as follows:

[0099] Step S5-1: Build a surface defect detection model based on the MBE-YOLO algorithm. Input the preprocessed feature map into the backbone layer (Backbone) to extract category features. The backbone layer includes the CBS module, MobileNetV3 network module, and SPPF module.

[0100] The CBS module consists of a 1×1 convolution kernel, a batch normalization (BN) layer, and a SILU activation layer. The input image is passed through a convolutional layer to adjust the dimension of the feature map. The feature map is then divided into different layers for processing. Within each layer, the feature map is extracted through a combination of multiple convolutional layers, batch normalization layers, and activation layers.

[0101] The backbone network is primarily responsible for extracting basic image features and generating feature maps of varying scales through a series of convolution operations. The MobileNetV3 module replaces the CSP module in the backbone network. First, the module input passes through a 1×1 convolution kernel to adjust the dimension of the feature map. The feature map then passes through a series of inverted residual structures, which include the following steps:

[0102] First, a 1×1 convolution operation is performed on the input feature map, and the number of channels of the feature map is adjusted through Batch Normalization (BN) and ReLU activation function.

[0103] Next, a depthwise convolution is performed on the resized feature map. This process consists of two steps: first, a 3×3 convolution is performed on each channel, followed by a 1×1 pointwise convolution on the convolved feature map to achieve cross-channel information fusion. This operation is also processed using batch normalization and the ReLU activation function.

[0104] Next, the output of the depthwise separable convolution is added to the original input feature map at the element level. This part is the residual connection.

[0105] The last part of the backbone layer processes the feature map through the SPPF (Spatial Pyramid Pooling-Fast) module, which is a spatial pyramid pooling layer. It is downsampled through three different maximum pooling layers and then spliced ​​and fused.

[0106] In step S5-2, the feature maps processed by the backbone network are input to the neck. Based on the backbone layer output, three feature maps of scales (20×20, 40×40, and 80×80) are generated and fed into the neck network. After convolution and upsampling at each layer, they are sent to the head for prediction. A Bi-directional Feature Pyramid Network (BiFPN) architecture is used to fuse information from feature maps of different scales.

[0107] BiFPN is an improved feature pyramid network designed to more efficiently fuse features from different scales to improve object detection performance. BiFPN achieves higher detection accuracy through a bidirectional feature fusion mechanism and a weighted feature fusion strategy. Building on the traditional feature pyramid network (FPN), it adds an upsampling path from high-level features to low-level features while retaining the downsampling path from low-level features to high-level features. This results in bidirectional feature fusion, allowing information to flow and share more fully across scales. To better fuse features from different layers, BiFPN introduces a weighted feature fusion mechanism. Specifically, feature fusion at each scale is weighted, meaning that learned weights determine the contribution of different feature maps to the final fusion result. This allows for dynamic adjustment of feature fusion methods to suit different detection tasks and scenarios.

[0108] The structure of BiFPN usually includes several units, each of which performs a bidirectional feature fusion. Figure 6 , the specific structure and calculation formula are as follows:

[0109] a) Initialization of feature map:

[0110] The three scale feature maps output from the backbone network are P3, P4 and P5, corresponding to 80×80, 40×40 and 20×20 respectively.

[0111] b) Top-Down Pathway Fusion:

[0112] The feature map of the previous level is upsampled, weighted and fused with the current feature map before convolution.

[0113]

[0114] c) Bottom-Up Pathway Fusion:

[0115] The next level feature map is downsampled, weighted fused with the current feature map and the feature map after fusion of the previous level, and then convolved.

[0116]

[0117] Among them, ω i is a learnable weight, ∈ is a small value (such as 10 -6 ) is used to prevent division by zero errors. Conv represents the convolution operation, UpSample and DownSample represent the upsampling and downsampling operations, respectively.

[0118] d) Weighted Feature Fusion:

[0119] At each scale, feature maps are fused by weighted summation:

[0120] P′4=w1·P4+w2·UpSample(P5)

[0121] P3=w3·P3+w4·U P Sample(P′4)

[0122] P′A=w5·P′4+w6·DownSample(P3)

[0123]

[0124] Where ω1, ω2, ...ω8 are learning weighted parameters, which are dynamically adjusted during the training process.

[0125] Specifically, the backbone network's 4th and 6th MobileNetV3_1 layers are concatenated with the neck's 12th and 16th BiFPN_Concat2_2 layers, respectively, to generate new feature maps: the first and second feature maps. The 6th MobileNetV3_2 layer is then concatenated with the 14th layer to generate the third feature map, which is then output to the detection head. This multi-layer feature fusion integrates comprehensive information and improves the model's expressiveness.

[0126] In step S5-3, after the neck is processed, feature maps of different sizes are input to the detection head. Before the detection head, an ECA attention mechanism is introduced to increase feature representation and global context information. The convolution kernel size of the one-dimensional convolution is adaptively adjusted using a function, allowing layers with larger channels to interact more frequently across channels, improving feature representation and object detection accuracy.

[0127] The calculation formula for the adaptive convolution kernel size is:

[0128]

[0129] Here, k represents the kernel size of the one-dimensional convolution, C is the number of channels of the input feature, and γ and b are hyperparameters. By taking the absolute value and rounding it down to the nearest odd number, we ensure that the kernel size is always odd. Once k is determined, the ECA module will perform a one-dimensional convolution on the input feature, aiming to capture the importance of each channel relative to the other channels. The formula is as follows:

[0130] [out=Conv1D k (in)]

[0131] This formula represents the conversion of input feature in into output feature out through a one-dimensional convolution operation (kernel size is k). Here, ConvlDk refers to a one-dimensional convolution operation with a kernel size of k.

[0132] YOLOv5's detection head uses an anchor-based mechanism, which sets a series of predefined anchor boxes on the feature map and uses these anchor boxes to predict the target category and bounding box location. Specifically, the detection head performs classification prediction and bounding box regression on each anchor box, outputting the target category, bounding box location, and confidence score for predicting four categories of defect types. Finally, NMS (Non-Maximum Suppression) is used to remove redundant detection boxes, retaining the final detection results and reducing false positives and missed detections. In object detection, the loss function is a key tool for evaluating model performance. It calculates the difference between the predicted results and the true labels to guide the update of model parameters to gradually improve the model's detection accuracy. Common object detection loss functions include classification loss and regression loss. Classification loss is used to evaluate the model's ability to predict the target category, while regression loss is used to evaluate the model's prediction accuracy for the target bounding box location.

[0133] IoU (Intersection over Union) is a commonly used evaluation metric and regression loss function in object detection. The calculation formula of IoU is:

[0134]

[0135] Where A and B represent the predicted bounding box and the ground-truth bounding box, respectively. The IoU value ranges from 0 to 1, with larger values ​​indicating greater overlap between the predicted and ground-truth bounding boxes and better model performance. IoU directly measures the degree of overlap between the predicted and ground-truth bounding boxes, making it intuitive and easy to understand. However, IoU also has some drawbacks. First, when the predicted and ground-truth bounding boxes are completely disjoint, the IoU value is 0, which does not reflect the distance between the predicted and ground-truth bounding boxes. Second, IoU exhibits discontinuity during the optimization process. That is, when the predicted and ground-truth bounding boxes do not overlap, the gradient of the loss function is zero, making it difficult for the model to update its parameters.

[0136] To overcome the shortcomings of IoU, GIoU (Generalized Intersection over Union) was introduced. GIoU not only considers the overlapping area between the predicted box and the ground-truth box but also introduces the concept of a minimum bounding rectangle to reflect the relative positional relationship between the predicted box and the ground-truth box. GIoU adjusts the IoU by subtracting the proportion of the uncovered area, allowing the loss function to still provide valid gradient information when the predicted box and the ground-truth box do not intersect, thus avoiding the discontinuity problem of IoU. Furthermore, GIoU can push the predicted box closer to the ground-truth box during the optimization process, improving the model's convergence speed and detection accuracy. The final detection head output sizes are 256×80×80, 512×40×40, and 1024×20×20.

[0137] Step S6: Input the surface image of the aircraft engine to be inspected into the trained MBE-YOLOv5 surface defect detection model, and output the aircraft engine surface defect detection result.

[0138] Example 2

[0139] This embodiment also relates to a verification test of an aircraft engine surface defect detection method.

[0140] This method was tested on a dataset of surface defect images of aircraft engine turbine blades, undergoing repeated parameter adjustment and training. The dataset contains 330 images containing defects, representing four defect types: scratches, dots, creases, and damage. These images were divided into a training set and a validation set in an 8:2 ratio for input training.

[0141] Mosaic data enhancement is also enabled during the image input process of the model. Four images are spliced ​​together using random cropping and scaling. The number of images actually involved in training is larger and the samples are richer.

[0142] To evaluate the quality of the model, this paper uses a variety of evaluation indicators for comparison. Among them, the mean average precision (mAP) and the number of frames per second (FPS) are the most commonly used evaluation indicators in the field of object detection. The mAP is calculated by the precision P (Precision) and the recall R (Recall):

[0143]

[0144] Among them, TP, FP, and FN are the number of actual positive samples predicted as positive samples, the number of actual negative samples predicted as positive samples, and the number of actual positive samples predicted as negative samples, respectively, and c is the number of defect types.

[0145] The operating environment of the algorithm of this method is Intel(R) Xeon(R) CPU@2.00GHz, GPU is NVIDIA TeslaP100 PCIe 16GB, Batchsize is 64, epochs training round number is 300, Adam is used as optimizer, and learning rate is 0.01. The training parameter results of YOLOv5 model and the method of the present invention on the surface of engine turbine blades are shown in Table 1, unit: %. By comparing the result data, it is found that compared with the original YOLOv5 model, the optimization model proposed by this method has a relatively large improvement in the detection accuracy of most defect types, and has a significant improvement for scratches and creases. The accuracy of each category reaches more than 90%, and the overall accuracy, recall rate and average accuracy all reach more than 90% and have been improved. At the same time, GFLOPs (floating point operations per second) has been reduced from the original 15.8 to 14.9, and the number of parameters (Params) has been reduced from the original 7.02*10 6 Reduced to 6.57*10 6 The reduction in GFLOPs means the model designed by this invention requires fewer computing resources and runs faster. The reduced number of parameters means the model complexity is reduced, and the storage space used is also reduced. Compared with the original YOLOv5, the model FPS (the time required to process each image) of this method has increased from 91.74 to 94.34, with enhanced real-time performance. It processes images faster and can process more frames in a shorter time. This is particularly important for real-time visual inspection applications in the aerospace field.

[0146] The optimization results improve operational efficiency and reduce resource consumption while maintaining model performance. This means that this method has high application value in aero-engine surface defect detection.

[0147] Table 1

[0148]

[0149] Example 3

[0150] The electronic device of the present invention includes a central processing unit (CPU), which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) or loaded from a storage unit into a random access memory (RAM). In the RAM, various programs and data required for device operation can also be stored. The CPU, ROM, and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.

[0151] Many components in a device are connected to the I / O interface, including: input units, such as a keyboard and mouse; output units, such as various types of displays and speakers; storage units, such as magnetic disks and optical disks; and communication units, such as network cards, modems, and wireless communication transceivers. The communication unit allows the device to exchange information / data with other devices via computer networks such as the Internet and / or various telecommunication networks.

[0152] The processing unit performs the various methods and processes described above, such as methods S1 to S6. For example, in some embodiments, methods S1 to S6 may be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as a storage unit. In some embodiments, part or all of the computer program may be loaded and / or installed onto the device via a ROM and / or a communication unit. When the computer program is loaded into the RAM and executed by the CPU, one or more steps of methods S1 to S6 described above may be performed. Alternatively, in other embodiments, the CPU may be configured to execute methods S1 to S6 by any other appropriate means (e.g., by means of firmware).

[0153] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0154] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. Such program code can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0155] In the context of the present invention, machine-readable medium can be a tangible medium that can contain or store a program for use with an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. Machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. More specific examples of machine-readable storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0156] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and such modifications or substitutions are intended to be within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A method for detecting surface defects of an aircraft engine, characterized in that: The method comprises the following steps: Step S1, acquiring an aircraft engine surface image; Step S2: Put the labeled surface defect dataset into the MBE-YOLOv5 surface defect detection model for training; Step S3: inputting the surface image of the aircraft engine to be inspected into the trained MBE-YOLOv5 surface defect detection model, and outputting the aircraft engine surface defect detection result; The MBE-YOLOv5 surface defect detection model is an improvement on the YOLOv5 model. The MBE-YOLOv5 surface defect detection model uses MobileNetV3 Large as the backbone network, optimizes the neck to BiFPN, and adds an ECA attention mechanism before the detection head. The backbone network of the MBE-YOLOv5 surface defect detection model includes a CBS module, a MobileNetV3 network module, and an SPPF module; The backbone network first performs a 1×1 convolution operation on the input feature map to adjust the dimension of the feature map. The feature map then passes through a series of inverted residual structures and is finally processed by the SPPF module to output feature maps of different scales to the neck. The 4th and 6th layers of MobileNetV3 of the backbone network are spliced ​​with the 12th and 16th layers of BiFPN in the neck, respectively, to obtain the first and second feature maps, respectively. The 6th layer of MobileNetV3 is then spliced ​​with the 14th layer of BiFPN to generate the third feature map, which is output to the detection head.

2. The method for detecting surface defects of an aircraft engine according to claim 1, characterized in that: The neck of the MBE-YOLOv5 surface defect detection model uses a BiFPN structure to perform bidirectional information fusion on feature maps of different scales, and a weighted feature fusion mechanism is used in the feature fusion process of each scale.

3. The method for detecting surface defects of an aircraft engine according to claim 1, characterized in that: The method includes: before training the MBE-YOLOv5 surface defect detection model, using the K-means clustering algorithm, obtaining optimized anchor points adapted to the surface defect data set according to the genetic algorithm, and putting them into the detection model for training.

4. The method for detecting surface defects of an aircraft engine according to claim 3, characterized in that: The process of obtaining an optimized anchor point adapted to the surface defect data set according to a genetic algorithm includes: Step S3-1, read the width and height of each image in the training set, as well as the width and height of all marked boxes; Step S3-2, scaling the maximum width and height of each image to a specified size; Step S3-3, converting the marking frame from relative coordinates to absolute coordinates; Step S3-4, filter the marker boxes and retain the marker boxes whose width and height are both greater than or equal to two pixels; Step S3-5, using K-means clustering algorithm to obtain k anchor points; Step S3-6: Use a genetic algorithm to randomly mutate the width and height of the anchor point. If the effect after mutation is better, assign the mutated result to the anchor point; if the effect after mutation is worse, skip it; In step S3-7, the anchor points obtained by the final mutation are sorted by area and returned for use in the training of the MBE-YOLOv5 surface defect detection model.

5. The method for detecting surface defects of an aircraft engine according to claim 1, characterized in that: The method includes: preprocessing the aircraft engine surface image to obtain a surface defect image, wherein the preprocessing includes data enhancement through inversion, translation, brightness change, and noise enhancement to increase the number of data samples.

6. The method for detecting surface defects of an aircraft engine according to claim 5, characterized in that: The method further includes: annotating defects in the surface defect image to obtain an annotation file, and the surface defect image and the corresponding annotation file constitute a surface defect dataset.

7. The method for detecting surface defects of an aircraft engine according to claim 5, characterized in that: Annotate the defects in the surface defect image to form an annotation file for each image, wherein the data format in the annotation file is set according to the YOLO format; The annotation file includes defect category information and defect location information.

8. An electronic device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

Images