A method for defect detection of aero-engine blades based on small sample learning

Abstract

The present application relates to a method for defect detection of aircraft engine blades based on small sample learning, the method comprising: obtaining a surface defect image of an aircraft engine turbine blade and establishing a mask true value annotation for it; dividing multiple surface defect images into a training set, a support set, and a query set according to the corresponding true value mask, and constructing a defect detection model, the defect detection model comprising a cross-layer multi-level feature aggregation module, a mixer with a large receptive field, and a foreground correction module; aggregating features of different levels of support images in a support set and query images in a query set through the cross-layer multi-level feature aggregation module to obtain defect features; processing defect features through a mixer with a large receptive field to obtain aggregate features; fusing and correcting aggregate features with extracted background features of the query image through the foreground correction module to obtain a refined segmentation result. The method improves the efficiency and cost-effectiveness of model training.

Description

Technical Field

[0001] The present application relates to the technical field of aircraft engine blade defect detection, and in particular to an aircraft engine blade defect detection method based on small sample learning. Background Art

[0002] At present, visual inspection of surface defects of aircraft engine turbine blades is a very challenging task, especially in the actual production process, the probability of defects is extremely low and presents a significant long-tail effect. Specifically, some types of defects occur very frequently, are easier to collect and have a relatively sufficient number of samples; while other types of defects are extremely rare and have a very low probability of occurrence, making it difficult to obtain sufficient sample data. This imbalance in data distribution poses great difficulties to traditional machine learning methods, because machine learning models usually rely on a large amount of labeled data to ensure their accuracy and efficiency; however, in the field of defect detection, the scarcity of defect samples makes traditional methods difficult to cope with, and for unseen categories, the model needs to be retrained. Therefore, there is an urgent need for a method based on small sample learning for defect detection on the surface of aircraft engine turbine blades. Summary of the invention

[0003] Based on this, it is necessary to provide an aero-engine blade defect detection method based on small sample learning, which includes:

[0004] S1: Obtain the surface defect image of the aircraft engine turbine blade and establish the mask truth annotation for it;

[0005] S2: dividing multiple surface defect images into a training set, a support set, and a query set according to corresponding true value masks, and constructing a defect detection model, wherein the defect detection model includes a cross-layer multi-level feature aggregation module, a mixer with a large receptive field, and a foreground correction module; and training the defect detection model through the training set;

[0006] S3: The defect features are obtained by aggregating the features of different levels of the support image in the support set and the query image in the query set through the cross-layer multi-level feature aggregation module;

[0007] S4: Processing the defect features through a mixer with a large receptive field to obtain aggregate features;

[0008] S5: The aggregated features are fused and corrected with the extracted background features of the query image through a foreground correction module to obtain a refined segmentation result.

[0009] Preferably, S1 comprises:

[0010] S1.1: obtaining an uncompressed original image of an aircraft engine turbine blade, and selecting an image containing defects from a plurality of original images as the surface defect image;

[0011] S1.2: Determine the category of each surface defect image through expert knowledge, and establish a mask truth annotation for the corresponding surface defect image based on the category.

[0012] Preferably, dividing the plurality of surface defect images into a training set, a support set, and a query set according to corresponding true value masks comprises:

[0013] Step 1: Divide the surface defect image into different category folders according to the corresponding true value mask;

[0014] Step 2: Randomly divide the training set, support set, and query set from any of the category folders;

[0015] Step 3: Repeat step 2, traverse all category folders, and divide multiple sets of training sets, support sets, and query sets from different category folders; multiple sets of training sets, support sets, and query sets are used for cross-validation.

[0016] Preferably, S3 includes:

[0017] S3.1: Use bilinear interpolation to adjust the size of the support image and its ground-truth mask, and correspond the different sizes of support images and their ground-truth masks from small to large to the levels of the cross-layer multi-level feature aggregation module;

[0018] S3.2: In the low-level and high-level layers of the cross-layer multi-level feature aggregation module, different processing methods are used, where ResNet-50 is used to extract low-level features of the support image and the query image respectively, and a dilated convolutional neural network is used to extract high-level features of the support image and the query image respectively, where the low-level features are the basic features of the target and the high-level features are the semantic features of the target;

[0019] S3.3: Based on the support images at different levels and the ground truth masks at their corresponding levels, support features at different levels are obtained;

[0020] S3.4: Superimpose the supporting features of all levels to the same dimension and normalize them to obtain the abnormal features;

[0021] S3.5: Concatenate the abnormal features with the low-level features and high-level features of the query image to obtain the query defect features;

[0022] S3.6: Use bilinear interpolation method to adjust the size of the query defect feature, connect query defect features of different sizes, and obtain the defect feature.

[0023] Preferably, the calculation formula of the supporting feature is:

[0024] ;

[0025] in, Indicates support features; Indicates support for images; express The corresponding ground truth mask; Represents pixel-by-pixel multiplication.

[0026] Preferably, S4 includes:

[0027] S4.1: an expanded spatial pyramid pooling structure is used, which includes a 1×1 convolution kernel, a 3×3 convolution kernel, and a global pooling layer. The defect features are multi-scale fused through convolution kernels with different expansion rates and aggregated through pooling to obtain the size of the aggregated defect features;

[0028] S4.2: Use bilinear interpolation to adjust the size of the defect features after polymerization;

[0029] S4.3: The resized defect features are connected in series through multiple convolution-activation-Dropout blocks to obtain the aggregated features.

[0030] Preferably, S5 includes:

[0031] S5.1: Extract background features of the query image through a background predictor based on an encoder-decoder structure;

[0032] S5.2: Fusing the background feature with the aggregate feature to obtain a fused feature;

[0033] S5.3: The fused features are corrected by a foreground corrector based on an encoder-decoder structure to obtain a refined segmentation result.

[0034] Preferably, the average intersection-and-union ratio and the frequency-based intersection-and-union ratio are used as evaluation indicators to evaluate the segmentation effect, and the calculation formula is:

[0035] ;

[0036] ;

[0037] Among them, mIoU represents mean intersection over union; C represents the number of categories; represents the intersection-over-union ratio of the i-th category; FB-IoU represents the frequency-based intersection-over-union ratio; represents the predicted bounding box; represents the ground-truth bounding box.

[0038] Beneficial effects: This method overcomes the problem that in the production process of aircraft engine blades, due to the long-tail effect of defects, some defect samples are difficult to collect, which makes traditional methods that rely on large data sets difficult to apply. Even when the number of samples is limited, a defect detection model with excellent performance can be trained, which not only reduces the dependence on a large amount of labeled data, but also improves the efficiency and cost-effectiveness of model training. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0040] Figure 1 This is a flow chart of an aircraft engine blade defect detection method based on small sample learning in an embodiment of the present application. DETAILED DESCRIPTION

[0041] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present application, so the present application is not limited by the specific embodiments disclosed below.

[0042] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0043] The defect detection method based on small sample learning aims to train the model through a small amount of labeled sample data, so as to accurately identify a large amount of new data that belongs to the same category as the training sample but in a different form. Compared with traditional machine learning methods, small sample learning has a significant advantage, that is, it can effectively reduce the dependence on large-scale labeled data. At the same time, the method also shows a strong generalization ability. Even in the face of new categories of defect images that have never been seen before, it can still be accurately detected. This feature makes small sample learning have a wide range of application potential in industrial production environments, and can better cope with the unforeseen complexity and uncertainty in the production process. The development of a defect detection method based on small sample learning not only helps to solve the problem of scarce defect data on the surface of turbine blades, but also can effectively improve the adaptability and generalization ability of the detection model, and meet the high-precision and high-efficiency detection requirements of the aero-engine manufacturing process. Therefore, this embodiment provides an aero-engine blade defect detection method based on small sample learning.

[0044] like Figure 1 As shown, this embodiment provides an aircraft engine blade defect detection method based on small sample learning, the method comprising:

[0045] S1: Obtain the surface defect image of the aircraft engine turbine blade and establish the mask truth annotation for it.

[0046] Specifically, the step includes:

[0047] S1.1: obtaining an uncompressed original image of an aircraft engine turbine blade, and selecting an image containing defects from a plurality of original images as the surface defect image;

[0048] S1.2: Determine the category of each surface defect image through expert knowledge, and establish a mask truth annotation for the corresponding surface defect image based on the category.

[0049] In this embodiment, the original images come from the aircraft engine blade surface defect detection datasets FSSD-12 and FSS-AEB, where FSSD-12 contains 600 images in 12 categories, and FSS-AEB contains 300 images in 6 categories. The dataset covers 16 different defect types commonly found on steel surfaces.

[0050] S2: Divide multiple surface defect images into a training set, a support set, and a query set according to corresponding true value masks, and construct a defect detection model, wherein the defect detection model includes a cross-layer multi-level feature aggregation module, a mixer with a large receptive field, and a foreground correction module; train the defect detection model through the training set.

[0051] Specifically, dividing the plurality of surface defect images into a training set, a support set, and a query set according to corresponding true value masks includes:

[0052] Step 1: Divide the surface defect image into different category folders according to the corresponding true value mask;

[0053] Step 2: Randomly divide the training set, support set, and query set from any of the category folders;

[0054] The training set, support set, and query set meet the following conditions:

[0055] ;

[0056] in, represents the training set; represents the test set, which includes the support set S and the test set Q; Indicates empty;

[0057] Step 3: Repeat step 2, traverse all category folders, and divide multiple sets of training sets, support sets, and query sets from different category folders; multiple sets of training sets, support sets, and query sets are used for cross-validation.

[0058] The training process includes: using the dataset class to define the images of the training set and their labels, using the data loader to load the image-label pairs in the dataset, using the defect detection model to infer the loaded image data, and calculating the loss between the model output and the label (the loss function can use the cross entropy loss and other common loss functions used for classification problems).

[0059] S3: The defect features are obtained by aggregating the features of different levels of the support image in the support set and the query image in the query set through the cross-layer multi-level feature aggregation module.

[0060] Specifically, the step includes:

[0061] S3.1: Use bilinear interpolation to adjust the size of the support image and its ground-truth mask, and correspond the different sizes of support images and their ground-truth masks from small to large to the levels of the cross-layer multi-level feature aggregation module;

[0062] S3.2: Different processing methods are used in the low-level and high-level layers of the cross-layer multi-level feature aggregation module. ResNet-50 pre-trained on ImageNet is used to extract low-level features of the support image and query image respectively, and dilated convolutional neural network is used to extract high-level features of the support image and query image respectively, so that the cross-layer multi-level feature aggregation module can extract rich defect features; the low-level features are the basic features of the target, and the high-level features are the semantic features of the target;

[0063] S3.3: Based on the support images at different levels and the true value masks at their corresponding levels, the support features at different levels are obtained; the calculation formula of the support features is:

[0064] ;

[0065] in, Indicates support features; Indicates support for images; express The corresponding ground truth mask; Represents pixel-by-pixel multiplication.

[0066] S3.4: Superimpose the supporting features of all levels to the same dimension and normalize them to obtain the abnormal features;

[0067] S3.5: Concatenate the abnormal features with the low-level features and high-level features of the query image to obtain the query defect features;

[0068] S3.6: Use bilinear interpolation method to adjust the size of the query defect feature, connect query defect features of different sizes, and obtain the defect feature.

[0069] S4: Process the defect features through a mixer with a large receptive field to obtain aggregated features.

[0070] Specifically, S4 includes:

[0071] S4.1: an expanded spatial pyramid pooling structure is used, which includes a 1×1 convolution kernel, a 3×3 convolution kernel, and a global pooling layer. The defect features are multi-scale fused through convolution kernels with different expansion rates and more context information is aggregated through pooling to capture defects of different sizes and obtain the size of the aggregated defect features.

[0072] The feature channels are fused through a 1×1 convolution kernel to enhance the expressiveness of features;

[0073] The global pooling layer is used to obtain statistical information of the defect features to enhance the model's perception of defects at different locations.

[0074] S4.2: Use bilinear interpolation to adjust the size of the aggregated defect features to ensure that defect features at different levels can be matched and fused;

[0075] S4.3: The resized defect features are connected in series through multiple convolution-activation-Dropout blocks to further extract and abstract features, enhance the generalization ability of the model, and obtain aggregated features.

[0076] S5: The aggregated features are fused and corrected with the extracted background features of the query image through a foreground correction module to obtain a refined segmentation result.

[0077] Specifically, the step includes:

[0078] S5.1: extracting background features of the query image through a background predictor based on an encoder-decoder structure (the background predictor is a general encoder-decoder structure and its operation process is also consistent with the general encoder-decoder structure);

[0079] S5.2: Fusing the background feature with the aggregate feature to obtain a fused feature;

[0080] S5.3: The fused features are corrected by a foreground corrector based on an encoder-decoder structure (the foreground corrector is a general encoder-decoder structure, and its operation process is also consistent with the general encoder-decoder structure) to obtain a refined segmentation result.

[0081] In this embodiment, the average intersection-and-union ratio and the frequency-based intersection-and-union ratio are used as evaluation indicators to evaluate the segmentation effect, and the calculation formula is:

[0082] ;

[0083] ;

[0084] Among them, mIoU represents mean intersection over union; C represents the number of categories; represents the intersection-over-union ratio of the i-th category; FB-IoU represents the frequency-based intersection-over-union ratio; represents the predicted bounding box; represents the ground-truth bounding box.

[0085] The aircraft engine blade defect detection method based on small sample learning provided in this embodiment has the following beneficial effects:

[0086] 1. This method overcomes the problem that in the production process of aircraft engine blades, due to the long-tail effect of defects, some defect samples are difficult to collect, which makes traditional methods that rely on large data sets difficult to apply by adopting a small sample learning method. Even when the number of samples is limited, a defect detection model with excellent performance can be trained, which not only reduces the dependence on a large amount of labeled data, but also improves the efficiency and cost-effectiveness of model training.

[0087] 2. This method has excellent generalization ability and can adapt to changing production environments. Faced with various types of defects that may occur in turbine blades, and these defects may appear in different forms, this method can learn generalized feature representations from limited training samples, so that the model can still accurately identify and detect new defect categories that have never been seen. This makes the method not only applicable to currently known defect types, but also can cope with new defects that may appear in the future, greatly enhancing the flexibility and reliability of the detection system.

[0088] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0089] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be construed as limiting the scope of the patent application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent application shall be subject to the attached claims.

Claims

1. A method for detecting defects in aircraft engine blades based on small sample learning, characterized in that: include: S1: Obtain the surface defect image of the aircraft engine turbine blade and establish the mask truth annotation for it; S2: dividing multiple surface defect images into a training set, a support set, and a query set according to corresponding true value masks, and constructing a defect detection model, wherein the defect detection model includes a cross-layer multi-level feature aggregation module, a mixer with a large receptive field, and a foreground correction module; and training the defect detection model through the training set; S3: The defect features are obtained by aggregating the features of different levels of the support image in the support set and the query image in the query set through the cross-layer multi-level feature aggregation module; S3.1: Use bilinear interpolation to adjust the size of the support image and its ground-truth mask, and correspond the different sizes of support images and their ground-truth masks from small to large to the levels of the cross-layer multi-level feature aggregation module; S3.2: In the low-level and high-level layers of the cross-layer multi-level feature aggregation module, different processing methods are used, where ResNet-50 is used to extract low-level features of the support image and the query image respectively, and a dilated convolutional neural network is used to extract high-level features of the support image and the query image respectively, where the low-level features are the basic features of the target and the high-level features are the semantic features of the target; S3.3: Based on the support images at different levels and the ground truth masks at their corresponding levels, support features at different levels are obtained; S3.4: Superimpose the supporting features of all levels to the same dimension and normalize them to obtain the abnormal features; S3.5: Concatenate the abnormal features with the low-level features and high-level features of the query image to obtain the query defect features; S3.6: Use a bilinear interpolation method to adjust the size of the query defect feature, connect the query defect features of different sizes, and obtain the defect feature; S4: Processing the defect features through a mixer with a large receptive field to obtain aggregate features; S5: The aggregated features are fused and corrected with the extracted background features of the query image through a foreground correction module to obtain a refined segmentation result.

2. The method for detecting defects in aircraft engine blades based on small sample learning according to claim 1, characterized in that: S1 includes: S1.1: obtaining an uncompressed original image of an aircraft engine turbine blade, and selecting an image containing defects from a plurality of original images as the surface defect image; S1.2: Determine the category of each surface defect image through expert knowledge, and establish a mask truth annotation for the corresponding surface defect image based on the category.

3. The method for detecting defects in aircraft engine blades based on small sample learning according to claim 1, characterized in that: The dividing of the plurality of surface defect images into a training set, a support set, and a query set according to the corresponding true value masks comprises: Step 1: Divide the surface defect image into different category folders according to the corresponding true value mask; Step 2: Randomly divide the training set, support set, and query set from any of the category folders; Step 3: Repeat step 2, traverse all category folders, and divide multiple sets of training sets, support sets, and query sets from different category folders; multiple sets of training sets, support sets, and query sets are used for cross-validation.

4. The method for detecting defects in aircraft engine blades based on small sample learning according to claim 1, characterized in that: The calculation formula for the support feature is: ； in, Indicates support features; Indicates support for images; express The corresponding ground truth mask; Represents pixel-by-pixel multiplication.

5. The method for detecting defects in aircraft engine blades based on small sample learning according to claim 1, characterized in that S4 include: S4.1: an expanded spatial pyramid pooling structure is used, which includes a 1×1 convolution kernel, a 3×3 convolution kernel, and a global pooling layer. The defect features are multi-scale fused through convolution kernels with different expansion rates and aggregated through pooling to obtain the size of the aggregated defect features; S4.2: Use bilinear interpolation to adjust the size of the defect features after polymerization; S4.3: The resized defect features are connected in series through multiple convolution-activation-Dropout blocks to obtain the aggregated features.

6. The method for detecting defects in aircraft engine blades based on small sample learning according to claim 1, characterized in that S5 include: S5.1: Extract background features of the query image through a background predictor based on an encoder-decoder structure; S5.2: Fusing the background feature with the aggregate feature to obtain a fused feature; S5.3: The fused features are corrected by a foreground corrector based on an encoder-decoder structure to obtain a refined segmentation result.

7. The method for detecting defects in aircraft engine blades based on small sample learning according to claim 1, characterized in that: The average intersection-and-union ratio and the frequency-based intersection-and-union ratio are used as evaluation indicators to evaluate the segmentation effect. The calculation formula is: ； ； Among them, mIoU represents mean intersection over union; C represents the number of categories; represents the intersection-over-union ratio of the i-th category; FB-IoU represents the frequency-based intersection-over-union ratio; represents the predicted bounding box; represents the ground-truth bounding box.

Images