A method for defect detection of aero-engine blades based on weakly supervised learning

Abstract

The present application relates to an aircraft engine blade defect detection method based on weakly supervised learning, which comprises the following steps: obtaining a surface image of an aircraft engine turbine blade and labeling the image by categories; obtaining a preliminary defect location map by using a classification network based on the surface image and its corresponding category labels, and generating a corresponding defect area boundary box; taking blocks of the surface image and the preliminary defect location map based on the defect area boundary box, constructing a pair of blocks, and performing cluster analysis on the pair of blocks; selecting a cluster of vectors from the clustering results, and adaptively suppressing the area mapped by the selected vectors on the preliminary defect location map and the surface image; fusing the suppressed surface image and the preliminary defect location map, and inputting the two into a classification network to obtain an updated defect location map; converting the updated defect location map to obtain an updated defect area boundary box; inputting the surface image and the updated defect area boundary box into a visual segmentation model to obtain a defect segmentation result.

Description

Technical Field

[0001] The present application relates to the technical field of aircraft engine blade defect detection, and in particular to an aircraft engine blade defect detection method based on weakly supervised learning. Background Art

[0002] Visual inspection of defects on the surface of aircraft engine turbine blades is a challenging task, especially when facing multi-scale and random targets. Intelligent inspection technology based on machine vision is efficient and feasible. However, traditional supervised learning methods require a large amount of high-quality pixel-level annotated data. Defect detection methods based on weak supervised learning only use coarse-grained annotation information to improve the automation level of defect detection. They can handle the uncertainty and incompleteness in the annotation information and learn object detection tasks from image-level labels. In addition, by learning weaker supervisory signals, the model may develop better generalization capabilities. Methods based on weak supervised learning will not overfit to incompletely annotated data or noisy data caused by human errors. By learning from limited annotated information, they can better adapt to the changing industrial environment.

[0003] However, considering complex industrial scenarios such as weak light or strong interference, and due to the large variance between different data sets, previous positioning or detection algorithms have poor generalization performance. Therefore, the detected high-confidence areas are very likely not defective objects, resulting in incorrect positioning results. Summary of the invention

[0004] Based on this, it is necessary to provide an aero-engine blade defect detection method based on weakly supervised learning, which includes:

[0005] S1: Obtain the surface image of the aircraft engine turbine blade and label it according to its category;

[0006] S2: Based on the surface image and its corresponding category label, a preliminary defect location map is obtained using a classification network, and a corresponding defect area boundary box is generated;

[0007] S3: taking blocks from the surface image and the preliminary defect location map based on the defect area boundary box, constructing block pairs, and performing cluster analysis on the block pairs;

[0008] S4: selecting a cluster of vectors from the clustering results, and adaptively suppressing the region mapped by the selected vectors on the preliminary defect location map and the surface image;

[0009] S5: fusing the suppressed surface image and the preliminary defect location map, and inputting them into the classification network to obtain an updated defect location map; converting the updated defect area boundary box based on the updated defect location map;

[0010] S6: Input the surface image and the updated defect area boundary box into a visual segmentation model to obtain a defect segmentation result.

[0011] Beneficial effects: This method uses the candidate areas generated by the initial positioning map and, guided by the results of clustering analysis, can optimize the problems of false detection and missed detection in the original positioning map, and achieve more accurate surface defect detection of aircraft engine blades. This method does not require a large amount of pixel annotation data, and only uses weakly annotated data (image-level labels) for surface defect detection of aircraft engine blades, which greatly reduces the cost and time of data annotation, and can better handle situations where the data volume is large but the annotation is difficult. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0013] Figure 1 This is a flow chart of an aircraft engine blade defect detection method based on weakly supervised learning according to an embodiment of the present application. DETAILED DESCRIPTION

[0014] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present application, so the present application is not limited by the specific embodiments disclosed below.

[0015] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0016] This embodiment provides a weakly supervised defect detection method based on aircraft engine turbine blade defect images and their category text labels, which can be applied to defect location detection under different complex backgrounds and scene conditions. Specifically, it is a weakly supervised learning aircraft engine blade defect detection method.

[0017] like Figure 1As shown, the method includes:

[0018] S1: Obtain surface images of aircraft engine turbine blades and label them by category.

[0019] In this embodiment, the category labeling includes:

[0020] According to an expert's prior judgment, the category corresponding to the surface image;

[0021] The surface image is labeled with a corresponding category label.

[0022] S2: Based on the surface image and its corresponding category label, a preliminary defect location map is obtained using a classification network, and a corresponding defect area boundary box is generated.

[0023] In this embodiment, the classification network uses the ResNet-50 network as the backbone network.

[0024] Furthermore, the process of obtaining a preliminary defect location map includes:

[0025] Step 1: Input the surface image and its corresponding category label into the ResNet-50 network;

[0026] Step 2: Perform an adaptive averaging operation on the output of the last layer of the ResNet-50 network, pass the feature map after the operation through a fully connected output head that only contains a fully connected layer to output a category prediction result, and pass the feature map after the operation through a convolutional output head that only contains a convolutional layer to output the preliminary defect location map; the category label and the category prediction result are used to calculate the loss function to train the classification network.

[0027] The calculation formula for the category prediction result and the preliminary defect location map is:

[0028] ;

[0029] ;

[0030] in, Represents the category prediction result; C It represents the preliminary defect location map; Indicates a fully connected output header; represents the convolution output head; Represents the feature map after operation.

[0031] Furthermore, the process of generating the corresponding defect area bounding box includes:

[0032] Step 1: Perform bicubic interpolation on the preliminary defect location map;

[0033] Step 2: Normalize the interpolated positioning map; the normalization formula is:

[0034] ;

[0035] in, Defect preliminary location diagram C The horizontal coordinate is i , the vertical axis is j Pixels; W Indicates the width of the preliminary defect location map; H Indicates the height of the preliminary defect location map;

[0036] Step 3: Binarize the normalized localization map by combining the binary threshold method and the OTSU algorithm to obtain a binary image;

[0037] Step 4: Calculate the range of the outer contour of any connected area in the binary image to obtain the minimum bounding box of each connected area. The minimum bounding box is the defect area bounding box. The value of the minimum bounding box includes the horizontal coordinate of the upper left corner. x 1 and the vertical axis y 1. The width of the bounding box w 1 and height h 1.

[0038] S3: based on the defect area boundary box, the surface image and the preliminary defect location map are blocked, a block pair is constructed, and a cluster analysis is performed on the block pair.

[0039] Specifically, the step includes:

[0040] S3.1: For any of the defect area boundary boxes, the local mapping of the area within the box corresponding to the preliminary defect positioning map and the surface image is cropped to obtain a positioning block and an image block respectively; the pixel coordinates of the upper left corner of the cropped area are ( x 1, y 1), the pixel coordinates of its lower right corner are ( x 1 +w 1, y 1 +h 1);

[0041] S3.2: resizing the sizes of the positioning block and the image block to 1 / 4 of the surface image by a bilinear interpolation algorithm, and combining the positioning block and the image block after the resizing in pairs to obtain a block pair, wherein the block pair is represented as: <image block, positioning block>, and recording the index value of the surface image corresponding to the image block and the area mapped in the surface image;

[0042] S3.3: Stack the image block pairs according to the number of image blocks to form a tensor; the size of the tensor is: ;in, P represents a tensor; N Indicates the number of image blocks; CH Represents the number of channels of the surface image;

[0043] S3.4: Input the tensor into a small convolutional neural network for feature extraction to obtain extracted features; the small convolutional neural network includes a structure of four groups, namely, a convolution layer, a normalization layer, and a pooling layer; the feature extraction formula is:

[0044] ;

[0045] in, Represents extracted features; represents a small convolutional neural network;

[0046] S3.5: taking an average of the extracted features according to the number of channels of the surface image and eliminating the number of channels to obtain a compressed feature representation of the tile pair; the size of the compressed feature representation is: ;

[0047] S3.6: Select from the compressed feature representation a feature from the same surface image or defect preliminary location map according to the recorded index value. N compressed feature vectors; N Indicates the number of image blocks;

[0048] S3.7: K-Means clustering algorithm is used based on N The compressed feature vectors are used to perform cluster analysis on the tile pairs and the tile pairs are clustered into two categories.

[0049] S4: selecting a cluster of vectors from the clustering results, and adaptively suppressing the area where the selected vectors are mapped on the preliminary defect location map and the surface image.

[0050] Specifically, the step includes:

[0051] S4.1: randomly selecting a cluster of vectors from the clustering results, and using the vectors as inhibitory feedback;

[0052] S4.2: The regions of the suppression feedback mapping on the defect preliminary location map and the surface image are taken out, and suppression processing is performed respectively; the suppression formula is:

[0053] ;

[0054] ;

[0055] ;

[0056] in, Preliminary defect localization diagram showing the vector mapping in the selected cluster after suppression No. i Regions; Surface image showing the vector map in the selected cluster after suppression No. i Regions; m The vertical coordinate of the region; n The horizontal coordinate of the region; Defect preliminary location diagram C No. i Regions; Representing surface images X No. i Regions; Represents a parameter set Middle Hyperparameters of Represents the adaptive average pooling operation; q represents the predefined number of clusters; s represents the index of the cluster that serves as inhibitory feedback; Indicates the number of regions;

[0057] S4.3: Re-add the suppressed area to its original position in the surface image or the preliminary defect location map.

[0058] S5: Fusing the suppressed surface image and the preliminary defect location map, and inputting them into the classification network to obtain an updated defect location map; converting the updated defect area boundary box based on the updated defect location map.

[0059] Specifically, the step includes:

[0060] S5.1: Fuse the suppressed surface image and the preliminary defect location map. The fusion formula is:

[0061] ;

[0062] in, represents the fused image; It represents the preliminary location map of the defect after suppression; represents the surface image after suppression;

[0063] S5.2: input the fused image into the classification network, output the second category prediction result through the fully connected output head, and output the defect update location map through the convolution output head; the category label and the second category prediction result are used to calculate the loss function to train the classification network;

[0064] The calculation formula for the second category prediction result and defect update location map is:

[0065] ;

[0066] ;

[0067] in, Indicates the prediction result of the second category; Indicates defect update location map; Indicates a fully connected output header; represents the convolution output head; Represents the feature map of the fused image output by the last layer of the classification network;

[0068] S5.3: The defect update location is converted according to the process of generating the corresponding defect area boundary box to obtain an updated defect area boundary; the value of the updated defect area boundary includes the horizontal coordinate of the upper left corner x 2 and the vertical axis y 2. Bounding box width w 2 and height h 2.

[0069] In this embodiment, the loss function calculation formula is:

[0070] ;

[0071] ;

[0072] ;

[0073] in, represents the loss function; represents the cross entropy loss function; Indicates k Category prediction results for surface images; Indicates k The category labels of the surface images; Indicates k The second category prediction result of the surface image; represents the first weight; represents the second weight; Indicates the starting point of iteration; Represents the number of training iterations. When the number of training iterations is greater than the iteration starting point, retraining is performed based on the second category prediction result. Otherwise, preliminary training is performed based on the category prediction result.

[0074] In this example, the aircraft engine blade surface defect detection dataset ABSDD is used, which contains a total of 2400 images, of which 1000 are defective images and 1400 are non-defective images. The dataset covers four typical defect types: holes, scratches, pits, and burns. In this example, 60% of the images are randomly selected for training, 20% for verification, and 20% for testing.

[0075] Until the loss function converges, a trained classification network is obtained, and the trained classification network is used to output category prediction and defect location results.

[0076] S6: Input the surface image and the updated defect area boundary box into a visual segmentation model to obtain a defect segmentation result.

[0077] In this embodiment, the visual segmentation model is a visual big model (SAM, Segment Anything Model).

[0078] Further, the surface image is input into the visual large model, and the horizontal coordinate of the upper left corner of the updated defect area boundary is x 2 and the vertical axis y 2. Bounding box width w 2 and height h 2 is input as a prompt input to the visual large model, and a mask image is output, which is the defect segmentation result.

[0079] The aircraft engine blade defect detection method based on weakly supervised learning provided in this embodiment has the following beneficial effects:

[0080] 1. This method uses the candidate areas generated by the initial positioning map and, guided by the clustering analysis results, can optimize the problems of false detection and missed detection in the original positioning map and achieve more accurate surface defect detection of aircraft engine blades.

[0081] 2. This method does not require a large amount of pixel annotation data. It only uses weakly labeled data (image-level labels) to detect surface defects of aircraft engine blades, which greatly reduces the cost and time of data annotation and can better handle situations where the data volume is large but the annotation is difficult.

[0082] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0083] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be construed as limiting the scope of the patent application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent application shall be subject to the attached claims.

Claims

1. A method for detecting defects in aircraft engine blades based on weakly supervised learning, characterized in that: include: S1: Obtain the surface image of the aircraft engine turbine blade and label it according to its category; S2: Based on the surface image and its corresponding category label, a preliminary defect location map is obtained using a classification network, and a corresponding defect area boundary box is generated; The classification network uses the ResNet-50 network as the backbone network; The process of obtaining a preliminary defect location map includes: Step 1: Input the surface image and its corresponding category label into the ResNet-50 network; Step 2: Adaptively average the output of the last layer of the ResNet-50 network, pass the feature map after the operation through the fully connected output head to output the category prediction result, and pass the feature map after the operation through the convolution output head to output the preliminary defect location map; the category label and the category prediction result are used to calculate the loss function to train the classification network; The process of generating the corresponding defect area bounding box includes: Step 1: Perform bicubic interpolation on the preliminary defect location map; Step 2: Normalize the interpolated positioning map; Step 3: Binarize the normalized positioning map by combining the binary threshold method and the OTSU algorithm to obtain a binary image; Step 4: Calculate the range of the outer contour of any connected area in the binary image to obtain the minimum bounding box of each connected area. The minimum bounding box is the defect area bounding box. The value of the minimum bounding box includes the horizontal coordinate of the upper left corner. x 1 and the vertical axis y 1. The width of the bounding box w 1 and height h 1; S3: taking blocks from the surface image and the preliminary defect location map based on the defect area boundary box, constructing block pairs, and performing cluster analysis on the block pairs; S3.1: For any of the defect area boundary boxes, the local mapping of the area within the box corresponding to the preliminary defect positioning map and the surface image is cropped to obtain a positioning block and an image block respectively; the pixel coordinates of the upper left corner of the cropped area are ( x 1, y 1), the pixel coordinates of its lower right corner are ( x 1 +w 1, y 1 +h 1); S3.2: resizing the sizes of the positioning block and the image block to 1 / 4 of the surface image by a bilinear interpolation algorithm, and combining the positioning block and the image block after the resizing in pairs to obtain a block pair, wherein the block pair is represented as: <image block, positioning block>, and recording the index value of the surface image corresponding to the image block and the area mapped in the surface image; S3.3: stacking the image block pairs according to the number of image blocks to form a tensor; S3.4: Inputting the tensor into a small convolutional neural network for feature extraction to obtain extracted features; the small convolutional neural network includes a structure of four groups, namely, a convolution layer, a normalization layer, and a pooling layer; S3.5: taking an average of the extracted features according to the number of channels of the surface image and eliminating the number of channels to obtain a compressed feature representation of the tile pair; S3.6: Select from the compressed feature representation a feature from the same surface image or defect preliminary location map according to the recorded index value. N compressed feature vectors; N Indicates the number of image blocks; S3.7: K-Means clustering algorithm is used based on N The compressed feature vectors are used to perform cluster analysis on the tile pairs and the tile pairs are clustered into two categories; S4: selecting a cluster of vectors from the clustering results, and adaptively suppressing the region mapped by the selected vectors on the preliminary defect location map and the surface image; S4.1: randomly selecting a cluster of vectors from the clustering results, and using the vectors as inhibitory feedback; S4.2: The regions of the suppression feedback mapping on the defect preliminary location map and the surface image are taken out, and suppression processing is performed respectively; the suppression formula is: ； ； ； in, Preliminary defect localization diagram showing the vector mapping in the selected cluster after suppression No. i Regions; Surface image showing the vector map in the selected cluster after suppression No. i Regions; m The vertical coordinate of the region; n The horizontal coordinate of the region; Defect preliminary location diagram C No. i Regions; Representing surface images X No. i Regions; Represents a parameter set Middle Hyperparameters of Represents the adaptive average pooling operation; q represents the predefined number of clusters; s represents the index of the cluster that serves as inhibitory feedback; Indicates the number of regions; W Indicates the width of the preliminary defect location map; H Indicates the height of the preliminary defect location map; S4.3: Re-adding the suppressed area to its original position in the surface image or the preliminary defect location map; S5: fusing the suppressed surface image and the preliminary defect location map, and inputting them into the classification network to obtain an updated defect location map; converting the updated defect area boundary box based on the updated defect location map; S6: Input the surface image and the updated defect area boundary box into a visual segmentation model to obtain a defect segmentation result.

2. The method for detecting defects in aircraft engine blades based on weakly supervised learning according to claim 1, characterized in that: The category annotations include: According to an expert's prior judgment, the category corresponding to the surface image; The surface image is labeled with a corresponding category label.

3. The method for detecting defects in aeroengine blades based on weakly supervised learning according to claim 1, characterized in that S5 include: S5.1: Fuse the suppressed surface image and the preliminary defect location map. The fusion formula is: ； in, represents the fused image; It represents the preliminary location map of the defect after suppression; represents the surface image after suppression; S5.2: input the fused image into the classification network, output the second category prediction result through the fully connected output head, and output the defect update location map through the convolution output head; the category label and the second category prediction result are used to calculate the loss function to train the classification network; S5.3: The defect update location is converted according to the process of generating the corresponding defect area boundary box to obtain an updated defect area boundary; the value of the updated defect area boundary includes the horizontal coordinate of the upper left corner x 2 and the vertical axis y 2. Bounding box width w 2 and height h 2.

4. The method for detecting defects in aeroengine blades based on weakly supervised learning according to claim 3, characterized in that S6 include: The visual segmentation model is a large visual model; The surface image is input into the visual large model, and the horizontal coordinate of the upper left corner of the updated defect area boundary is x 2 and the vertical axis y 2. Bounding box width w 2 and height h 2 is input as a prompt input to the visual large model, and a mask image is output, which is the defect segmentation result.

5. The method for detecting defects in aircraft engine blades based on weakly supervised learning according to claim 3, characterized in that: The loss function calculation formula is: ； ； ； in, represents the loss function; represents the cross entropy loss function; Indicates k Category prediction results for surface images; Indicates k The category labels of the surface images; Indicates k The second category prediction result of the surface image; represents the first weight; represents the second weight; Indicates the starting point of iteration; represents the number of training iterations, when the number of training iterations is greater than the iteration starting point, retraining is performed based on the second category prediction result, otherwise preliminary training is performed based on the category prediction result; Until the loss function converges, a trained classification network is obtained, and the trained classification network is used to output category prediction and defect location results.

Images