A method and system for identifying surface defects of aeroengine blades
Through image feature mapping, segmented pseudo-labels are generated and defect position a priori enhancement features are used. Combined with global feature extraction and classification network, the error and cost problems of surface defect detection of aero engine turbine blades are solved, and efficient and accurate automated detection is achieved.
Patent Information
- Application Number
- CN202311758620.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-12-20
- Publication Date
- 2025-07-18
- Estimated Expiration
- 2043-12-20
AI Technical Summary
The surface defect detection of existing aircraft engine turbine blades relies on manual identification, which has problems such as error, high cost, and easy to miss inspection and miss detection, making it difficult to effectively identify small and weak feature defects.
The image feature mapping relationship is used to generate segmented pseudo-labels, and the feature values are enhanced by defect location priors, combined with global feature extraction and classification network to predict defects, and automated detection is achieved.
The precise classification of small and weak feature defects is achieved, the error and cost of manual detection is avoided, and the detection efficiency and accuracy are improved.
Smart Images
Figure CN117649398B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of defect detection, and in particular to a method and system for identifying surface defects of aeroengine blades. Background Art
[0002] Currently, the detection of surface defects on existing aeroengine turbine blades usually relies on manual identification by professional inspectors with long-term experience in turbine blade inspection. However, the manual inspection and identification method has inevitable limitations: First, some tiny defects may be easily overlooked under human eye observation, so there may be errors in the detection results; and manual inspection is affected by the subjective factors of inspectors, and there are differences in the identification and evaluation of defects by different personnel; at the same time, manual inspection usually requires professionally trained inspectors, resulting in high labor costs; in addition, performing repetitive inspection tasks for a long time may cause fatigue or reduced concentration of inspectors, increasing the possibility of missed or false detections.
[0003] Therefore, the existing method for detecting defects on aeroengine turbine blades by manual inspection has inevitable limitations. How to develop a surface defect technology for aeroengine turbine blades that does not rely on manual labor and strictly detects tiny and weak feature surface defects remains an urgent problem to be solved in the current technical field of surface defect detection of aeroengine turbine blades. Summary of the Invention
[0004] Based on this, it is necessary to provide a method and system for identifying surface defects of aeroengine blades.
[0005] The present invention provides a method for identifying surface defects of aeroengine blades, including:
[0006] S1: Obtain the surface image of the aeroengine turbine blade and the image feature mapping relationship;
[0007] S2: Generate a segmentation pseudo-label of the surface image according to the image feature mapping relationship;
[0008] S3: Obtain the defect position prior provided by the segmentation pseudo-label;
[0009] S4: Enhance the feature values of the defect area based on the defect position prior to obtain an enhanced image;
[0010] S5: Connect the enhanced image with the output of the segmentation network to obtain global features, and input the global features into the classification network to obtain a defect prediction result.
[0011] Preferably, in S1, ResNet18 is used to obtain the image feature mapping relationship.
[0012] Preferably, in S2, generating a segmentation pseudo label of the surface image according to the image feature mapping relationship includes:
[0013] S2.1: Add the acquired image feature mapping relationships in the first dimension to obtain M F ;
[0014] S2.2: For M F Redistribution is performed, and the calculation formula is:
[0015] ;
[0016] Among them, M F Represents the image feature mapping relationship after addition, minM F Indicates M F The minimum value of maxM F Indicates M F The maximum value of
[0017] S2.3: After redistribution, M F Perform bilinear interpolation to obtain the feature map;
[0018] S2.4: Use canny operator to extract edge information of feature map;
[0019] S2.5: adding the extracted edge information along the horizontal coordinate to obtain a one-dimensional array having a length equal to the width of the surface image;
[0020] S2.6: Eliminate the edge information abscissa corresponding to the one-dimensional array whose sum of all pixel values is less than or equal to the first threshold;
[0021] S2.7: Filter the feature map processed by S2.6, and use a mask value of "0" to mask pixels whose pixel values are greater than a second threshold value to obtain a filtered feature map;
[0022] S2.8: Based on the filtered feature map, the grayscale operation provided by the opencv library is performed, and then threshold filtering is performed, that is, the value greater than the third threshold is set to 0, so as to obtain the segmentation pseudo label.
[0023] Preferably, in S3, the defect location prior provided by the segmentation pseudo-label is obtained including defect location information.
[0024] Preferably, in S4, the step of enhancing the characteristic value of the defect area based on the defect position a priori to obtain an enhanced image comprises:
[0025] S4.1: Based on the surface image and the defect location prior, the trained segmentation network obtains an output matrix of 2×H×W; where H×W is the size of the surface image, and 2 indicates that the output has two layers;
[0026] S4.2: Subtract the output of the 0th layer from the output of the 1st layer to obtain a matrix of size H×W;
[0027] S4.3: Use the sigmoid function to map the matrix of size H×W to obtain a defect feature intensity matrix of the same size;
[0028] S4.4: Multiply the defect feature intensity matrix by the surface image to obtain the enhanced image.
[0029] Preferably, the segmentation network includes an input layer, a convolutional layer, and an output layer; the input layer is used to input the original image and the defect location prior; the convolutional layer includes three convolutional layers, and the three convolutional layers have different sizes of convolutional kernels respectively, and the three convolutional layers are used to extract features of different sizes; the extracted features will be sent to the output layer with a convolutional kernel of 1×1 for output; the process of training the segmentation network includes:
[0030] Step 1: Establish a segmentation network and construct a training set, where the training set includes an aviation blade image sample library with annotation information;
[0031] Step 2: Input the training set images and obtain predicted values through the segmentation network;
[0032] Step 3: Calculate the loss by using the predicted values and the labels through a loss function; the loss function is a cross-entropy loss function, and the formula is:
[0033] ;
[0034] where p(x i ) is the true value of the i-th sample, q(x i ) is the predicted value of the i-th sample, and n represents the number of samples;
[0035] Step 4: Backpropagate the cross-entropy loss to optimize the segmentation network.
[0036] Preferably, in S5, the connection of the enhanced image and the output of the segmentation network to obtain global features, and the input of the global features into the classification network to obtain the defect prediction result includes:
[0037] S5.1: Extract the maximum value of the output of the 1st layer in the trained segmentation network;
[0038] S5.2: Use the argmax function to query the maximum value of the 2×H×W matrix output by the trained segmentation network to obtain a segmentation map of size H×W; in the segmentation map, the pixel value of the defective pixel is 1, and the pixel value of the non-defective pixel is 0;
[0039] S5.3: Sum all the values in the segmented image using a summation function to obtain a first numerical value;
[0040] S5.4: Connect the maximum value output by the first layer, the first numerical value, and the enhanced image to obtain global features, and input the global features into the classification network to obtain the defect prediction result.
[0041] Preferably, the segmentation network includes a fully convolutional neural network; the classification network includes a feedforward neural network.
[0042] Preferably, the expression of the sigmoid function is:
[0043] ;
[0044] where S(a) represents the sigmoid function, and a is an element in the matrix of size H×W.
[0045] The present invention also provides a surface defect recognition system for an aeroengine blade, including:
[0046] An acquisition module, configured to acquire the surface image of the aeroengine turbine blade and the image feature mapping relationship;
[0047] A pseudo-label generation module, configured to generate a segmentation pseudo-label of the surface image according to the image feature mapping relationship;
[0048] A defect prior mining module, configured to obtain the defect position prior provided by the segmentation pseudo-label;
[0049] A defect prior mining module, configured to enhance the feature value of the defect area based on the defect position prior to obtain an enhanced image;
[0050] A global information extraction and classification prediction module, configured to connect the enhanced image with the output of the segmentation network to obtain global features, and input the global features into the classification network to obtain the defect prediction result.
[0051] Beneficial effects: The method provided by the present invention generates a segmentation pseudo-label of the original image through the image feature mapping relationship, uses the defect position prior provided by the segmentation pseudo-label to enhance the features of the defect, further extracts the global features of the defect to obtain the defect prediction result, realizes the accurate classification of tiny and weak feature defects, does not require manual detection, saves time and effort, and at the same time avoids detection accuracy problems caused by missed detection, misdetection, and other factors, improving the detection accuracy and efficiency. Description of the Drawings
[0052] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the accompanying drawings required for the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.
[0053] Figure 1 It is a flowchart of the defect detection method for the embodiments of the present application. Specific embodiments
[0054] To make the above objects, features, and advantages of the present application more obvious and understandable, the following will provide a detailed description of the specific embodiments of the present application with reference to the accompanying drawings. Many specific details are set forth in the following description to fully understand the present application. However, the present application can be implemented in many other ways different from those described herein. Those skilled in the art can make similar improvements without departing from the connotation of the present application. Therefore, the present application is not limited by the specific embodiments disclosed below.
[0055] In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one of such features. In the description of the present application, "a plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.
[0056] In the field of aero-engine turbine blade inspection, the commonly used manual inspection method has problems such as high labor costs, easy missed inspection and misjudgment, and other industrial product surface inspection technologies cannot accurately detect overly small and weak feature defects. Therefore, this embodiment provides a method for identifying surface defects of aero-engine blades based on semantic prior-guided defect perception. The method includes:
[0057] S1: Obtain the surface image of the aero-engine turbine blade and the image feature mapping relationship.
[0058] In this embodiment, the surface image of the aero-engine turbine blade used is the ABSDD dataset, which contains a total of 2,400 pictures of aero-engine turbine blades from an actual production line, among which 1,000 are defective images and 1,400 are non-defective images. ResNet18 is used to obtain the mapping relationship between the image and the features, that is, the image feature mapping relationship.
[0059] S2: Generate a segmentation pseudo-label of the surface image according to the image feature mapping relationship.
[0060] Specifically, it includes the following process:
[0061] S2.1: Add the obtained image feature mapping relationships in the first dimension to obtain M F ;
[0062] S2.2: Redistribute M F The calculation formula is:
[0063] ;
[0064] where M F represents the added image feature mapping relationship, minM F represents the minimum value of M F ; maxM F represents the maximum value of M F ;
[0065] S2.3: Perform bilinear interpolation on the redistributed M F to obtain a feature map;
[0066] S2.4: Use the canny operator to extract the edge information of the feature map;
[0067] S2.5: Add the extracted edge information along the abscissa to obtain a one-dimensional array with the length of the surface image width;
[0068] S2.6: Eliminate the abscissas of the edge information corresponding to the one-dimensional arrays whose sum of pixel values is less than or equal to the first threshold;
[0069] S2.7: Filter the feature map processed in S2.6, and use the mask value "0" to cover the pixels whose pixel values are greater than the second threshold to obtain a filtered feature map;
[0070] S2.8: Based on the filtered feature map, perform the grayscale operation provided by the opencv library, and then perform threshold filtering, that is, set the values greater than the third threshold to 0 to obtain the segmentation pseudo-label.
[0071] In this embodiment, the third threshold can be set according to the actual situation.
[0072] Through the above process, the mapping relationship of image features is used to generate a high-precision segmentation label, so as to mine more abundant defect semantic priors than classification labels. And through the filtering operation of the feature map, the interference of the blade edge features on the defect features is avoided, and a high-precision segmentation pseudo-label is accurately generated.
[0073] S3: Obtain the defect position prior provided by the segmentation pseudo-label; the defect position prior includes defect position information.
[0074] S4: Based on the prior of the defect position, enhance the eigenvalue of the defect area to obtain an enhanced image.
[0075] Specifically, the process of obtaining the enhanced image includes:
[0076] S4.1: Based on the surface image and the prior of the defect position, the trained segmentation network obtains a matrix with an output of 2×H×W; where H×W is the size of the surface image, and 2 indicates that there are two layers of output;
[0077] S4.2: Subtract the output of the 0th layer from the output of the 1st layer to obtain a matrix with a size of H×W;
[0078] S4.3: Use the sigmoid function to map the matrix with a size of H×W to obtain a defect feature intensity matrix with the same size;
[0079] The expression of the sigmoid function is:
[0080] ;
[0081] where S(a) represents the sigmoid function, and a is an element in the matrix with a size of H×W;
[0082] S4.4: Multiply the defect feature intensity matrix by the surface image to obtain the enhanced image.
[0083] In this embodiment, the segmentation network is used to perceive defect features of different scales. It has three different sizes of convolutional kernels to extract features of different scales, and sends the extracted features to a 1×1 convolution to reduce the amount of calculation; this segmentation network can be any segmentation network, such as a fully convolutional neural network, etc., and can be replaced according to different scenarios.
[0084] The segmentation network includes an input layer, a convolutional layer, and an output layer; the input layer is used to input the original image and the prior of the defect position; the convolutional layer includes three, and the three convolutional layers respectively have different sizes of convolutional kernels, and the three convolutional layers are used to extract features of different sizes; the extracted features will be sent to the output layer with a convolutional kernel of 1×1 for output. The process of training the segmentation network includes:
[0085] Step 1: Establish a segmentation network and construct a training set, and the training set includes an aviation blade image sample library with annotation information;
[0086] Step 2: Input the training set images and obtain predicted values through the segmentation network;
[0087] Step 3: Calculate the loss by using the loss function with the predicted values and the labels; the loss function is the cross-entropy loss function, and the formula is:
[0088] ;
[0089] where p(x i ) is the true value of the i-th sample, and q(x i ) is the predicted value of the i-th sample, and n represents the number of samples;
[0090] Step 4: Backpropagate the cross-entropy loss to optimize the segmentation network.
[0091] S5: Connect the enhanced image with the output of the segmentation network to obtain global features, and input the global features into the classification network to obtain a defect prediction result.
[0092] Specifically, the process of obtaining the defect prediction result includes:
[0093] S5.1: Extract the maximum value of the output of the first layer in the trained segmentation network;
[0094] S5.2: Use the argmax function to query the maximum value of the 2×H×W matrix output by the trained segmentation network to obtain a segmentation map with a size of H×W; in the segmentation map, the pixel value of the defective pixel is 1, and the pixel value of the non-defective pixel is 0;
[0095] S5.3: Use the summation function to sum all the values in the segmentation map to obtain a first value;
[0096] S5.4: Connect the maximum value of the output of the first layer, the first value, and the enhanced image to obtain global features, and input the global features into the classification network to obtain the defect prediction result.
[0097] In this embodiment, the classification network can be any segmentation network, such as a feedforward neural network, etc., and can be replaced according to different scenarios.
[0098] By adopting the above process, by extracting the probability that the image contains defects and the prior of the defect size in the image, the accuracy of the prediction result of the classification network is further improved, and the global features of the defects are perceived from a more comprehensive perspective, helping the network to achieve high-precision defect classification work.
[0099] This embodiment also provides an aeroengine blade surface defect recognition system, which includes:
[0100] An acquisition module, configured to acquire the surface image of the aeroengine turbine blade and the image feature mapping relationship;
[0101] A pseudo-label generation module, configured to generate a segmentation pseudo-label of the surface image according to the image feature mapping relationship;
[0102] A defect prior mining module, configured to obtain the defect location prior provided by the segmentation pseudo-label;
[0103] A defect enhancement perception module, configured to enhance the feature values of the defect area based on the defect location prior to obtain an enhanced image;
[0104] A global information extraction and classification prediction module, configured to connect the enhanced image with the output of the segmentation network to obtain global features, and input the global features into a classification network to obtain a defect prediction result.
[0105] The method and system for identifying surface defects of aero-engine blades provided in this embodiment have the following beneficial effects:
[0106] 1. This method generates a segmentation pseudo-label of the original image through an image feature mapping relationship, enhances the defect features according to the defect location prior provided by the segmentation pseudo-label, then extracts the global features of the defects, and obtains the final defect prediction result through a classification network, so as to achieve accurate classification of tiny and weak feature defects, and improves the problem that the existing industrial detection methods have a low detection rate for the too small and weak feature defects of aero-turbine blades.
[0107] 2. This method can be independent of manual detection, avoiding the problems of high labor cost, easy missed detection and misdetection in manual detection, and effectively improving the detection efficiency on the premise of ensuring the detection rate.
[0108] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this specification.
[0109] The above embodiments only represent several implementation manners of the present application. The description is relatively specific and detailed, but it cannot be understood as a limitation on the scope of the patent application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.
Claims
1. A method for identifying surface defects of aeroengine blades, characterized in that Including: S1: Obtain the surface image of the aero-engine turbine blade and the image feature mapping relationship; S2: Generate the segmentation pseudo-label of the surface image according to the image feature mapping relationship, including: S2.1: Add the obtained image feature mapping relationships in the first dimension to obtain M F ; S2.2: Redistribute M F using the following calculation formula: ； Among them, M F represents the image feature mapping relationship after addition, and minM F represents the minimum value of M F ; maxM F represents the maximum value of M F ; S2.3: Perform bilinear interpolation on the redistributed M F to obtain a feature map; S2.4: Use the canny operator to extract the edge information of the feature map; S2.5: Add the extracted edge information along the abscissa to obtain a one-dimensional array with the length equal to the width of the surface image; S2.6: Eliminate the abscissa of the edge information corresponding to the one-dimensional array whose sum of pixel values is less than or equal to the first threshold; S2.7: Filter the feature map processed in S2.6, and use the mask value "0" to cover the pixels whose pixel values are greater than the second threshold to obtain the filtered feature map; S2.8: Perform the grayscale operation provided by the opencv library on the filtered feature map, and then perform threshold filtering, that is, set the values greater than the third threshold to 0 to obtain the segmentation pseudo-label; S3: Obtain the defect position prior provided by the segmentation pseudo-label; S4: Enhance the eigenvalue of the defect area based on the defect position prior to obtain the enhanced image, including: S4.1: Based on the surface image and the defect position prior, the trained segmentation network obtains a matrix with an output of 2×H×W; where H×W is the size of the surface image, and 2 indicates that there are two output layers; S4.2: Subtract the output of the first layer from the output of the zero layer to obtain a matrix with a size of H×W; S4.3: Use the sigmoid function to map the matrix with the size of H×W to obtain a defect feature intensity matrix with the same size; S4.4: Multiply the defect feature intensity matrix by the surface image to obtain the enhanced image; S5: Connect the enhanced image with the output of the segmentation network to obtain the global feature, and input the global feature into the classification network to obtain the defect prediction result.
2. The method for identifying surface defects of an aeroengine blade according to claim 1, wherein In S1, ResNet18 is used to obtain the image feature mapping relationship.
3. The method for identifying surface defects of an aeroengine blade according to claim 1, wherein, In S3, the obtaining of the defect position prior provided by the segmentation pseudo-label includes defect position information.
4. The method for identifying surface defects of an aeroengine blade according to claim 1, characterized in that, The segmentation network includes an input layer, a convolutional layer, and an output layer; the input layer is used to input the surface image and the defect position prior; the convolutional layer includes three, and the three convolutional layers have different sizes of convolutional kernels respectively, and the three convolutional layers are used to extract features of different sizes; The extracted features will be sent to the output layer with a convolutional kernel of 1×1 for output; The process of training the segmentation network includes: Step 1: Establish a segmentation network and construct a training set, and the training set includes an aviation blade image sample library with annotation information; Step 2: Input the training set image and obtain the predicted value through the segmentation network; Step 3: Calculate the loss by using the loss function with the predicted value and the label; the loss function is the cross-entropy loss function, and the formula is: ； where p(xi) is the true value of the i-th sample, q(xi) is the predicted value of the i-th sample, and n represents the number of samples; Step 4: Perform backpropagation on the cross-entropy loss to optimize the segmentation network.
5. The method for identifying surface defects of an aeroengine blade according to claim 1, wherein In S5, the enhanced image is connected to the output of the segmentation network to obtain a global feature, and the global feature is input into the classification network to obtain a defect prediction result, including: S5.1: Extract the maximum value of the first layer output in the trained segmentation network; S5.2: Use the argmax function to perform a maximum value query on the 2×H×W matrix output by the trained segmentation network to obtain a segmentation map of size H×W; in the segmentation map, the pixel value of the defective pixel is 1, and the pixel value of the non-defective pixel is 0; S5.3: using a summation function to sum all the values in the segmentation graph to obtain a first value; S5.4: Connect the maximum value outputted by the first layer, the first value and the enhanced image to obtain global features, and input the global features into the classification network to obtain the defect prediction result.
6. The method for identifying surface defects of an aero-engine blade according to claim 5, wherein, The segmentation network includes a fully convolutional neural network; the classification network includes a feedforward neural network.
7. The method for identifying surface defects of an aeroengine blade according to claim 6, wherein The expression of the sigmoid function is: ； Wherein, S(a) represents the sigmoid function, and a is an element in the matrix of size H×W.
8. An aero-engine blade surface defect recognition system, characterized in that, include: An acquisition module, used to acquire a surface image of an aircraft engine turbine blade and an image feature mapping relationship; A pseudo label generation module, used to generate segmentation pseudo labels of the surface image according to the image feature mapping relationship, comprising: S2.1: Add the obtained image feature mapping relationships in the first dimension to obtain M F ; S2.2: Redistribute M F using the following calculation formula: ； Among them, M F represents the image feature mapping relationship after addition, and minM F represents the minimum value of M F ; maxM F represents the maximum value of M F ; S2.3: Perform bilinear interpolation on the redistributed M F to obtain a feature map; S2.4: Use canny operator to extract edge information of feature map; S2.5: adding the extracted edge information along the horizontal coordinate to obtain a one-dimensional array having a length equal to the width of the surface image; S2.6: Eliminate the edge information abscissa corresponding to the one-dimensional array whose sum of all pixel values is less than or equal to the first threshold; S2.7: Filter the feature map processed by S2.6, and use a mask value of "0" to mask pixels whose pixel values are greater than a second threshold value to obtain a filtered feature map; S2.8: Based on the filtered feature map, a grayscale operation provided by the opencv library is performed, and then a threshold filtering is performed, that is, the value greater than the third threshold is set to 0, so as to obtain the segmentation pseudo label; Defect prior mining module, used to obtain defect location priors provided by segmentation pseudo-labels; The defect enhancement perception module is used to enhance the characteristic value of the defect area based on the defect position a priori to obtain an enhanced image, including: S4.1: Based on the surface image and the defect location prior, the trained segmentation network obtains an output matrix of 2×H×W; where H×W is the size of the surface image, and 2 indicates that the output has two layers; S4.2: Subtract the output of layer 0 from the output of layer 1 to obtain a matrix of size H×W; S4.3: Mapping the matrix of size H×W using a sigmoid function to obtain a defect feature intensity matrix of the same size as the matrix; S4.4: multiplying the defect feature intensity matrix by the surface image to obtain the enhanced image; The global information extraction and classification prediction module is used to connect the enhanced image with the output of the segmentation network to obtain global features, and input the global features into the classification network to obtain defect prediction results.