Engine blade surface crack detection method based on U2-Net and Canny operators

Abstract

The invention relates to an engine blade surface crack detection method based on U2-Net and Canny operators, belongs to the technical field of image segmentation and defect detection, and solves the problems that in the prior art, engine blade surface crack detection is low in efficiency and high in cost, depends on subjective personal judgment and has potential safety hazards. Comprising the following steps: S1, collecting an aero-engine blade image, and preprocessing the aero-engine blade image to obtain a test set; s2, a road crack data set CFD is obtained, and a training set and a verification set are obtained through processing; s3, constructing a network model, and performing preliminary training by using the training set; s4, setting evaluation indexes, selecting algorithm parameters, and establishing a loss function; s5, training, testing and optimizing the network model after preliminary training to obtain a final network model; and S6, detecting and evaluating the surface image of the to-be-detected engine blade by using the final network model to obtain a crack detection result of the aero-engine blade.

Description

Technical Field

[0001] The present invention relates to the technical field of image segmentation and defect detection, and particularly to a method for detecting cracks on the surface of engine blades based on U2-Net and Canny operator. Background Art

[0002] As one of the key components of an aircraft, an aeroengine plays a crucial role in ensuring flight safety and performance. Among them, the engine blade, as a key component for propelling air flow and converting energy, its integrity has a significant impact on the engine performance and service life. Due to the influence of extreme working conditions such as high temperature and high pressure, foreign object damage and erosion damage often occur to the engine blade, so cracks are likely to appear on the blade surface, and these tiny cracks may expand during flight and lead to serious accidents. In order to reduce the accident risk and improve the operation safety of the engine, it is particularly important to detect and accurately identify the cracks on the engine blade in a timely manner.

[0003] In the prior art, there are mainly two types of methods for detecting cracks on the surface of engine blades: traditional defect detection algorithms and detection methods based on deep learning.

[0004] Traditional methods for detecting cracks on engine blades often use ultrasonic flaw detection, magnetic particle flaw detection, X-ray flaw detection, etc. However, these methods are only applicable to detecting internal damage of the blade and not applicable to surface cracks; the detection of cracks on the surface of engine blades is mostly completed manually, which has low efficiency, high cost, relies on subjective personal judgment, and has potential safety hazards. Image processing and deep learning technologies have achieved excellent performance in the field of crack detection and have also been widely applied in the detection of cracks on the surface of engine blades. Traditional image processing methods have a relatively solid theoretical foundation, are easy to understand and implement, do not require complex model frameworks or a large amount of data support, and are suitable for small-scale segmentation of cracks on the surface of engine blades. However, they usually face challenges such as the need for manual design of feature extraction methods, poor adaptability in complex scenarios, and manual adjustment of parameters, and the detection effect is poor in complex backgrounds.

[0005] Deep learning-based methods can learn crack features from a large amount of data, which helps to capture effective information in images. Among them, semantic segmentation can achieve pixel-level classification of images and obtain the specific location and morphology of cracks on engine blades. However, due to the deepening of the network model, continuous convolution and pooling operations are likely to filter out small defects in the image and miss them when the cracks are small. To overcome this shortcoming, an attention mechanism is introduced into the neural network to improve the accuracy of crack detection. However, using deep learning algorithms for crack detection requires a large amount of labeled data and a large amount of computing resources for training. For the task of engine blade crack detection, the engine interior is highly confidential and the blades are located deep inside, making it very difficult to obtain a large amount of high-quality labeled data. Moreover, the complex structure of engine blades and the normal state in most cases result in a low proportion of crack samples in the overall data. In the case of limited images, the effect obtained by training the network using deep learning methods is not ideal. Therefore, in the prior art, it has not been possible to accurately and quickly detect cracks on the surface of engine blades with less data.

[0006] Currently, the main difficulties in detecting cracks on the surface of aero-engine blades in the industrial field with small samples are as follows: First, it is difficult to collect images of engine blades. The manufacturing technology of engine blades is excellent, with fewer defects and located deep inside the surface. It is very difficult to obtain a large amount of high-quality images, and the proportion of crack images is very low. Second, the cracks on the surface of engine blades have various shapes and are all very small, while the background of engine blades is complex, resulting in an unbalanced foreground and background situation, and they are easily overlooked.

[0007] Under the above difficulties, traditional detection methods are vulnerable to noise and cannot accurately segment cracks on the surface of engine blades. Although semantic segmentation algorithms are helpful to a certain extent in improving accuracy, their high requirements for data sets and computational volume limit their development in the industrial small-sample field, and they lack interpretability. In such a situation, there is an urgent need for an improved method that can accurately detect cracks on the surface of aero-engine blades in the industrial field with small samples. Summary of the Invention

[0008] In view of the above problems, the present invention provides a method for detecting cracks on the surface of engine blades based on U2-Net and Canny operator. This method has a small computational volume and good accuracy, and realizes the detection of cracks on the surface of engine blades in the industrial field with small samples.

[0009] The present invention provides a method for detecting cracks on the surface of engine blades based on U2-Net and Canny operator, including the following steps:

[0010] Step S1. Collect images of aero-engine blades and preprocess them to obtain a crack data set as the test set;

[0011] Step S2. Obtain the road crack dataset CFD, and perform selection and division to obtain a training set and a validation set;

[0012] Step S3. Construct a network model for detecting cracks on the surface of aero-engine blades, including a first module, a second module, and a superimposing module, and use the training set for preliminary training to obtain a preliminarily trained network model, where the first module is CA-U2Net and the second module is the Canny operator WiCanny integrated with a Wiener filter;

[0013] Step S4. Set evaluation metrics for training the network model, select algorithm parameters for the network model, and establish a loss function for optimizing the training process of the network model;

[0014] Step S5. Train, test, and optimize the preliminarily trained network model to obtain a final network model;

[0015] Step S6. Use the final network model to detect and evaluate the surface image of the engine blade to be detected, obtain the detection result of the aero-engine blade crack, and save and output the detection result of the aero-engine blade crack to the maintenance team for maintenance decision-making.

[0016] Optionally, step S1 specifically includes the following steps:

[0017] Step S1-1. Collect aero-engine blade images, which are images with cracks on the surface of aero-engine blades;

[0018] Step S1-2. Preprocess the collected aero-engine blade images to obtain a crack dataset as a test set. The preprocessing includes median filtering and histogram equalization.

[0019] Optionally, step S2 specifically includes the following steps:

[0020] Step S2-1. Obtain the road crack dataset CFD, and perform selection to obtain a selected road crack dataset. The selection includes selecting pictures with semantic information similar to the cracks on the surface of aero-engine blades from the road crack dataset CFD, and further selecting the undamaged images and corresponding label images that have been labeled from them as the selected road crack dataset;

[0021] Step S2-2. Divide the selected road crack dataset according to a certain proportion to obtain a training set and a validation set.

[0022] Optionally, step S3 specifically includes the following steps:

[0023] Step S3-1. Construct the first module and the second module of the network model, where the first module is CA-U2Net and the second module is the Canny operator WiCanny integrated with a Wiener filter. The first module CA-U2Net adds a coordinate attention mechanism to the semantic segmentation network U2-Net to optimize the network performance;

[0024] Step S3-2. Construct the superimposing module of the network model, which is used to superimpose the results obtained by the first module and the second module to obtain a network model including the first module, the second module and the superimposing module;

[0025] Step S3-3. Use the training set obtained in Step S2 to preliminarily train the network model to obtain a preliminarily trained network model.

[0026] Optionally, Step S4 specifically includes the following steps:

[0027] Step S4-1. Set the evaluation metrics of the network model, including the mean intersection over union (MIoU), accuracy, precision, recall, and F1-score as evaluation metrics to evaluate the crack segmentation effect;

[0028] Step S4-2. Select the algorithm parameters of the network model, including selecting the running framework, processor, optimizer, batch size, initial learning rate, minimum learning rate, number of training epochs, and double-threshold parameters of the network model;

[0029] Step S4-3. Establish the loss function of the first module CA-U2Net of the network model.

[0030] Optionally, the evaluation metrics in Step S4-1 include:

[0031] The mean intersection over union (MIoU) is the average of the intersection over union of each class in the datasets of the true value and the predicted value. The calculation formula is as follows:

[0032]

[0033] where i represents the true value, j represents the predicted value, p ij represents the number of samples with the true label i and the predicted label j, p ii represents the number of samples with the true label i and the predicted label i, p ji represents the number of samples with the true label j and the predicted label i, k + 1 represents the number of classes. In this formula, the number of classes k = 1. The value range of the mean intersection over union (MIoU) is [0, 1]. The closer the value is to 1, the higher the degree of coincidence between the predicted result and the true result, and the better the segmentation effect;

[0034] The precision is the pixel-level precision between the predicted segmentation result and the true segmentation result, and the calculation formula is as follows:

[0035]

[0036] Among them, TP is the number of cracks in the engine blade correctly detected, and FP is the number of non-cracks regarded as cracks;

[0037] The recall represents the proportion of correct predictions by the network model among the results where the true value is the positive class, and the calculation formula is as follows:

[0038]

[0039] Among them, FN is the number of cracks regarded as non-cracks;

[0040] The accuracy is the pixel-level accuracy between the predicted segmentation result and the true segmentation result, and the calculation formula is as follows:

[0041]

[0042] Among them, TN is the number of non-engine blade cracks correctly detected;

[0043] The F1-score is based on the precision and recall of the network model, and comprehensively evaluates the performance of the network model. The calculation formula is as follows:

[0044]

[0045] Optionally, step S5 specifically includes the following steps:

[0046] Step S5-1. Use the test set obtained in step S2 and the algorithm parameters selected in step S4 to train the preliminarily trained network model, and monitor the loss function and evaluation metrics during the training process to ensure that the network model gradually converges and reaches the expected performance, and obtain the trained network model;

[0047] Step S5-2. Test and evaluate the network model. Use the test set obtained in step S1 to test the trained network model to obtain the test results, and comprehensively evaluate the performance of the network model using the preset evaluation metrics;

[0048] Step S5-3. Evaluate and analyze the test results, and judge whether the performance of the current network model meets the preset requirements. If it meets the preset requirements, save the current network model as the final network model and end this step. Otherwise, return to step S4, adjust the algorithm parameters, and re-train and test.

[0049] Optionally, step S6 specifically includes the following steps:

[0050] Step S6-1: Obtain the aero-engine blade image to be detected and perform preprocessing to obtain the dataset to be detected. The preprocessing includes median filtering and histogram equalization on the surface image of the aero-engine blade;

[0051] Step S6-2: Input the dataset to be detected obtained in step S6-1 into the final network model for detection to obtain the detection result;

[0052] Step S6-3: Comprehensively evaluate the detection result obtained in step S6-2 using a preset evaluation index, perform visual analysis on the detection result, obtain the detection result of the aero-engine blade crack, and save and output the detection result of the aero-engine blade crack to the maintenance team for maintenance decision-making.

[0053] Compared with the prior art, the engine blade surface crack detection method based on U2-Net and Canny operator provided by the embodiment of the present invention has at least the following beneficial effects:

[0054] (1) The constructed network module (FCU2C) for detecting cracks on the surface of aero-engine blades has the characteristics of simple network, small computation amount, high accuracy, high real-time performance, etc., and realizes the accurate segmentation of cracks on the surface of small-sample engine blades. Traditional image processing methods have a relatively solid theoretical basis, are easy to understand and implement, do not require complex model frameworks or a large amount of data support, and are suitable for the segmentation of cracks on the surface of small-scale engine blades. However, they usually face challenges such as the need for manual design of feature extraction methods, poor adaptability in complex scenarios, and manual adjustment of parameters, and the detection effect is poor in complex backgrounds. Deep learning-based methods can learn crack features from a large amount of data, which helps to capture effective information in the image. Among them, semantic segmentation can achieve pixel-level classification of images and obtain the specific location and shape of engine blade cracks. However, due to the deepening of the network model, continuous convolution and pooling operations are likely to filter out small defects in the image, and cracks will be missed when they are thin; effectively overcome the shortcomings of traditional image processing and semantic segmentation-based methods, and accurately and quickly detect cracks on the surface of engine blades in the case of small samples.

[0055] (2) The present invention adds a coordinate attention mechanism to the U2-Net network to improve the network's attention to cracks; adopts a transfer learning method to solve the problem of insufficient data. This part serves as the feedforward of the system to enhance the stability and robustness of the system; adds a Wiener filter to the Canny operator to reduce the sensitivity to noise during crack detection and make the cracks clearer, which helps the detection of cracks on the surface of engine blades. Description of the Drawings

[0056] The accompanying drawings are only for the purpose of showing specific embodiments and are not considered as limiting the present invention.

[0057] Figure 1 It is the overall structural diagram of the engine blade surface crack detection method based on U2-Net and Canny operator provided according to an embodiment of the present invention.

[0058] Figure 2 It is the network model diagram of the first module CA-U2Net constructed according to the engine blade surface crack detection method based on U2-Net and Canny operator provided according to an embodiment of the present invention.

[0059] Figure 3 It is the specific module structural diagram of the first module CA-U2Net constructed according to the engine blade surface crack detection method based on U2-Net and Canny operator provided according to an embodiment of the present invention.

[0060] and Figure 4 It is the specific module structural diagram of the first module CA-U2Net constructed according to the engine blade surface crack detection method based on U2-Net and Canny operator provided according to an embodiment of the present invention.

[0061] Figure 5 It is the system monitoring flowchart of the engine blade surface crack detection method based on U2-Net and Canny operator provided according to an embodiment of the present invention.

[0062] Figure 6 It is the visualization result diagram obtained by applying an example of the engine blade surface crack detection method based on U2-Net and Canny operator provided according to an embodiment of the present invention. Specific Embodiments

[0063] In order to be able to more clearly understand the above-mentioned objects, features and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, without conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other.

[0064] Many specific details are set forth in the following description in order to fully understand the present invention, but the present invention can also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited by the specific embodiments disclosed below.

[0065] The following will describe in detail the engine blade surface crack detection method based on U2-Net and Canny operator provided according to an embodiment of the present invention with reference to the accompanying drawings.

[0066] Embodiments of the present invention provide a method for detecting cracks on the surface of engine blades based on U2-Net and the Canny operator, constructing and optimizing a network model (FCU2C) for detecting cracks on the surface of aeroengine blades, and achieving high-precision detection of cracks on the surface of aeroengine blades. This method aims to achieve high-precision detection of cracks on the surface of aeroengine blades under small-sample conditions in the industrial field and solve the problem of the lack of effective detection techniques in this field. By combining the powerful semantic segmentation ability of U2-Net with the edge detection advantage of the Canny operator, this method improves the accuracy of crack detection.

[0067] The method for detecting cracks on the surface of engine blades based on U2-Net and the Canny operator provided by the embodiments of the present invention significantly improves the generalization ability of the model through transfer learning, and selects the road crack dataset CFD, which has semantic information similar to that of the crack images on the surface of aeroengine blades, as the training data. Specifically, first, the crack dataset is obtained by preprocessing the collected crack images on the surface of aeroengine blades to optimize the data quality and improve the model training effect. Then, the selected crack dataset is divided into a training set and a validation set according to a ratio of 9:1 for network training. During the training process, the parameters of the model are saved and used for subsequent testing. In the testing stage, by evaluating the performance of the model in the test set, various evaluation indexes are obtained, and the segmentation results of the test set are visually analyzed. This method has the advantages of not requiring a large amount of data sets, high detection accuracy, and strong real-time performance, and can effectively detect cracks on the surface of aeroengine blades under small-sample conditions. By introducing transfer learning and combining the advanced U2-Net semantic segmentation network with the Canny edge detection operator, this method effectively improves the accuracy of crack detection.

[0068] As Figures 1 to 5 shown, the method for detecting cracks on the surface of engine blades based on U2-Net and the Canny operator provided by the embodiments of the present invention includes the following steps.

[0069] Step S1. Collect aeroengine blade images and preprocess them to obtain a crack dataset as the test set. Step S1 specifically includes the following steps.

[0070] Step S1-1. Collect aeroengine blade images. The aeroengine blade images cover various forms and sizes of cracks on the surface of aeroengine blades to ensure that the constructed model can cope with the diversity in actual applications. The sources of the aeroengine blade images should include the surface images of aeroengine blades with surface cracks in the actual operating environment.

[0071] Step S1-2. Preprocess the collected aero-engine blade images to obtain a crack dataset as the test set. This preprocessing includes using median filtering to process the images, removing noise, and improving data quality; and performing histogram equalization to enhance the image contrast, ensuring that the displayed crack features are more obvious and preparing for the subsequent steps.

[0072] Step S2. Obtain the road crack dataset CFD, and perform selection and division to obtain the training set and the validation set. Select a road crack dataset (the publicly available road crack dataset CFD can be selected) with a crack morphology similar to that of the aero-engine blade surface to utilize transfer learning technology to improve the generalization ability of the model. Step S2 specifically includes the following steps.

[0073] Step S2-1. Obtain the road crack dataset CFD and perform selection to obtain the selected road crack dataset. Since the aero-engine blade crack image data is scarce, select a road crack dataset CFD with a crack morphology similar to that of the aero-engine blade surface for training and validation. For example, the road crack dataset CFD can be set to include 118 images with a size of 480×320 pixels. Each image has a manually marked crack contour, and the images contain noise such as shadows and water stains, and the crack morphologies are diverse, which is semantically similar to the aero-engine blade surface cracks. Further, 110 labeled undamaged images and corresponding label images can be selected from them as the selected road crack dataset, which helps the implementation of transfer learning.

[0074] Step S2-2. Divide the selected road crack dataset to obtain the training set and the validation set. Divide the selected road crack dataset obtained in Step S2-1 into the training set and the validation set according to a ratio of 9:1 to ensure that the constructed model can fully learn the crack features during the training and validation processes.

[0075] Step S3. Build a network model (FCU2C) for detecting cracks on the aero-engine blade surface and perform preliminary training to obtain a preliminarily trained network model. Build a network model for detecting cracks on the aero-engine blade surface and perform preliminary training on this network model in the following steps to verify the effectiveness of the constructed network model, and obtain a preliminarily trained network model for detecting cracks on the aero-engine blade surface. Step S3 specifically includes the following steps.

[0076] Step S3-1. Construct the first module and the second module, where the first module is CA-U2Net (U2-Net network based on the attention mechanism), and the second module is the Canny operator WiCanny integrated with the Wiener filter.

[0077] In this embodiment, in order to detect various shaped cracks on the surface of small-sample engine blades, a U2-Net network and a Canny operator WiCanny are used to extract texture information and edge information from the aero-engine blade images, and an open-loop feedforward end-to-end image segmentation model is designed. The network model includes a first module and a second module. Among them, the first module is a U2-Net network based on the attention mechanism (abbreviated as CA-U2Net), which adds a coordinate attention mechanism to optimize the network performance on the basis of the semantic segmentation network U2-Net; the second module is the Canny operator WiCanny integrated with the Wiener filter, which is used to further accurately segment the cracks of the aero-engine blades. The Wiener filter included in the Canny operator WiCanny integrated with the Wiener filter can effectively reduce the noise in the aero-engine blade images, restore the blurred images, improve the image quality, and further improve the detection accuracy of the Canny operator.

[0078] The first module CA-U2Net consists of a feature extraction structure and a CA-RSU (Attention-Residual U-shaped module). Among them, the CA-RSU is a U-shaped structure formed by stacking RSU modules (ReSidual U-blocks structure) containing coordinate attention mechanisms. The CA-RSU can obtain more crack image information at different scales through the U-shaped structure and the coordinate attention module to better detect the cracks of the engine blades. Although CA-U2Net can perform a preliminary segmentation of the aero-engine blade cracks in the WiCanny image, there are pooling operations and the like in the CA-RSU, and edge information is likely to be missing during connection. Therefore, the aero-engine blade cracks still cannot be completely detected directly using this network. In order to further segment the aero-engine blade cracks in the aero-engine blade images, the second module uses the Canny operator WiCanny integrated with the Wiener filter. The Canny operator WiCanny integrated with the Wiener filter consists of a Wiener filter and a Canny operator. The Wiener filter effectively removes the noise in the aero-engine blade images, enhances the edge information, makes the image clearer and smoother, and improves the segmentation effect of the engine blade cracks in the aero-engine blade images.

[0079] Step S3-2. Construct a superimposing module for superimposing the results obtained by the first module and the second module to obtain a network model for detecting the cracks on the surface of the aero-engine blades. The obtained network model includes a first module, a second module, and a superimposing module.

[0080] During the execution of the preliminary segmentation process by the first module CA-U2Net, when detecting cracks on the surface of engine blades, small and relatively thin cracks are ignored. The second module uses the Canny operator WiCanny that integrates the Wiener filter to perform further segmentation on the segmentation result of the first module. However, due to the influence of the background, there will still be some slight noise. Therefore, it is hoped to integrate the results of the two to achieve a noise-free and accurate segmentation result. Thus, in the network model, a superimposition module is also constructed. When detecting through this network model, the segmentation results obtained by the first module and the second module are superimposed pixel by pixel through this superimposition module. The expression is as follows:

[0081] result(x,y) = U(x,y) + UC(x,y)

[0082] Among them, result(x,y) represents the result image after superimposition by the superimposition module, U(x,y) represents the result after segmenting the crack of the aero-engine blade image by the first module CA-U2Net, UC(x,y) represents the result after segmenting U(x,y) by the second module using the Canny operator WiCanny that integrates the Wiener filter, and (x,y) represents the pixel value in the aero-engine blade image.

[0083] Step S3-3. Use the training set obtained in Step S2 to conduct preliminary training on the constructed network model to verify the effectiveness of the network model, and adjust the parameters of the network model according to the training results obtained from the preliminary training to ensure that the structure of the network model can adapt to the actual data, and obtain the network model after preliminary training. The parameters of this adjusted network model can include parameters such as the learning rate and batch size in the network model.

[0084] Step S4. Set the evaluation metrics for training the network model, select the algorithm parameters of the network model, and establish a loss function for optimizing the training process of the network model. First, determine the evaluation metrics of the network model to evaluate the performance of the network model in the crack detection task; then select appropriate algorithm parameters, including the learning rate, batch size, number of training epochs, etc., and design an appropriate loss function (such as cross-entropy loss or a custom loss function) to optimize the training process of the model. Finally, perform hyperparameter tuning to find the best algorithm parameter configuration.

[0085] This Step S4 specifically includes the following steps.

[0086] Step S4-1. Set the evaluation metrics for the network model. In the engine blade crack segmentation experiment of this embodiment, in order to more comprehensively evaluate the model, the mean intersection over union (MIoU), accuracy, precision, recall, and F1-score are used as evaluation metrics to evaluate the crack segmentation effect.

[0087] The intersection over union (IoU) refers to the ratio of the intersection to the union of the two sets of the ground truth and the predicted segmentation, and its calculation formula is as follows:

[0088]

[0089] Among them, TP is the number of engine blade cracks correctly detected; TN is the number of non-engine blade cracks correctly detected; FP is the number of non-cracks regarded as cracks; FN is the number of cracks regarded as non-cracks.

[0090] The mean intersection over union (MIoU) is the average of the intersection over union of each class in the datasets of the two sets of the ground truth and the predicted segmentation, and the calculation formula is as follows:

[0091]

[0092] Among them, i represents the ground truth, j represents the predicted value, p ij represents the number of samples with the true label i and the predicted label j, p ii represents the number of samples with the true label i and the predicted label i, p ji represents the number of samples with the true label j and the predicted label i, k + 1 represents the number of classes (including the background). In the engine blade crack segmentation task of this embodiment, only cracks need to be segmented, so the number of classes k = 1. The value range of the mean intersection over union (MIoU) is [0, 1], and the closer the value is to 1, the higher the degree of coincidence between the predicted result and the true result, and the better the segmentation effect.

[0093] The precision is the pixel-level precision between the predicted segmentation result and the true segmentation result, and the calculation formula is as follows:

[0094]

[0095] The recall represents the proportion of correct predictions by the network model among the results where the ground truth is the positive class, and the calculation formula is as follows:

[0096]

[0097] The accuracy is the pixel-level accuracy between the predicted segmentation result and the ground truth segmentation result, and the calculation formula is as follows:

[0098]

[0099] The F1-score takes into account the precision and recall of the network model and can more comprehensively evaluate the performance of the network model. The calculation formula is as follows:

[0100]

[0101] Evaluate the performance of the network model in the crack detection task through the above indicators to ensure the reliability of the detection results.

[0102] Step S4-2. Selection of algorithm parameters of the network model, including selecting the running framework, processor, optimizer, batch size, initial learning rate, minimum learning rate, number of training epochs, double-threshold parameters, etc. For example, the running framework can use Python3.11, Pytorch2.1.1 and cuda12.1 frameworks. The processor can use an Intel Core i7-14700K CPU@5.60GHz processor and an NVIDIA GeForce RTX 4070ti GPU. During the experiment, for example, the CA-U2Net model uses Adam (adaptive moment estimation) to update the training parameters, the batch size is set to 2, the initial learning rate is 1e-4, the minimum learning rate is 1e-6, and a total of 300 training epochs are trained; when using the WiCanny operator of the fusion Wiener filter for engine blade crack image segmentation, the double-threshold parameters can be selected as [0.1, 0.29].

[0103] Step S4-3. Establish the loss function of the first module CA-U2Net.

[0104] The calculation formula of the network loss function L of the first module CA-U2Net is as follows:

[0105]

[0106] Among them, m represents the number of output saliency maps of the first module, M represents the final number of output saliency maps of the first module. In CA-U2Net, M = 6. This loss function can be regarded as two parts, where is the loss of the output saliency map of the first module l fuseis the loss of the final fused output saliency map of the first module. and ω fuse are the weights of each loss term and l fuse respectively. For each loss term in the above two parts of the loss function, the standard binary cross-entropy can be used to calculate:

[0107]

[0108] where (r, c) are the pixel coordinates, (H, W) are the height and width of the image size, P G(r,c) represents the image pixel value, and P S(r,c) represents the predicted saliency probability map. The training process attempts to minimize the overall loss. During the testing process, the loss l fuse of the final fused output saliency map is selected as the final saliency map.

[0109] Step S5. Train, test, and optimize the preliminarily trained network model to obtain the final network model. Use the training set obtained in Step S2 and the algorithm parameters selected in Step S4 to train the network model built in Step S3 to obtain the trained network model. Test the trained network model on the test set obtained in Step S1, and comprehensively evaluate the performance of the network model using the evaluation metrics set in Step S4. Perform visual analysis on the test results obtained from the test to intuitively understand the performance of the network model in the actual crack detection task. If the effect of the network model does not meet the expectations, return to Step S4, adjust the algorithm parameters, and perform retraining and testing until satisfactory detection performance is achieved.

[0110] The specific steps of Step S5 are as follows.

[0111] Step S5-1. Network model training: Use the test set obtained by dividing in Step S2 and the algorithm parameters selected in Step S4 to train the preliminarily trained network model obtained in Step S3, and monitor the loss function and evaluation metrics during the training process to ensure that the network model gradually converges and reaches the expected performance, obtaining the trained network model.

[0112] Step S5-2. Network model testing and evaluation: Use the test set of aero-engine blade images obtained in Step S1 to test the trained network model to obtain the test results, and comprehensively evaluate the performance of the network model using the preset evaluation metrics. Perform visual analysis on the test results to intuitively understand the performance of the network model in the aero-engine blade surface crack detection task.

[0113] Step S5-3. Network model optimization and retraining. According to the evaluation and analysis of the test results, determine whether the performance of the current network model meets the preset requirements. If it meets the preset requirements, save the current network model as the final network model and end this step; otherwise, return to Step S4, adjust the algorithm parameters, and retrain and test again.

[0114] Step S6. Use the final network model to detect and evaluate the surface images of the engine blades to be detected, obtain the crack detection results of the aero-engine blades, and save and output the crack detection results of the aero-engine blades to the maintenance team for maintenance decision-making. Use the final network model obtained in Step S5 to detect cracks in the collected surface images of the engine blades, comprehensively evaluate the crack detection results of the engine blade surface by the evaluation metrics in Step S4, and perform visual analysis on the results.

[0115] This Step S6 specifically includes the following steps.

[0116] Step S6-1: Obtain the surface images of the aero-engine blades to be detected and perform preprocessing to obtain the dataset to be detected. This preprocessing includes using median filtering to process the images to remove noise and improve data quality, and performing histogram equalization.

[0117] Step S6-2: Input the dataset to be detected obtained in Step S6-1 into the final network model for detection to obtain the detection results.

[0118] Step S6-3: Comprehensively evaluate the detection results obtained in Step S6-2 using the preset evaluation metrics. Perform visual analysis on the detection results to obtain the crack detection results of the aero-engine blades, and save and output the crack detection results of the aero-engine blades to the maintenance team for maintenance decision-making.

[0119] The following describes a schematic embodiment of the engine blade surface crack detection method based on U2-Net and Canny operator provided by the embodiments of the present invention to verify the effectiveness and beneficial effects of the method. The exemplary embodiment is as follows:

[0120] Example 1

[0121] This Example 1 uses 12 surface images of aero-engine blades collected to verify the crack detection in the case of small samples, so as to verify the accuracy and effectiveness of the engine blade surface crack detection method based on U2-Net and Canny operator proposed by the present invention. The surface image data of the aero-engine blades is provided by Shanxi Zhidian Technology Co., Ltd.

[0122] In this Embodiment 1, a network model for detecting cracks on the surface of small-sample engine blades based on the U2-Net semantic segmentation network and the Canny operator is established. The overall structure diagram of this network model is as shown in Figure 1 shown. The network model established by this method includes a first module CA-U2Net, a second module WiCanny of the Canny operator integrated with a Wiener filter, and a superimposing module for superimposing the results processed by the first module and the results processed by the second module to obtain a superimposed result.

[0123] Among them, the specific structure diagram of the established first module CA-U2Net is as shown in Figure 2 shown, and the RSU module (CA-RSU) with an embedded coordinate attention mechanism is as shown in Figure 3 and Figure 4 shown.

[0124] The superimposing module superimposes the results processed by the first module and the results processed by the second module:

[0125] result(x,y) = U(x,y) + UC(x,y)

[0126] Among them, result(x,y) represents the superimposed result image, U(x,y) represents the result after the first module CA-U2Net performs crack segmentation on the engine blade, UC(x,y) represents the result of the second module WiCanny of the Canny operator integrated with a Wiener filter after segmenting U(x,y), and (x,y) represents the pixel value in the image.

[0127] Next, the flowchart for using the constructed network model to detect cracks on the surface of engine blades is as shown in Figure 5 shown. The specific steps are as follows: Collect an image and input it into the network model. The input image is first sent to the feedforward network CA-U2Net for processing, and the obtained result is sent to the Canny operator WiCanny with a Wiener filter for further segmentation. To ensure the accuracy of the segmentation result, the superimposing module fuses the results obtained by the first module and the second module, superimposes the results of the two to obtain the superimposed result, and performs an opening operation in morphology on the fused result to obtain the final crack segmentation result.

[0128] In this Embodiment 1, since only 12 aviation engine blade images are collected, if such a small dataset is directly used to train the CA-U2Net network, a good effect cannot be obtained. Therefore, the publicly available road crack dataset CFD is used to train the CA-U2Net network, and then the trained network model is used to predict the crack images on the surface of engine blades.

[0129] To explore the effectiveness of the network model of the present invention in segmenting cracks on the surface of engine blades, the segmentation results of engine blade cracks obtained using the deeplabV3+, U-Net, and U2-Net networks of the prior art were used as comparative examples for analysis and comparison. Each model in this comparative example was trained for 300 training rounds on the road crack dataset CFD, and the experimental parameters were set as shown in Table 1, and the evaluation criteria were compared as shown in Table 2. FCU2C is the network model obtained in Example 1 of the above-described embodiment of the present invention.

[0130] Table 1 Experimental model parameter table

[0131]

[0132] Table 2 Comparison of evaluation indexes for crack detection on the surface of aeroengine blades in Example 1 and comparative examples

[0133]

[0134]

[0135] To further compare the performance capabilities of the above four methods in the task of aeroengine blade crack detection and further visualize the segmentation results, Figure 6 show a comparison diagram of partial engine blade crack results.

[0136] From Table 2 and Figure 6It can be seen that after training with the road crack dataset CFD, all four methods can segment the cracks on the engine blade to varying degrees. Among all the comparison methods, the comprehensive segmentation performance of the network model FCU2C in Example 1 is the best, the segmentation effect of deeplabv3+ in the comparative example is the worst, and the segmentation effect of the U2-Net network is second only to the network model FCU2C in Example 1. Compared with other networks, the convolutional structure of deeplabv3+ in the comparative example can effectively capture the context information of large-scale images and performs better in segmenting large objects, but performs poorly in segmenting small targets such as cracks on engine blades; in the comparative example, U2-Net fuses multi-scale information of images and extracts information from feature maps at different levels through skip connections, and performs better than deeplabv3+ and U-Net in the task of detecting cracks on engine blades; compared with the U2-Net network in the comparative example, the network model FCU2C in Example 1 first adds a coordinate attention mechanism to U2-Net to make the model pay more attention to the cracks on the engine blade, and secondly uses CA-U2Net as the feedforward of the network model FCU2C in Example 1 to improve the robustness of the model; finally, adding a Canny detection operator further improves the segmentation ability of the model in Example 1 for fine cracks, which can not only completely detect fine cracks, but also show quite high accuracy and robustness in detecting cracks with various shapes. Its characteristic of paying attention to details enables it to effectively segment cracks from the background and perform well in detecting the position and shape of cracks. The F1-score of the network model FCU2C in Example 1 in the task of segmenting cracks on engine blades reaches 95.65%, which is higher than other comparison networks in the comparative example.

[0137] From the above results, it can be seen that the model FCU2C constructed by applying the method for detecting cracks on the surface of an engine blade based on U2-Net and the Canny operator provided by the present invention can well segment the cracks on the surface of an aero-engine blade with fewer samples, and its performance is better than that of the remaining semantic segmentation algorithms. The method for detecting cracks on the surface of an engine blade based on U2-Net and the Canny operator proposed by the present invention has the characteristics of requiring no large amount of data sets, high detection accuracy, and strong real-time performance, and overcomes the deficiencies of existing traditional edge detection algorithms and semantic segmentation algorithms. By introducing transfer learning and combining the advanced U2-Net semantic segmentation network with the Canny edge detection operator, it can effectively detect the cracks on the surface of an aero-engine blade under the condition of small samples. This method improves the accuracy of crack detection and effectively fills the gap in the detection of cracks on the surface of engine blades in the small-sample industrial field.

[0138] All the above optional technical solutions can be combined arbitrarily to form optional embodiments of the present application, which will not be elaborated here one by one.

[0139] It should be understood that the sequence numbers of the steps in the above embodiments do not indicate the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.

[0140] The above is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed by the present invention should be covered by the protection scope of the present invention.

Claims

1. An engine blade surface crack detection method based on U2-Net and Canny operator, characterized in that: The following steps are involved: Step S1. Collecting aircraft engine blade images and preprocessing them to obtain a crack dataset as a test set; Step S2. Obtain a road crack data set CFD, and select and divide it to obtain a training set and a validation set; Step S3. construct a network model for surface crack detection of aircraft engine blades including a first module, a second module and a superposition module, and use a training set for preliminary training to obtain a network model after preliminary training, wherein the first module is CA-U2Net, and the second module is a Canny operator WiCanny fused with a Wiener filter; Step S4. Setting evaluation indicators for training the network model, selecting algorithm parameters for the network model, and establishing a loss function for optimizing the training process of the network model; Step S5. Train, test and optimize the network model after preliminary training to obtain the final network model; Step S6. Use the final network model to detect and evaluate the surface image of the engine blade to be detected, obtain the aircraft engine blade crack detection result, and save and output the aircraft engine blade crack detection result to the maintenance team for maintenance decision making.

2. The engine blade surface crack detection method based on U2-Net and Canny operator according to claim 1 is characterized in that: The step S1 specifically includes the following steps: Step S1-1. Collecting an image of an aircraft engine blade, wherein the image of the aircraft engine blade is an image having cracks on the surface of the aircraft engine blade; Step S1-2. Preprocess the collected aircraft engine blade images to obtain a crack data set as a test set. The preprocessing includes median filtering and histogram equalization.

3. The engine blade surface crack detection method based on U2-Net and Canny operator according to claim 2 is characterized in that: The step S2 specifically includes the following steps: Step S2-1. Obtaining a road crack dataset CFD, and selecting to obtain a selected road crack dataset, the selection includes selecting pictures with similar semantic information to cracks on the surface of an aircraft engine blade from the road crack dataset CFD, and further selecting annotated undamaged images and corresponding label images therefrom as the selected road crack dataset; Step S2-2: Divide the selected road crack dataset in proportion to obtain a training set and a validation set.

4. The engine blade surface crack detection method based on U2-Net and Canny operator according to claim 3 is characterized in that: The step S3 specifically comprises the following steps: Step S3-1. Construct the first module and the second module of the network model, wherein the first module is CA-U2Net, and the second module is the Canny operator WiCanny fused with the Wiener filter, wherein the first module CA-U2Net adds a coordinate attention mechanism to the semantic segmentation network U2-Net to optimize the network performance; Step S3-2. Constructing a superposition module of the network model, for superimposing the results obtained by the first module and the second module to obtain a network model including the first module, the second module and the superposition module; Step S3-3. Use the training set obtained in step S2 to perform preliminary training on the network model to obtain a network model after preliminary training.

5. The engine blade surface crack detection method based on U2-Net and Canny operator according to claim 4 is characterized in that: The step S4 specifically comprises the following steps: Step S4-1. Set the evaluation indicators of the network model, including mean intersection over union (MIoU), accuracy, precision, recall and F1-score as evaluation indicators to evaluate the crack segmentation effect; Step S4-2. Algorithm parameter selection of the network model, including selecting the operating framework, processor, optimizer, batch size, initial learning rate, minimum learning rate, training rounds and dual threshold parameters of the network model; Step S4-3. Establish the loss function of the first module CA-U2Net of the network model.

6. The engine blade surface crack detection method based on U2-Net and Canny operator according to claim 5 is characterized in that: The evaluation indicators of step S4-1 include: The mean intersection over union (MIoU) is the average of the intersection over union (MIoU) of each class in the dataset of the two sets of true values ​​and predicted values. The calculation formula is as follows: Among them, i represents the true value, j represents the predicted value, and p ij represents the number of true labels i and predicted labels j, p ii represents the number of predicted labels whose real labels are i, p ji represents the number of true labels j and predicted labels i, k+1 represents the number of categories, where the number of categories k=1, and the value range of MIoU is [0,1]. The closer the value is to 1, the higher the consistency between the predicted result and the true result, and the better the segmentation effect. The precision is the pixel-level accuracy between the predicted segmentation result and the actual segmentation result. The calculation formula is as follows: Among them, TP is the number of engine blade cracks that are correctly detected, and FP is the number of non-cracks that are considered as cracks; The recall rate recall indicates the proportion of correct predictions made by the network model in the results where the true value is the positive class. The calculation formula is as follows: Where FN is the number of cracks treated as non-cracks; Accuracy is the pixel-level accuracy between the predicted segmentation result and the actual segmentation result. The calculation formula is as follows: Where TN is the number of correctly detected non-engine blade cracks; F1-score comprehensively evaluates the performance of the network model based on the precision and recall of the network model. The calculation formula is as follows:

7. The engine blade surface crack detection method based on U2-Net and Canny operator according to claim 6 is characterized in that: The step S5 specifically comprises the following steps: Step S5-1. Use the test set obtained by dividing in step S2 and the algorithm parameters selected in step S4 to train the network model after preliminary training, and monitor the loss function and evaluation index during the training process to ensure that the network model gradually converges and achieves the expected performance, thereby obtaining the trained network model; Step S5-2. Testing and evaluating the network model: using the test set obtained in step S1 to test the trained network model to obtain test results, and using pre-set evaluation indicators to comprehensively evaluate the performance of the network model; Step S5-3. Evaluate and analyze the test results to determine whether the performance of the current network model meets the preset requirements. If it meets the preset requirements, save the current network model as the final network model and end this step. Otherwise, return to step S4, adjust the algorithm parameters, and re-train and test.

8. The engine blade surface crack detection method based on U2-Net and Canny operator according to claim 7 is characterized in that: The step S6 specifically comprises the following steps: Step S6-1: obtaining an image of an aircraft engine blade to be detected and performing preprocessing to obtain a data set to be detected, wherein the preprocessing includes median filtering and histogram equalization of the surface image of the aircraft engine blade; Step S6-2: input the data set to be tested obtained in step S6-1 into the final network model for testing to obtain a test result; Step S6-3: The detection results obtained in step S6-2 are comprehensively evaluated using pre-set evaluation indicators, and the detection results are visualized and analyzed to obtain the aircraft engine blade crack detection results, and the aircraft engine blade crack detection results are saved and output to the maintenance team for maintenance decision-making.

Images