Engine stator part small target crack length measurement method based on deep convolutional neural network

Abstract

The invention relates to an engine stator part small target crack length measurement method based on a deep convolutional neural network, and the method comprises the steps: carrying out the labeling of an initial fatigue crack image obtained through carrying out a fatigue crack propagation experiment, converting the initial fatigue crack image into a PNG label, and carrying out the expansion of the original image and the label through a data intensifier, constructing a VOC (volatile organic compound) format small target crack data set; with the help of the strong feature extraction capability and fine distinguishing capability of a deep convolutional neural network, a semantic segmentation network for small target crack segmentation is constructed based on a full convolutional network, a U-shaped architecture and jump connection, and training is performed to obtain a trained model weight for high-quality crack segmentation; and based on the segmented image, extracting a crack skeleton with a pixel width by adopting preprocessing means such as expansion and a refinement algorithm, counting the number of pixel points in different arrangement modes based on the crack skeleton, calculating to obtain a crack pixel length, and performing scale conversion to obtain an actual crack length. According to the invention, high-precision measurement of the crack length of a large batch of small targets is realized.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of material mechanical property prediction and strength evaluation, and relates to an engine stator small target crack length measurement method based on a deep convolutional neural network. BACKGROUND

[0002] With the continuous improvement of the performance requirements of advanced aero-engines, the turbine inlet temperature and rotational speed of the aero-engines are increasingly high, the centrifugal load, thermal load and vibration load are coupled, the service load is complex, the service environment is harsh, and the indexes of long service life and high reliability are more demanding. Cracks caused by factors such as foreign object impact and metallurgical defects bring great safety risks to aero-engines, and whether the cracks can be accurately identified and measured in field maintenance becomes one of the keys to ensure the safe service of engine structures.

[0003] Crack measurement, that is, determining the geometric properties of cracks, such as length, width, area and the like, currently mainly adopts manual inspection and gauge measurement, optical measurement, ultrasonic measurement and the like. Manual inspection and gauge measurement are simple and fast to operate, but the measurement precision is limited, the reading to a certain extent depends on the operation experience and subjective judgment of the measurement personnel, and there are many inconveniences in measuring cracks at positions such as internal surfaces; in addition, contact-type measurement methods such as three-coordinate measuring machines and the like with the aid of high-precision instruments improve the measurement precision, however, the measurement process is tedious, and the collision and friction between the measuring head and the surface of the component may generate new scratches and damage, which interferes with the measurement results, at the same time, in order to ensure the standardization of the measurement, this method is also difficult to meet the task demand of large quantities of maintenance in a short time. Ultrasonic measurement has a wide detection range, high speed and low cost, but the measurement precision is not high, and is easily disturbed by noise. Optical measurement methods with the aid of endoscopes have the advantages of high-resolution imaging and rapid inspection, and are widely used in the daily maintenance of aero-engine structures, however, this method has strict requirements for equipment and environmental lighting conditions, and the detection field of view is limited. The published Chinese invention patent application (application number CN202310240077.5) “Fatigue crack length measurement method based on video analysis” adopts traditional computer vision-based image processing means such as matrix processing of images, OTSU threshold segmentation and the like to extract crack features, this type of method has good adaptability to high-contrast, simple background long cracks (length > 25 mm) detection and segmentation, but the precision is often greatly discounted when applied to small target crack and complex background recognition and segmentation, and the feature extraction capability is very limited.

[0004] The crack segmentation model based on the deep convolutional neural network has strong feature extraction capability, and is obviously superior in feature recognition and quantization, but the crack size photographed in the actual field stator component maintenance is small, and the proportion in the image is low, leading to the imbalance between positive and negative samples in the crack identification and measurement of the aero-engine component, the identification and segmentation precision of the network is limited, and there is a lack of data set for model training. SUMMARY

[0005] The application provides an engine stator small target crack length measurement method based on a deep convolutional neural network.

[0006] The technical solution of the application is as follows:

[0007] The engine stator small target crack length measurement method based on the deep convolutional neural network comprises the following steps:

[0008] Step one, fatigue crack propagation experiment is carried out to obtain an initial fatigue crack image, crack labeling is carried out to convert the initial fatigue crack image into a PNG label, a data enhancer is used to expand the initial fatigue crack image and the PNG label at the same time, and a VOC format small target crack data set is constructed;

[0009] Step two, a semantic segmentation network is constructed based on a full convolution network, a U-shaped architecture and a skip connection, the semantic segmentation network presents an encoder-decoder architecture, a pre-trained VGG-16 is used as an encoder, a decoder part refers to a U-Net semantic segmentation network, the encoder is connected with the decoder through a skip connection, a focal loss and a dice loss are selected as a joint loss function, and a CBAM attention mechanism is embedded in the decoder;

[0010] Step three, the initial fatigue crack image and the PNG label are used as training inputs, a crack segmentation image is used as training output, the semantic segmentation network is trained in a transfer learning mode, the hyperparameters in the network are adjusted so that the model reaches a convergence state on the training set and the validation set, that is, the joint loss function values corresponding to the two sets tend to be stable, the model weight after training is obtained, and new crack images are inferred based on this to generate a crack segmentation image reflecting crack features;

[0011] Step four, based on the crack segmentation image, a pre-processing method is adopted to smooth the image, and a one-pixel-width crack skeleton is extracted by using the skeletonize function in the skimage library based on the Zhang-Suen thinning algorithm.

[0012] Step five, based on the crack skeleton, the number of pixels arranged horizontally and vertically on the skeleton, and the number of pixels arranged diagonally are counted respectively, the crack pixel length is calculated, and the crack actual length is calculated according to the image actual size-pixel size conversion scale.

[0013] The beneficial effects of the present application compared with the existing model parameter acquisition method are:

[0014] (1) The initial fatigue crack image is obtained by carrying out fatigue crack propagation experiment, and the initial collected image and label are expanded at the same time by using the data enhancer, so that the number of crack samples is sufficient, the data type is variable, and the small target crack well simulates the phenomenon that the crack is rare and the sample occupies a small amount of total space in the structure of the aero-engine;

[0015] (2) The semantic segmentation network is constructed by taking the pre-trained VGG-16 as the encoder, taking the focal loss and the dice loss as the joint loss function, and embedding the CBAM (Convolutional Block Attention Module) attention mechanism, so as to solve the crack segmentation problem caused by the imbalance of positive and negative sample distribution in the initial fatigue crack image with complex background, and high segmentation precision of small target crack is achieved;

[0016] (3) Based on the high-quality small target crack segmentation image, the crack skeleton is extracted, the number of pixels arranged in different ways is counted based on the crack skeleton, and the crack pixel length and the crack actual length are calculated, so as to avoid the subjective judgment error in the manual measurement process, ensure the uniformity of the measurement results of different operators, and realize high-precision measurement of a large number of small target crack lengths.

[0017] In summary, the present application carries out fatigue crack propagation experiment to obtain initial fatigue crack image and realizes data set expansion by using data enhancer, so as to ensure sufficient number of crack samples and variable data types; at the same time, the pre-trained VGG-16 is introduced as the encoder, the focal loss and the dice loss are taken as the joint loss function, and the CBAM attention mechanism is introduced to solve the imbalance problem of positive and negative sample distribution in the crack image, realize high-quality segmentation of small target crack, extract crack skeleton based on the segmentation image and count the number of pixels arranged in different ways, and realize high-precision measurement of small target crack length. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 The flow chart of the engine stator small target crack length measurement method based on the deep convolutional neural network of the present application.

[0019] Fig. 2 (a) is a schematic diagram of VGG-16 network structure.

[0020] Fig. 2(b) is a schematic diagram of a semantic segmentation network with a pre-trained VGG-16 as an encoder and embedded attention mechanism.

[0021] Figure 3 A schematic diagram of the statistical crack pixel point arrangement method of the present application.

[0022] Figure 4 A comparison of the measured length and the true length of the small target crack of the present application. DETAILED DESCRIPTION

[0023] Exemplary embodiments of the present disclosure will be described in greater detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure can be more thoroughly understood, and the scope of the present disclosure can be accurately conveyed to those skilled in the art.

[0024] Figure 1 A flowchart of the engine stator small target crack length measurement method based on a deep convolutional neural network of the present application. Below, the method is described with reference to Figure 1 As shown in Figure 1 , the method comprises:

[0025] Step 1: Conduct fatigue crack propagation experiments to obtain initial fatigue crack images, perform crack labeling to convert to PNG labels, and use a data augmenter to simultaneously augment the initial fatigue crack images and PNG labels to construct a VOC format small target crack data set with sufficient sample quantity and diverse data types.

[0026] The VOC format small target crack data set is composed of ImageSets, JPEGImages, and SegmentationClass, and the training set and the validation set are divided in a 7:3 ratio. The ImageSets folder stores a subfolder named Segmentation, which stores a series of TXT text files that record the file names of each image in the training set and the validation set. The JPEGImages folder stores the original JPG format images, i.e., the initial fatigue crack images. The SegmentationClass folder stores PNG label files converted from JSON format label files.

[0027] In the initial fatigue crack images, the proportion of crack pixels to the total number of image pixels is small (less than 2%), and the data set has a significant foreground and background imbalance problem, simulating the phenomenon of few cracks and small total space quantity in actual aircraft engine maintenance.

[0028] Step two, a semantic segmentation network is constructed based on a full convolution network, a U-shaped architecture and a skip connection. The semantic segmentation network presents an encoder-decoder architecture. A pre-trained VGG-16 is used as the encoder. The decoder part refers to the U-Net semantic segmentation network. The encoder is connected to the decoder by a skip connection. A focal loss and a dice loss are selected as a joint loss function. A CBAM attention mechanism is embedded in the decoder.

[0029] FIG. 2(a) is a schematic diagram of a VGG-16 network structure. FIG. 2(b) is a schematic diagram of a semantic segmentation network with a pre-trained VGG-16 as an encoder and an embedded attention mechanism.

[0030] The VGG-16 is pre-trained based on a PASCAL VOC 2012 dataset and an SBD (Semantic Boundaries Dataset) dataset. The original VGG-16 includes convolution layers, pooling layers and fully connected layers. When training, only the pre-trained weights of the convolution layers and the pooling layers are loaded. The pre-trained weights of the fully connected layers do not need to be loaded. The decoder part is composed of a plurality of modules connected in series. Each module includes upsampling, splicing of feature maps corresponding to layers of the encoder, twice 3x3 padding convolution and an activation function. After each upsampling, the number of channels of the feature maps remains unchanged, and the spatial dimension is doubled. The decoder finally maps the high-dimensional features extracted by the encoder back to the spatial resolution of the original image and allocates a corresponding class label to each pixel.

[0031] Step three, the initial fatigue crack image and the PNG label are used as training inputs, and the crack segmentation image is used as training output. The semantic segmentation network is trained in a transfer learning manner. The hyperparameters in the network are adjusted so that the model reaches a converged state on the training set and the validation set, that is, the values of the joint loss functions corresponding to the two sets tend to be stable. The trained model weight is obtained. Based on this, a new crack image is inferred to generate a high-quality segmentation image reflecting crack features.

[0032] The precision evaluation index of the trained semantic segmentation network model is the mean Intersection over Union (mIoU) quantitative evaluation, that is, the average value of the Intersection over Union of each class of samples. The Intersection over Union is the ratio of the intersection to the union between the crack region in the crack segmentation image and the crack region in the real segmentation image, which measures the similarity between the predicted value and the true value. The calculation formula is as follows:

[0033] ,

[0034] wherein k is the number of target sample categories, k+1 indicates that the background is also counted as a category, p ij is the number of pixels of the i-th category predicted as the j-th category, p iiThe number of pixels of the i category predicted as the i category, that is, the prediction is correct.

[0035] Step four, based on the crack segmentation image, a pre-processing method is taken to smooth the image, and a one-pixel width crack skeleton is extracted by using the skeletonize function in the Zhang-Suen thinning algorithm based on the skimage library.

[0036] The pre-processing method includes expansion, median filtering, and closing operation.

[0037] Figure 3 The schematic diagram for the application for counting the arrangement mode of crack pixel points is shown. The black curve represents the crack skeleton, and the arrow length is allocated according to the adjacent relationship. The solid arrow length is 1 pixel, and the dotted arrow length is pixels.

[0038] Step five, based on the crack skeleton, the number of pixel points arranged horizontally and vertically adjacent on the skeleton, the number of pixel points arranged diagonally, and the crack pixel length are counted respectively, and the actual crack length is obtained by converting the scale of the actual image size and the pixel size.

[0039] Step five includes: respectively counting the number of pixel points arranged horizontally and vertically adjacent on the one-pixel width crack skeleton, denoted as , the number of pixel points arranged diagonally, denoted as , and the crack pixel length is obtained according to the following formula:

[0040] ,

[0041] The size calibration information is collected synchronously when the camera is shooting, and the actual crack length is calculated by multiplying the crack pixel length by the actual length corresponding to a single pixel based on the pre-established image actual size-pixel size conversion scale. The above length measurement method integrates the pre-trained encoder VGG-16, the focal loss and the dice loss, and the attention mechanism CBAM into the semantic segmentation network applied to the small target crack segmentation task to solve the problem of unbalanced distribution of positive and negative samples between the target crack and the background area.

[0042] The following describes an application example of the above-described method by taking a 7075-T6 aluminum alloy planar sheet specimen for fatigue crack propagation experiment as an example. The detailed steps are as follows:

[0043] ①Through the fatigue crack propagation experiment to collect the initial fatigue crack image, the specimen size is 300x100x1mm, in the form of a flat sheet, the center hole diameter is 10mm, and a 1mm pre-crack is processed on both sides of the center hole. The fatigue load loading mode is set to constant amplitude triangular wave loading, the maximum load is 50MPa, the load frequency ratio is 0.1, and the loading frequency is 10Hz. The LEICA CCD camera is used for image acquisition, a uniform light source is used to vertically irradiate the specimen surface, the camera is vertically shot to the specimen surface, and the fatigue crack monitoring area is about 15.85x10.57mm 2 , the corresponding image resolution is 3072x2048 pixels, and 1 pixel length is approximately equivalent to 5.2μm length on the specimen surface.

[0044] ②Labelme is used to complete the segmentation and label generation of small target cracks in JSON format, and the JSON format is converted into PNG format crack image label; the initial image and label are expanded by vertical flipping, horizontal flipping, rotation, cropping, local magnification and other means through the data enhancer, so as to ensure that the number of crack samples is sufficient, the data type is variable, and the proportion of crack pixels in the total number of image pixels is less than 2%.

[0045] ③A small target crack dataset in VOC format is constructed, which consists of ImageSets, JPEGImages and SegmentationClass. The ImageSets folder stores a subfolder named Segmentation, which stores a series of TXT text files, each recording the file name of each picture in the training set and the validation set. The JPEGImages folder stores the original JPG format image; the SegmentationClass folder stores the PNG label file converted from the JSON format label file. The dataset is divided into training set and validation set according to the ratio of 7:3.

[0046] ④Based on the full convolution network, U-shaped architecture and skip connection, a semantic segmentation network is constructed under the Pytorch framework. The pre-trained VGG-16 based on PASCAL VOC 2012 dataset and SBD dataset is used as the encoder. The model trained based on large amount of data samples can more fully master the edge contour, color, texture and other shallow features. Based on the pre-trained weight, the network can converge faster and improve the generalization ability of the model. The VGG-16 framework is shown in Fig. 2(a). The decoder maps the image features back to the pixel space through upsampling, the encoder and the decoder are connected by skip connection, the focal loss and the dice loss are selected as the joint loss function, the CBAM attention mechanism is embedded in the decoder, and the overall network architecture is shown in Fig. 2(b).

[0047] ⑤Take the actual crack image and PNG crack label image as the training input, and take the crack segmentation image as the training output. Train the semantic segmentation network in a transfer learning manner, adjust the hyperparameters in the network to make the model converge on both the training set and the validation set, that is, the joint loss function values corresponding to the two sets tend to be stable, and the training parameters are as follows:

[0048] The model is trained in a GPU environment, the GPU model is RTX A5000, the training round is 100 epochs, the weight is reserved once every 5 iterations, the batch size is 4, the initial learning rate is 0.0001, the learning rate decay strategy is cosine annealing decay, the Adam optimizer is selected, the momentum factor is set to 0.937, and the decay factor is set to 0.

[0049] ⑥The mean Intersection over Union (mIoU) is selected to quantitatively evaluate the precision of the semantic segmentation network model, that is, the average value of the intersection over union of each class of samples, the intersection over union is the ratio of the intersection and union between the crack area in the predicted segmentation image and the crack area in the real segmentation image, which measures the similarity between the predicted value and the true value, and the calculation formula is as follows:

[0050] ,

[0051] Where k is the number of target sample categories, k+1 means that the background is also counted as a category, p ij is the number of pixels of i category predicted as j category, p ji is the number of pixels of j category predicted as i category, and p ii is the number of pixels of i category predicted as i category. The larger the mIoU value, the closer the segmentation result to the label, and the better the segmentation effect. In this example, it is a small target crack and background binary classification, and the final trained model segmentation precision reaches 92.30%.

[0052] ⑦Based on the trained weights, infer new crack images to generate high-quality segmentation images reflecting crack features. Based on the crack segmentation image, first, perform dilation operation to enhance the small bright area in the image and expand the crack outline; then perform median filtering operation to smooth the image while reducing noise; closed operation processing combines the erosion and dilation steps, first erodes the image and then dilates it to fill the small holes and breaks in the image, while removing burrs, making the cracks in the image smoother and more complete.

[0053] ⑧The preprocessed image uses the skeletonize function in the skimage library based on the Zhang-Suen thinning algorithm to extract the crack skeleton, and finally obtains a one-pixel-wide crack skeleton.

[0054] 9. Based on the crack skeleton, the number of pixel points arranged horizontally and vertically on the skeleton, and the number of pixel points arranged diagonally are counted respectively, and the crack pixel length is calculated. Figure 3

[0055] 10. The results of batch measurement of small target crack length are shown in Table 1. Figure 4

[0056] The application is described above in conjunction with the drawings, but the application is not limited to the above specific implementation process, and the above specific implementation is only an example. Any invention and creation that does not exceed the claims of the application is within the protection of the application.​​

Claims

1. A method for measuring the crack length of small targets of engine stator parts based on deep convolutional neural network, characterized in that: The implementation steps are as follows: Step 1: Conduct fatigue crack growth experiments to obtain initial fatigue crack images, convert crack annotations into PNG tags, and use data enhancers to simultaneously expand the initial fatigue crack images and PNG tags to construct a small target crack dataset in VOC format. Step 2: Build a semantic segmentation network based on a fully convolutional network, a U-shaped architecture, and skip connections. The semantic segmentation network is presented as an encoder-decoder architecture, using the pre-trained VGG-16 as the encoder. The decoder part refers to the U-Net semantic segmentation network. The encoder and decoder are skip-connected, and the focal loss and dice loss are selected as the joint loss function. The CBAM attention mechanism is embedded in the decoder. Step 3: Using the initial fatigue crack image and PNG label as training input and the crack segmentation image as training output, the semantic segmentation network is trained using transfer learning. The hyperparameters in the network are adjusted so that the model converges on both the training set and the validation set, that is, the joint loss function values ​​corresponding to the two tend to be stable. The trained model weights are obtained, and based on this, new crack images are inferred to generate crack segmentation images that reflect the crack characteristics. Step 4: Based on the crack segmentation image, preprocessing measures are taken to smooth the image, and the skeleton of the crack with a width of one pixel is extracted using the skeletonize function in the skimage library based on the Zhang-Suen thinning algorithm; Step 5: Based on the crack skeleton, the number of pixels arranged horizontally and vertically, and the number of pixels arranged diagonally on the skeleton are counted respectively to calculate the pixel length of the crack. The actual length of the crack is calculated based on the actual image size-pixel size conversion scale.

2. The method for measuring the small target crack length of an engine stator based on a deep convolutional neural network according to claim 1 is characterized in that: Step 1 includes: using Labelme to segment and label small target cracks, generate JSON labels, and convert them into PNG format crack image labels; using a data enhancer, the initial image and labels are simultaneously expanded by vertical flipping, horizontal flipping, rotation and cropping, and local magnification.

3. The method for measuring the small target crack length of an engine stator based on a deep convolutional neural network according to claim 1 is characterized in that: The VOC format small target crack dataset consists of three parts: ImageSets, JPEGImages, and SegmentationClass. The training set and validation set are divided in a ratio of 7:

3. The ImageSets folder contains a subfolder named Segmentation, which contains a series of TXT text files that record the file names of each image in the training set and validation set respectively; the JPEGImages folder contains the original JPG format images, i.e., the initial fatigue crack images; and the SegmentationClass folder contains PNG label files.

4. The method for measuring the small target crack length of an engine stator based on a deep convolutional neural network according to claim 1, characterized in that: In the initial fatigue crack image, the number of crack pixels accounts for less than 2% of the total number of image pixels. The dataset has a significant foreground-background imbalance problem, which simulates the phenomenon that cracks are rare and occupy a small number of overall spaces during actual aircraft engine maintenance.

5. The method for measuring small target crack length of an engine stator based on a deep convolutional neural network according to claim 1, characterized in that: VGG-16 was pre-trained on the PASCAL VOC 2012 dataset and the Semantic Boundary dataset. The original VGG-16 contains convolutional, pooling, and fully connected layers. During training, only the pre-trained weights for the convolutional and pooling layers are loaded; the pre-trained weights for the fully connected layers do not need to be loaded. The decoder consists of several modules connected in series. Each module includes upsampling, concatenation of feature maps from the corresponding encoder layer, two 3×3 padded convolutions, and an activation function. After each upsampling, the number of feature map channels remains unchanged, but the spatial dimension is doubled. The decoder eventually maps the high-dimensional feature map extracted by the encoder back to the spatial resolution of the original image step by step and assigns a corresponding category label to each pixel.

6. The method for measuring small target crack length of an engine stator based on a deep convolutional neural network according to claim 5, characterized in that: Step 3 includes: training in a GPU environment, the GPU model is RTX A5000, the training round is 100 epochs, the weight is retained once every 5 iterations, the batch size is 4, the initial learning rate is 0.0001, the learning rate decay strategy is cosine annealing decay, the Adam optimizer is selected, the momentum factor is set to 0.937, and the decay factor is set to 0.

7. The method for measuring small target crack length of an engine stator based on a deep convolutional neural network according to claim 5, characterized in that: The accuracy evaluation index of the trained semantic segmentation network model is the average intersection-over-union (IoU) quantitative evaluation. The IoU is the ratio of the intersection and union between the crack area in the crack segmentation image and the crack area in the true segmentation image. It measures the similarity between the predicted value and the true value. The calculation formula is as follows: ， Among them, k is the number of target sample types, k+1 means that the background is also counted as a category, p ij Refers to the number of pixels of category i that are predicted to be category j, p ii It refers to the number of pixels in category i that are predicted to be category i, that is, the prediction is correct. The larger the mIoU value, the closer the segmentation result is to the annotation, and the better the segmentation effect.

8. The method for measuring small target crack length of an engine stator based on a deep convolutional neural network according to claim 1, characterized in that: Preprocessing methods include dilation, median filtering, and closing operations.

9. The method for measuring small target crack length of an engine stator based on a deep convolutional neural network according to claim 8, characterized in that: Preprocessing includes: First, a dilation operation is performed to enhance the small bright areas in the image and expand the outline of the crack; Then perform median filtering to smooth the image and reduce noise; The closing operation combines the two steps of erosion and dilation. The image is first eroded and then dilated to fill the small holes and breaks in the image, while removing burrs to make the cracks in the image smoother and more complete.

10. The method for measuring small target crack length of an engine stator based on a deep convolutional neural network according to claim 1, characterized in that: Step five includes: Count the number of pixels arranged horizontally and vertically adjacent to each other on the crack skeleton of one pixel width, and record it as , the number of pixels arranged diagonally is recorded as , the calculation formula of crack pixel length is obtained as: ， The size calibration information is collected synchronously when the camera is shooting. Based on the pre-established image actual size-pixel size conversion scale, the crack pixel length is multiplied by the actual length corresponding to a single pixel to calculate the actual crack length.

Images