Fracture scanning electron microscope image automatic identification method based on convolutional neural network

Abstract

The invention discloses an automatic fracture scanning electron microscope image recognition method based on a convolutional neural network, and the method comprises the steps: collecting a fracture scanning electron microscope image, and manually calibrating the type of the fracture scanning electron microscope image; secondly, performing image preprocessing to form a data set; then, based on a deep learning convolutional neural network, an improved CNN network model is adopted, and based on a plurality of convolution kernels, a maximum pooling layer and a global average pooling algorithm, the metal fracture scanning image is analyzed, and the type of the metal fracture image is determined; and finally, inputting a to-be-recognized image, and performing statistics on a classification model of the image by using a global average pooling algorithm. According to the method, the risk that the accuracy is reduced due to experience or long-time work of personnel can be avoided, the time cost of analysis work is reduced, the type of metal fracture analysis is obtained, and the analysis result is reliable and efficient.

Description

Technical Field

[0001] The present invention relates to the field of material failure analysis, and more specifically, to a recognition method based on material scanning electron microscope images. Background Art

[0002] The fracture is the surface of a material after it breaks under the action of external force. Its morphology and characteristics can reflect information such as the material's performance, fracture mechanism, and use environment. Traditional fracture analysis methods mainly rely on microscopic observation and manual judgment by experienced experts. In recent years, with the rapid development of computer vision and machine learning technology, automated fracture analysis technology based on image processing and pattern recognition has gradually emerged. Among them, convolutional neural networks (CNNs) have performed well in tasks such as image classification, object detection, and image segmentation, and have become an important tool for research on automatic fracture recognition. However, current convolutional neural network models still have problems such as insufficient feature extraction, low classification accuracy, and poor generalization ability when processing highly complex fracture images. Summary of the invention

[0003] In order to overcome the shortcomings of existing evaluation methods, the present application provides a method for automatic recognition of metal fracture scanning electron microscope images based on convolutional neural networks, which achieves the result of accurate and automatic recognition of unknown image attributes by automatically extracting and training fracture scanning electron microscope image features.

[0004] To achieve the above-mentioned purpose, the technical solution adopted by the present invention is specifically as follows.

[0005] A method for automatic recognition of fracture scanning electron microscope images based on convolutional neural network, the steps are as follows:

[0006] Step 1, collect metal fracture scanning electron microscope images, and manually calibrate the fracture type; there are four types in total, namely, dimple morphology scanning electron microscope image, intergranular morphology scanning electron microscope image, cleavage morphology scanning electron microscope image, and fatigue fracture morphology scanning electron microscope image;

[0007] Step 2, constructing a data set using the metal fracture scanning electron microscope image collected in step 1,

[0008] Step 2.1, marking different types of metal fracture scanning electron microscope images collected in step 1, forming corresponding data sets according to different types, and dividing the data sets into two sub-data sets: a training set and a test set;

[0009] Step 2.2, performing image denoising and image sharpening preprocessing on each sub-data set;

[0010] Step 3, constructing a convolutional neural network model, and saving the optimal weight value after training the model;

[0011] Step 3.1: Build and train a convolutional neural network model;

[0012] The convolutional neural network model is composed of multiple convolution kernels, multiple maximum pooling layers and an attention mechanism. The training set images of each fracture type preprocessed in step 2.2 are input into the convolutional neural network model. The pixel values ​​of the images are passed through the multiple convolution kernels and the maximum pooling layer to obtain extraction values. The extraction values ​​are then calculated through the attention mechanism to obtain the type feature values ​​of the images.

[0013] Step 3.2: Optimizing the convolutional neural network model based on the training set and the test set processed by step 2.2 through training iterations to obtain the optimal weight of the model;

[0014] Step 3.2.1, first, input the training set images of each metal fracture type as input data into the convolutional neural network model; then, back-propagate the input data to update the weights; then, input the test set images of the corresponding type into the convolutional neural network model for verification;

[0015] Step 3.2.2, perform the above step 3.2.1 once for each training cycle, and record the loss function value and classification accuracy of each training cycle. After a preset number of complete iterations, the training cycle ends, and the corresponding training cycle is selected as the optimal training cycle according to the principle of minimum loss function value or maximum classification accuracy, and the model parameters corresponding to the optimal training cycle are used as the optimal weights of the model, and the optimal weights are recorded and saved; wherein, the loss function value refers to the numerical value of the difference between the model prediction result and the manual calibration, and the classification accuracy refers to the ratio of the number of correctly predicted samples to the total number of samples;

[0016] Step 3.3: Perform full connection layer processing on the pixel values ​​of all images calculated in step 3.2 to generate four types of values ​​for the metal fracture scanning image;

[0017] Step 4, identifying the type of the SEM image of the fracture to be tested; preprocessing the SEM image of the fracture to be tested in the same preprocessing method as step 2.2, calling the convolutional neural network model trained and tested in step 3.2.1 and the optimal weights saved in step 3.2.2, inputting the SEM image of the fracture to be tested into the convolutional neural network model, calling the optimal weights for detection, calculating the type feature value of the SEM image of the fracture to be tested, comparing it with the four type values ​​described in step 3.3, processing it through the softmax activation function, generating a probability distribution, the closest one is the type to which it belongs, and finally outputting the name of the type to which it belongs.

[0018] Furthermore, the attention mechanism described in step 3.1 adopts a global average pooling algorithm, that is, the global average pooling algorithm of the deep learning convolutional neural network is used to compress the spatial dimension of the input feature map into a single vector to generate a classification model for the image.

[0019] Furthermore, the fully connected layer processing described in step 3.3 is performed by adding the bias term to the product of the input vector and the weight matrix, and then performing a nonlinear transformation through the activation function ReLU to generate four types of values ​​for the metal fracture scanning image.

[0020] Furthermore, the pre-set number of traversals in step 3.2.2 is greater than 100.

[0021] Compared with the prior art, the beneficial effects of this application are:

[0022] 1. It can realize the automatic extraction of fracture SEM image features and the automatic recognition of unknown fracture SEM images, which can reduce the complicated feature annotation of fracture SEM images and the inaccuracy of manual recognition. The accuracy of current fracture SEM image recognition methods is generally 85%-90%, and the accuracy of this application can reach more than 97%, which greatly improves the recognition accuracy.

[0023] 2. The existing model recognition speed is about 0.0095 images / second, and the recognition speed of this application is 0.076 images / second, which is about 8 times faster. This application greatly shortens the recognition time, effectively improves work efficiency, and reduces time costs.

[0024] 3. It can avoid the risk of decreased accuracy due to experience or long working hours, obtain the type of metal fracture, and make the analysis results reliable and efficient. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 is an overall flow chart of the method of the present invention;

[0026] Figure 2 It is a model diagram based on a convolutional neural network of the method described in the present invention;

[0027] Figure 3 It is a training iterative flow chart of the fracture scanning electron microscope image based on the convolutional neural network of the method described in the present invention. DETAILED DESCRIPTION

[0028] The technical solution of the present application is described in detail below with reference to the accompanying drawings and specific embodiments.

[0029] A method for automatic recognition of fracture scanning electron microscope images based on convolutional neural network, the process is as follows Figure 1 shown.

[0030] Step 1, collecting metal fracture scanning electron microscope images, and manually calibrating the fracture type, which has four types, namely, dimple morphology scanning electron microscope image, intergranular morphology scanning electron microscope image, cleavage morphology scanning electron microscope image, and fatigue fracture morphology scanning electron microscope image;

[0031] Step 2, constructing a data set using the metal fracture scanning electron microscope image collected in step 1,

[0032] Step 2.1, marking different types of metal fracture scanning electron microscope images collected in step 1, forming corresponding data sets according to different types, and dividing the data sets into two sub-data sets: a training set and a test set;

[0033] Step 2.2, performing image denoising and image sharpening preprocessing on each sub-data set;

[0034] Step 3, constructing a convolutional neural network model, and saving the optimal weight value after training the model;

[0035] Step 3.1: Build and train a convolutional neural network model;

[0036] The convolutional neural network model is composed of multiple convolution kernels, multiple maximum pooling layers and an attention mechanism. The training set images of various types preprocessed in step 2.2 are input into the convolutional neural network model. The pixel values ​​of the images are passed through the multiple convolution kernels and the maximum pooling layer to obtain extraction values. The extraction values ​​are then calculated through the attention mechanism (the attention mechanism used in this embodiment is a global average pooling algorithm) to obtain the type feature value of the image; that is, the global average pooling algorithm of the deep learning convolutional neural network is used to compress the spatial dimension of the input image into a single vector, thereby generating a classification model for the image;

[0037] The specific structure of the convolutional neural network model in step 3.1 is shown in the attached figure. Figure 2As shown. The first convolution block has 2 convolution layers and 1 pooling layer. Each convolution layer has 64 3×3 convolution kernels, a stride of 1, a pooling layer with a maximum pooling layer, a 2×2 window, and a stride of 2. The second convolution block has 2 convolution layers and 1 pooling layer. Each convolution layer has 128 3×3 convolution kernels, a stride of 1, a pooling layer with a maximum pooling layer, a 2×2 window, and a stride of 2. The third convolution block has 3 convolution layers and 1 pooling layer. Each convolution layer has 256 3×3 convolution kernels, a stride of 1, a pooling layer with a maximum pooling layer, a 2×2 window, and a stride of 2. The fourth convolution block has 3 convolution layers and 1 pooling layer. Each convolution layer has 128 3×3 convolution kernels, a stride of 1, a pooling layer with a maximum pooling layer, a 2×2 window, and a stride of 2. There are 512 3×3 convolution kernels with a stride of 1, and the pooling layer is a maximum pooling layer with a 2×2 window and a stride of 2. There are 3 convolution layers and 1 pooling layer in the fifth convolution block. Each convolution layer has 512 3×3 convolution kernels with a stride of 1, and the pooling layer is a maximum pooling layer with a 2×2 window and a stride of 2. The convolution layer is followed by a global average pooling module, and then three fully connected layers, two of which include 4096 neurons, an activation layer, and a Dropout layer. The third fully connected layer is then processed by a softmax activation function to generate a probability distribution, indicating the probability that the image belongs to each category.

[0038] Shallow convolutional layers focus more on basic features such as edges and textures, while deep convolutional layers can capture more advanced and abstract features. The types of fractures are diverse and vary greatly, so the network needs a larger receptive field to capture global information. The global average pooling module integrates information in a global range to provide the model with a stronger global information perception capability, helping the model to better understand the overall structure and global features of the input data. The model embedded with the global average pooling module can capture both local and global features of metal fracture images. Local features help identify the details and texture of the fracture, while global features help understand the overall shape and structure of the fracture. By stacking multiple convolutional layers and pooling layers, the model can extract features at different levels, gradually building a comprehensive understanding of metal fracture images from low to high levels.

[0039] Step 3.2: Optimizing the convolutional neural network model based on the training set and the test set processed by step 2.2 through training iterations to obtain the optimal weight of the model;

[0040] As attached Figure 3 As shown, in step 3.2.1, first, the training set images of the metal fracture types are used as input data and input into the convolutional neural network model; then, the input data is back-propagated to update the weights; then, the test set images of the corresponding type are input into the convolutional neural network model for verification;

[0041] First, the training set images of each metal fracture type are used as input data and input into the convolutional neural network model; then, the input data is back-propagated to update the weights; then, the test set images of the corresponding type are input into the convolutional neural network model for verification;

[0042] Step 3.2.2, perform the above step 3.2.1 once for each training cycle, and record the loss function value and classification accuracy of each training cycle. After a preset number of complete iterations, the training cycle ends, and the corresponding training cycle is selected as the optimal training cycle according to the principle of minimum loss function value or maximum classification accuracy, and the model parameters corresponding to the optimal training cycle are used as the optimal weights of the model, and the optimal weights are recorded and saved; wherein, the loss function value refers to the numerical value of the difference between the model prediction result and the manual calibration, and the classification accuracy refers to the ratio of the number of correctly predicted samples to the total number of samples;

[0043] Step 3.3: Perform a fully connected layer processing on the pixel values ​​of all images calculated in step 3.2, add the bias term by multiplying the input vector by the weight matrix, and then perform a nonlinear transformation through the activation function ReLU to generate four types of values ​​for the metal fracture scanning image;

[0044] Step 4, identifying the type of the SEM image of the fracture to be tested; preprocessing the SEM image of the fracture to be tested in the same preprocessing method as step 2.2, calling the convolutional neural network model trained and tested in step 3.2.1 and the optimal weight value saved in step 3.2.2, inputting the SEM image of the fracture to be tested into the convolutional neural network model, calling the optimal weight value for detection, calculating the type feature value of the SEM image of the fracture to be tested, comparing it with the four type values ​​described in step 3.3, processing it through the softmax activation function, generating a probability distribution, the closest one is the type to which it belongs, and finally outputting the name of the class type.

[0045] The formulas involved in the convolutional neural network include:

[0046] The convolutional layer formula is: Where X(i, j) represents the value of the input feature map at position (i, j), and K(m, n) represents the weight of the convolution kernel at position (m, n);

[0047] The maximum pooling formula is: Y(i, j) = max{X(2i, 2j), X(2i+1, 2j), (2i, 2j+1), (2i+1, 2j+1)}, where X is the input feature map, Y is the input feature map after pooling, and (i, j) is the upper left corner position of the pooling window;

[0048] The formula of the fully connected layer is: y = Wx + b, where W is the weight matrix, b is the bias vector, x is the input vector (the flattened feature map), and y is the output vector;

[0049] The ReLU activation function formula is: ReLU(x)=max(0,x);

[0050] The Softmax activation function formula is: Where zi is the score of the ith category, n is the total number of categories, is the exponent of zi, is the exponential sum of all category scores;

[0051] The loss function formula is: Where yi is the true label and pi is the predicted probability obtained by Softmax;

[0052] The formula for the gradient descent method is: Where θ is the model parameter (such as convolution kernel, weight of fully connected layer, etc.), α is the learning rate, and L is the loss function. is the gradient of the loss function with respect to the parameter θ;

[0053] The global average pooling formula is: where y c is the pooled output of the cth channel (scalar value), X c (i, j) is the feature value of the cth channel at position (i, j) in the input feature map, and H and W are the height and width of the feature map, respectively.

[0054] Comparing the traditional method (CNN method, i.e. the existing convolutional neural network method) and the method described in the present invention (the improved CNN method, i.e. the CNN method including convolution kernel, maximum pooling layer, global average pooling, and full connection layer), it can be seen that the method described in the present invention has high recognition accuracy and faster recognition speed. The comparative test conditions are: SEM images are obtained through the Internet, papers and shooting methods to obtain fracture SEM images of different sizes and resolutions, and the computer environment is 12th Gen Intel(R)Core(TM)i7-12700H 2.30GHz processor, NVIDIA GeForce RTX4060 Laptop GPU, and Pycharm framework. The comparison results of the fracture image recognition type accuracy and recognition speed are shown in Table 1. It can be seen that the recognition accuracy has increased by nearly 10%, and the recognition speed has increased by 5-8 times, with significant effects:

[0055]

[0056] For the proper expansion of this method, the data set of collected images can be increased, each module can be refined according to different classifications, and the parameters can be appropriately changed according to different data sets. It can be extended to more SEM image classification tasks.

[0057] The above is only a preferred specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can make equivalent substitutions or changes within the technical scope disclosed in the present application according to the technical solution and inventive concept of the present application, which should be covered by the protection scope of the present application.

Claims

1. A method for automatic recognition of metal fracture scanning electron microscope images based on convolutional neural network, comprising: Step 1, collecting a scanning electron microscope image of a metal fracture and manually calibrating the fracture type; Step 2, constructing a data set using the metal fracture scanning electron microscope image collected in step 1, Step 2.1, marking different types of metal fracture scanning electron microscope images collected in step 1, forming corresponding data sets according to different types, and dividing the data sets into two sub-data sets: a training set and a test set; Step 2.2, performing image denoising and image sharpening preprocessing on each sub-data set; It is characterized in that after step 2, the step of constructing a convolutional neural network model is performed, which is specifically as follows: Step 3, constructing a convolutional neural network model, and saving the optimal weight value after training the model; Step 3.1: Build and train a convolutional neural network model; The convolutional neural network model is composed of multiple convolution kernels, multiple maximum pooling layers and an attention mechanism. The training set images of each fracture type preprocessed in step 2.2 are input into the convolutional neural network model. The pixel values ​​of the images are passed through the multiple convolution kernels and the maximum pooling layer to obtain extraction values. The extraction values ​​are then calculated through the attention mechanism to obtain the type feature values ​​of the images. Step 3.2: Optimizing the convolutional neural network model based on the training set and the test set processed by step 2.2 through training iterations to obtain the optimal weight of the model; Step 3.2.1, first, input the training set images of each metal fracture type as input data into the convolutional neural network model; then, back-propagate the input data to update the weights; then, input the test set images of the corresponding type into the convolutional neural network model for verification; Step 3.2.2, perform the above step 3.2.1 once for each training cycle, and record the loss function value and classification accuracy of each training cycle. After a preset number of complete iterations, the training cycle ends, and the corresponding training cycle is selected as the optimal training cycle according to the principle of minimum loss function value or maximum classification accuracy, and the model parameters corresponding to the optimal training cycle are used as the optimal weights of the model, and the optimal weights are recorded and saved; wherein, the loss function value refers to the numerical value of the difference between the model prediction result and the manual calibration, and the classification accuracy refers to the ratio of the number of correctly predicted samples to the total number of samples; Step 3.3: Perform full connection layer processing on the pixel values ​​of all images calculated in step 3.2 to generate four types of values ​​for the metal fracture scanning image; Step 4, identifying the type of the SEM image of the fracture to be tested; preprocessing the SEM image of the fracture to be tested in the same preprocessing method as step 2.2, calling the convolutional neural network model trained and tested in step 3.2.1 and the optimal weights saved in step 3.2.2, inputting the SEM image of the fracture to be tested into the convolutional neural network model, calling the optimal weights for detection, calculating the type feature value of the SEM image of the fracture to be tested, comparing it with the four type values ​​described in step 3.3, processing it through the softmax activation function, generating a probability distribution, the closest one is the type to which it belongs, and finally outputting the name of the type to which it belongs.

2. The method for automatic recognition of metal fracture scanning electron microscope images based on convolutional neural network according to claim 1, characterized in that: The attention mechanism described in step 3.1 adopts a global average pooling algorithm, that is, the global average pooling algorithm of the deep learning convolutional neural network is used to compress the spatial dimension of the input metal fracture scanning electron microscope image into a single vector to generate an image classification model.

3. The method for automatic recognition of metal fracture scanning electron microscope images based on convolutional neural network according to claim 1, characterized in that: The fully connected layer processing described in step 3.3 is to generate four types of values ​​for the metal fracture scanning image by multiplying the input vector by the weight matrix plus the bias term, and then performing a nonlinear transformation through the activation function ReLU.

4. The method for automatic recognition of metal fracture scanning electron microscope images based on convolutional neural network according to claim 1, characterized in that: The pre-set number of traversals described in step 3.2.2 is greater than 100.

Images