Material strain prediction method in in-situ tensile EBSD experiment based on CNN

Abstract

A CNN-based method for predicting material strain in in-situ tensile EBSD experiments involves first collecting EBSD Euler diagrams and material strain data from in-situ tensile EBSD experiments; then, data augmentation is performed on the constructed dataset using a sliding window cropping method; finally, a ResNet18 network is built and trained, using Euler diagram images as input and material strain as output, to predict the corresponding tensile strain based on the EBSD Euler diagrams at different deformation stages of the alloy material. This method is the first to construct a CNN-based pipeline for predicting material strain, achieving a mean square error of only 0.56, effectively linking microstructural evolution with macroscopic deformation processes through neural networks.

Description

Technical Field

[0001] The present invention belongs to the field of material microstructure characterization and property prediction. In order to apply deep learning to EBSD experiments and model the relationship between material microstructure characterization and material mechanical properties, a CNN-based material strain prediction method in in-situ tensile EBSD experiments is proposed. Background Art

[0002] The various microstructural characteristics formed during the manufacturing process, such as grain size distribution, texture, and inclusions, determine the macroscopic deformation behavior and mechanical properties of alloy materials during service. This macroscopic deformation process is often accompanied by the dynamic evolution of the associated microstructure. Electron backscatter diffraction (EBSD) technology, based on scanning electron microscopy (SEM), determines crystal structure, orientation, and related information by analyzing the diffraction Kikuchi bands formed by the SEM electron beam on the tilted sample surface. This technology can efficiently collect and analyze microstructural evolution information, such as grain rotation and orientation difference evolution, during alloy material deformation.

[0003] Deep learning technology has rapidly advanced, achieving significant success in fields such as autonomous driving, data processing, and drug design. Simultaneously, its application in materials science is also increasing. Convolutional neural networks (CNNs) are used in materials research to model the relationship between material microstructural images and their corresponding material properties, enabling prediction tasks. Carl Kusche et al. used high-resolution SEM to perform panoramic imaging of DP800 dual-phase steel during and after deformation. They then used a CNN to classify the damage within the high-resolution microstructural images into inclusion-induced damage and deformation-induced damage. A new CNN was then trained to further classify the deformation-induced damage images into four damage mechanism categories. Using a neural network for automated image damage analysis surpasses traditional threshold analysis of grayscale images, saving human researchers significant analysis time. Kondo et al. used a CNN to learn features from ceramic microstructural images to predict the material's ionic conductivity, demonstrating experimentally that the CNN's predictions outperformed traditional machine learning algorithms. Li et al. proposed a framework for predicting the effective material properties of multiphase heterogeneous materials. They used a CNN to establish a mapping between the mesoscale microstructure of shale samples and their effective modulus. First, SEM images of the shale microstructure are captured, and a random reconstruction algorithm is used to generate a large number of shale samples. The finite element method is then used to calculate the effective modulus of the random shale samples, which is used as the label for the training samples. A CNN is trained based on the images and effective modulus of the random shale samples. The low prediction error achieved on the test set demonstrates the good predictive performance of the CNN. However, few studies have applied deep learning methods to analyze and process microstructural evolution data associated with the macroscopic deformation process of materials during in situ experiments, and even fewer studies have used microstructure to predict material deformation behavior.

[0004] Matej et al. designed a system for predicting the mechanical properties of materials based on metallographic images. They first designed a visual algorithm to extract the proportions of the material's microstructure from the metallographic images as image features. They then used a deep neural network to predict multiple mechanical properties of the material based on these calculated image features. This algorithmic system can save the expensive experiments required to measure and verify the mechanical properties of materials. However, this algorithmic system requires manual feature selection and relies on the subjective choices of researchers. It does not take advantage of the automatic feature extraction advantages of deep learning and lacks universality for various microstructures. Summary of the Invention

[0005] The present invention aims to overcome the above-mentioned defects of existing research and proposes a material strain prediction method in in-situ tensile EBSD experiments based on CNN.

[0006] The present invention obtains the Euler orientation distribution image data required for neural network training through multiple sets of in-situ tensile EBSD experiments. The collected data set is enhanced by clipping to meet the requirements of neural network training. A ResNet18 network is built, and the EBSD Euler diagram image is used as the network input to output the corresponding material strain. The network training is performed in a supervised manner. The neural network built and trained by the present invention can predict the corresponding material macroscopic strain based on the EBSD Euler diagram with low error, verifying the feasibility of the neural network in associating microstructural evolution with the macroscopic deformation process.

[0007] The CNN-based material strain prediction method in in-situ tensile EBSD experiments includes the following steps:

[0008] 1) Collect in situ tensile EBSD experimental data;

[0009] In-situ tensile specimens were obtained by electrospark cutting the alloy base material. The sample surface was smoothed by sandpaper grinding and mechanical polishing. The mechanically polished sample was electropolished to remove residual stress on the sample surface. In-situ tensile testing was performed on the sample using an in-situ tensile EBSD combined testing system. During the tensile process, several deformation stages were selected based on the real-time tensile curve, and several areas of the sample surface were characterized using EBSD, and the corresponding strains were recorded. The EBSD data was analyzed and processed to obtain Euler diagrams corresponding to different strains.

[0010] 2) Constructing EBSD Euler diagram dataset;

[0011] We used a sliding window method to crop the Euler graph images, selecting a small sliding step size to obtain the required amount of Euler graph data for neural network training. The resulting images were scaled down to 224×224 pixels to accommodate the neural network input while retaining sufficient texture information. The dataset was randomly partitioned into training, validation, and test sets.

[0012] 3) Build the ResNet18 network;

[0013] ResNet18 takes a 224×224×3 RGB image as input and has 18 parameter layers. ResNet is based on the VGG network and has been modified. Convolutional layers use 3×3 filters with a stride of 2 when downsampling and 1 otherwise. Whenever the feature map size is halved, the number of channels is doubled to maintain the complexity of each layer. For every two stacked layers, a residual learning block is constructed, which learns the residual between the output and input features through short-circuit connections. The network concludes with a global average pooling layer and a fully connected layer. The fully connected layer outputs a single value as the regression result of the strain.

[0014] 4) Train the ResNet18 model on the Euler graph data;

[0015] The network is trained using a supervised approach. Each time a set of images is fed into the network, the output is calculated based on the network's current weights. A suitable loss function is used to calculate the difference between the network output and the true label. Backpropagation is then used to update the network weights with the goal of minimizing this loss. Each completed training session is called a batch of data, and once all the data has been used for training, an epoch is completed. After each epoch, the training loss and the loss on the validation set are output to monitor the training progress and inform hyperparameter adjustments.

[0016] The present invention performs linear normalization on the Euler graph image before inputting it into the network, and uses batch normalization after each convolution and before activation. MSE (mean square error) is used as the loss function:

[0017]

[0018] Where n is the number of samples, Y i is the true value of the strain, The strain values ​​predicted for the network.

[0019] 5) Apply the ResNet18 model to predict material strain;

[0020] Collect the EBSD Euler diagram image of the material to be predicted, crop the area with sufficient texture information on the image, and then scale it to 224×224 pixels. Input the trained ResNet18 model, and the output of the model is the predicted value of the material strain corresponding to the image.

[0021] The advantages of this invention are that the constructed and trained convolutional neural network can effectively extract features from EBSD images and predict the corresponding macroscopic strain with low error. This provides new ideas for the characterization and data analysis of the dynamic evolution of in-situ microstructures. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a flow chart of the method of the present invention.

[0023] Figure 2 This is the architecture diagram of the ResNet18 network used in the present invention. DETAILED DESCRIPTION

[0024] The following description is intended to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are for illustrative purposes only, and those skilled in the art will readily appreciate other obvious variations. The basic principles of the present invention defined in the following description may be applied to other embodiments, variations, improvements, equivalents, and other technical solutions that do not depart from the spirit and scope of the present invention.

[0025] Those skilled in the art should understand that, in the disclosure of the present invention, the terms "longitudinal", "transverse", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like to indicate orientations or positional relationships are based on the orientations or positional relationships shown in the accompanying drawings, which are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, the above terms should not be understood as limiting the present invention.

[0026] In the present invention, the term "a" or "an" in the claims and the specification should be understood as "one or more." That is, in one embodiment, the number of an element may be one, while in another embodiment, the number of the element may be multiple. Unless the disclosure of the present invention clearly indicates that the number of the element is only one, the term "a" or "an" should not be understood as a unique or singular element, and the term "a" or "an" should not be understood as a limitation on the quantity.

[0027] In the description of the present invention, it should be understood that the terms "first", "second", etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance. In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "connected" and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through a medium. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0028] In the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0029] The CNN-based material strain prediction method in in-situ tensile EBSD experiments includes the following steps:

[0030] 1) Collect in situ tensile EBSD experimental data;

[0031] Experiments were conducted using Inconel 718 nickel-based superalloy, TC4 titanium alloy, and duplex steel. In situ tensile specimens were obtained by electrospark cutting the alloy base materials. The in situ tensile specimens were polished with sandpaper in stages and then mechanically polished to a mirror finish using diamond paste. The mechanically polished specimens were then electropolished using an electrolytic polishing solution. The in situ tensile experiments were performed using an in-situ tensile stage, which can be installed within the SEM chamber and, together with the SEM and its associated EBSD probe, forms an in-situ tensile EBSD combined testing system. During the tensile process, several deformation stages were selected based on the real-time tensile curve. EBSD was used to characterize several regions of the sample surface, and the corresponding strains were recorded. The EBSD data were analyzed and processed using Channel 5 software to obtain Euler plots corresponding to different strains.

[0032] 2) Constructing EBSD Euler diagram dataset;

[0033] A sliding window of a certain size is used to perform a moving cropping operation on the Euler diagram with a fixed pixel step size. This involves moving the window from left to right and top to bottom across the original image with a fixed step size. Each time, a sub-image within the window is captured, and the strain corresponding to the original image forms a new set of data. This yields a sufficient set of Euler diagram-strain data. The resulting images are then scaled down to 224×224 pixels to accommodate the neural network input. The dataset is randomly partitioned, with 80% used as a training set, 10% as a validation set, and 10% as a test set.

[0034] 3) Build the ResNet18 network;

[0035] ResNet18 takes a 224×224×3 RGB image as input and has 18 parameter layers. ResNet is based on the VGG network and has been modified. The convolutional layer uses a 3×3 filter with a stride of 2 when downsampling and 1 otherwise. Whenever the size of the feature map is halved, the number of channels is doubled to maintain the complexity of each layer. For every two stacked layers, a residual learning block is constructed, which learns the residual between the output features and the input features through short-circuit connections. The network finally uses a global average pooling layer and a fully connected layer. The fully connected layer is made to output a single value as the regression result of the strain. ResNet18 is implemented using the PyTorch framework.

[0036] 4) Train the ResNet18 model on the Euler graph data;

[0037] The network is trained in a supervised manner. Each time, a set of Euler diagram images is linearly normalized and then input into the network. The types of materials corresponding to these Euler diagrams vary depending on the stage of the tensile test. The output is calculated based on the current weights of the network, and batch normalization is used after each convolution and before activation. MSE (mean squared error) is used as the loss function to calculate the difference between the network output and the true label. Then, with the goal of minimizing the loss, the network weights are updated through backpropagation. After each epoch of training, the training loss and the loss on the validation set are output to observe the training status and decide on the adjustment of hyperparameters accordingly. The Adam optimizer is used, and the learning rate is set to 0.0001. Training is completed based on NVIDIA GPU. The batch size is set to 128, and the training iteration is 200 epochs.

[0038] 5) Apply the ResNet18 model to predict material strain;

[0039] The EBSD Euler diagram image in the test set is input into the trained ResNet18 model. The output of the model is the predicted value of the material strain corresponding to the image, and the calculated prediction mean square error is 0.56.

[0040] Through the above five steps, a ResNet18 neural network can be trained, which can effectively extract the features of the EBSD Euler diagram, more accurately predict the corresponding macroscopic strain based on the Euler diagram, and model the relationship between the material microstructure characterization and the material mechanical properties.

[0041] The contents described in the embodiments of this specification are merely an enumeration of the implementation forms of the inventive concept. The scope of protection of the present invention should not be regarded as limited to the specific forms described in the embodiments. The scope of protection of the present invention also extends to equivalent technical means that can be conceived by those skilled in the art based on the inventive concept.

Claims

1. The CNN-based material strain prediction method in in-situ tensile EBSD experiments includes the following steps: 1) Collect in situ tensile EBSD experimental data; In-situ tensile specimens were obtained by electrospark cutting the alloy base material. The sample surface was smoothed by sandpaper grinding and mechanical polishing. The mechanically polished sample was electropolished to remove residual stress on the sample surface. In-situ tensile testing was performed on the sample using an in-situ tensile EBSD combined testing system. During the tensile process, several deformation stages were selected based on the real-time tensile curve, and several areas of the sample surface were characterized using EBSD, and the corresponding strains were recorded. The EBSD data was analyzed and processed to obtain Euler diagrams corresponding to different strains. 2) Constructing EBSD Euler diagram dataset; The sliding window method was used to crop the Euler diagram images, with a small sliding step size selected to obtain the required number of Euler diagram data for neural network training. The resulting images were reduced to 224×224 pixels to fit the neural network input while retaining sufficient texture information. The dataset was randomly divided into training, validation, and test sets. 3) Build the ResNet18 network; ResNet18 takes a 224×224×3 RGB image as input and has 18 parameter layers. ResNet is based on the VGG network and has been modified. The convolutional layers use 3×3 filters with a stride of 2 when downsampling and 1 otherwise. Whenever the feature map size is halved, the number of channels is doubled to maintain the complexity of each layer. For every two stacked layers, a residual learning block is constructed, which learns the residual between the output features and the input features through short-circuit connections. The network finally uses a global average pooling layer and a fully connected layer. The fully connected layer is made to output a single value as the regression result of the strain. 4) Train the ResNet18 model on the Euler graph data; The network is trained in a supervised manner; Each time a set of images is input into the network, the output is calculated based on the current weights of the network; the difference between the network output and the true label is calculated using a suitable loss function, and then the network weights are updated through backpropagation with the goal of minimizing the loss; Each time such training is completed, it is called processing a batch of data. When all the data are used for training once, it is called completing an epoch iteration. After each epoch of training, the training loss and the loss on the validation set are output to observe the training situation and decide on the hyperparameter adjustment accordingly. The Euler graph image is linearly normalized and then input into the network. Batch normalization is used after each convolution and before activation. The mean square error (MSE) is used as the loss function: Where n is the number of samples, Y i is the true value of the strain, The strain value predicted for the network; 5) Apply the ResNet18 model to predict material strain; Collect the EBSD Euler diagram image of the material to be predicted, crop the area with sufficient texture information on the image, and then scale it to 224×224 pixels. Input the trained ResNet18 model, and the output of the model is the predicted value of the material strain corresponding to the image.

Images