Method for analyzing thin film crystal grains of scanning electron microscope based on artificial intelligence technology

Abstract

The invention discloses a method for analyzing thin film crystal grains of a scanning electron microscope based on an artificial intelligence technology, and the method comprises the steps: carrying out the successful crystal grain segmentation of a scanning electron microscope image of a thin film through a convolutional network model, and carrying out the physical feature extraction of each crystal grain; the extracted features are subjected to standardization and Pearson's correlation coefficient feature engineering; according to input data processed by feature engineering, a data set is divided into a training set and a test set, machine learning network models such as a support vector machine (SVM) and a random forest (RF) are utilized to carry out score verification on the data set, and the importance degree of the influence of the morphological features of thin film crystal grains on the efficiency of a device is analyzed through an SHAP value. According to the method, high-precision prediction can be realized by using a small-sample-size data set, and meanwhile, errors caused by analysis depending on human experience in the past are solved.

Description

Technical Field

[0001] The present invention relates to the intersection of artificial intelligence and physics, chemistry and materials, and in particular to a method for analyzing thin film grains using a scanning electron microscope using artificial intelligence. Background Art

[0002] A scanning electron microscope is a type of electron microscope that produces an image of a sample's surface by scanning the surface of the sample with a focused electron beam. Scanning electron microscopes can achieve nanometer-level resolution. Scanning electron microscopes detect secondary electrons emitted by atoms excited by the electron beam. The number of secondary electrons that can be detected depends on the topography of the sample, among other factors. By scanning the sample and collecting the emitted secondary electrons using a special detector, an image is created that shows the topography of the surface.

[0003] Machine learning, deep learning and other technologies can perform grain data mining from scanning electron microscope thin film data sets, establish models for the linear and nonlinear relationships between the physical information characteristics of thin film grains and target properties to achieve precision and efficiently predict thin film grains, thereby guiding experimental data analysis. Currently, "Berg S, Kutra D, Kroeger T, et al. Ilastik: interactive machine learning for (bio) image analysis [J]. Nature Methods, 2019, 16 (12): 1226-1232" uses deep learning to segment, divide, count and track cell morphology, and users can interactively provide sparse training annotations for nonlinear classifiers; "Sadanandan S K, Ranefall P, Le Guyader S, et al. Automated training of deep convolutional neural networks for cell segmentation [J]. Scientific reports, 2017, 7 (1): 1-7" is trained on the ground truth automatically created using fluorescently labeled cells through a deep convolutional network DCNN model, with high values ​​within the cell area and low values ​​at the edge, and the final segmentation results of individual cells are automatically generated by using Cell Profiler's watershed-based segmentation: "Chen D, Sarkar S, Candia J, et al. Machine learning based methodology to identify cell shape phenotypes associated withmicroenvironmental cues[J].Biomaterials,2016,104:104-118” 22 shape metric features extracted from images of single cells were analyzed by machine learning support vector machine (SVM) algorithm. However, this type of method has its limitations in grain segmentation of thin film datasets and statistical physical characteristics of grain morphology. For example, if grain segmentation is incomplete, its physical properties will be missing; if the grain morphology characteristics are not fully analyzed, the analysis results will be affected. Therefore, to a certain extent, artificial intelligence technology is limited in exploring the intrinsic relationship between scanning electron microscope film images and device performance. Summary of the invention

[0004] The technical problem to be solved by the present invention is to accurately segment the grains in the thin film scanning electron microscope image through deep learning and to count the physical characteristics of the grains. After data preprocessing and feature engineering, the thin film grains of the present invention are input into the machine learning model for prediction and screening, so as to analyze the impact of the change of the morphology of the thin film grains on the device efficiency.

[0005] To achieve the above purpose, the technical solution of the present invention is as follows:

[0006] A method for analyzing the impact of scanning electron microscope thin film grains on device performance based on artificial intelligence, including the construction and preprocessing of thin film grain data sets, feature engineering of data sets, construction and training of machine learning models, inputting training and test data sets into the machine learning model for scoring judgment, and analyzing which grain morphology features are most critical to device efficiency through the output results of the model using SHAP values.

[0007] Preferably, the thin film grains used in the data set are perovskite layers.

[0008] Preferably, the construction and preprocessing of the thin film grain data set is to segment the grains and the background of the collected scanning electron microscope images of the perovskite layer, and then generate morphological features of the grain features of the obtained thin film grains through ImageJ software.

[0009] Preferably, the grain and background segmentation is performed by setting the grain diameter parameter on the thin film grain image using Cellpose software based on the U-Net network model. Specifically, after the collected scanning electron microscope thin film images are manually preliminarily photographed and screened, the thin film grains are extracted by setting the parameters of Cellpose software based on the U-Net convolutional network model to generate a thin film grain mask grayscale image.

[0010] Preferably, the grayscale image of the film grain mask obtained by the pre-processing is used to generate morphological features of the grain features using ImageJ software.

[0011] Preferably, the feature engineering of the data set is to use manual screening, standardization, and Pearson correlation coefficient matrix algorithm on the extracted thin film grains, and divide the thin film grain data set into a ratio of training set:test set=8:2.

[0012] The method for constructing and training a machine learning model is preferably as follows: according to the input data processed by feature engineering of the data set, the machine learning model is trained by grid search to obtain a test set with an accuracy rate of up to 97.2% under the Random Forest classification model. The present invention uses Cellpose software based on the U-Net convolutional network model to perform grain segmentation on a scanning electron microscope film image, generates morphological features of the segmented film grain mask grayscale image by ImageJ software, verifies the test set of the data set by the machine learning model according to the input data processed by feature engineering, finds the best parameters, and obtains the highest verification score, performs feature importance analysis on the features of the grain morphology by SHAP value, and obtains the results of the influence of the film grain morphology on the device efficiency.

[0013] The data set construction and preprocessing are preferably as follows:

[0014] (1) Cellpose software based on the U-Net convolutional network model, by importing the scanning electron microscope film image into the software, setting its Avg cell diameter, submitting it, and adjusting the parameters multiple times to retain the best segmentation effect, and downloading and saving it as a mask grayscale image with the suffix .png;

[0015] (2) Use ImageJ software to generate grain morphology features from the saved thin film grain mask grayscale image, and obtain a grain morphology feature dataset as the input of the machine learning classification model dataset.

[0016] The thin film image used in the data set is preferably a scanning electron microscope thin film image of a perovskite LED.

[0017] The above-mentioned method for analyzing the influence of grain morphology on device performance in scanning electron microscope thin film images based on artificial intelligence technology specifically comprises the following steps:

[0018] Step a: First, the collected scanning electron microscope thin film images are preprocessed to separate the grains from the background and generate a thin film grain mask grayscale image;

[0019] Step b: The generated thin film grain mask grayscale image is used to generate grain morphology characteristics through ImageJ software, and then the grain morphology characteristic unit is converted from pixel to parameter with length as the unit using Photoshop software;

[0020] Step c: The 42 feature data sets generated from the entire film were preprocessed using standardization and Pearson correlation coefficient matrix algorithms, 18 features were deleted, and 24 features were retained for the entire film; the retained grain morphology feature data sets were passed through five machine learning classification models, including support vector machine (SVM), random forest (RF), and decision tree, to score and verify the test set in the data set, and the feature importance of the entire film features was analyzed using the SHAP value, and the features with the most direct impact were selected as labels for exploring the impact of a single film grain on device efficiency. The standard deviation and Pearson correlation coefficient matrix algorithms were used again to perform feature processing on the data sets from the retained 24 grain morphology feature data sets, and finally 8 features were retained for training the machine learning model to determine the impact of grain morphology changes on device efficiency.

[0021] The above method for influencing the performance of a device with thin film grains is preferably in step a:

[0022] (1) Sampling the thin film image of the perovskite layer by scanning electron microscopy;

[0023] (2) The collected film images are used to set the parameters of the film grain diameter using the Avg cell diameter function in the Cellpose software to separate the grains from the background and generate a mask grayscale image;

[0024] In the above-mentioned thin film grain analysis method, preferably in step b:

[0025] (1) Import the generated thin film grain mask grayscale image into ImageJ software, and convert the mask image into 16-bit format through Image-type for ImageJ processing;

[0026] (2) The converted film grain mask grayscale image is used to generate grain morphology features through Plugins-MorphoLibJ-Analyze-AnalyzeRegions in ImageJ software;

[0027] (3) Using Photoshop software, the original image and the generated mask grayscale image are compared pixel by pixel to calculate the real length of the grain morphology feature in length units;

[0028] The above method for influencing the performance of the thin film grains on the device is preferably in step c:

[0029] (1) The calculated true characteristic length of the thin film grains is first analyzed as a whole, that is, the characteristics of all grains in the entire image are averaged for feature input;

[0030] (2) Based on the processed overall data set, the preprocessing was performed using standardization and Pearson correlation coefficient matrix algorithm, and the overall data set was analyzed using five machine learning models such as support vector machine (SVM) and random forest (RF);

[0031] (3) Optimize parameters of different machine learning models through grid search to find the optimal model parameters;

[0032] (4) Select verification accuracy as the performance evaluation indicator to evaluate the quality of the network model during training;

[0033] (5) The importance of grain morphology characteristics is analyzed by SHAP value for the overall data, and it is concluded that the film grain area has the greatest impact on the entire film image;

[0034] (6) Then, a single grain in the thin film image is taken as the research object for analysis, and steps (2), (3), (4), and (5) are performed in the same manner. Finally, it is concluded that the shape and area of ​​the thin film grains have the most direct impact on the device efficiency. When the grain shape is close to a circle, the device efficiency is good, and the grain area is between 20000-60000nm. 2 The device efficiency is good.

[0035] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0036] (1) The present invention effectively solves the problems of missing perovskite grains and unclear edge segmentation from scanning electron microscope film images through Cellpose software based on the U-Net convolutional network model; at the same time, the powerful information extraction ability of the deep learning neural network framework model can fully mine the edge information of the grain structure on the film, realizing the use of film grain information. The ImageJ software effectively solves the problem of extracting grain morphology feature information from the segmented film image. The accuracy of the machine learning model test set can be as high as 97.2%. The SHAP value can effectively analyze which grain morphology features have the most direct impact on the efficiency of the device. Based on the artificial intelligence method, the influence of grain morphology features in the scanning electron microscope film image on the device performance is effectively analyzed, which solves the previous analysis method relying on human experience.

[0037] (2) The artificial intelligence method established by the present invention can achieve high-precision prediction using a small sample size data set. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is an algorithm flow chart of the grain analysis method in the scanning electron microscope thin film image;

[0039] Figure 2Comparison of thin film grain segmentation using Cellpose software, a is the original image, b is the grain segmentation image;

[0040] Figure 3 It is the grayscale image of the film grain mask input into ImageJ software;

[0041] Figure 4 It is the Pearson correlation coefficient matrix heat map of film grain characteristics;

[0042] Figure 5 Score graphs for different machine learning models when analyzing the overall data;

[0043] Figure 6 This is the distribution diagram of the effect of grain morphology on device efficiency when analyzing the entire film with SHAP value;

[0044] Figure 7 Score graphs for different machine learning models when analyzing single grain morphology data on thin films;

[0045] Figure 8 This is a distribution diagram of the impact of grain morphology on device efficiency when analyzing the morphology of a single grain on a SHAP value film. DETAILED DESCRIPTION

[0046] The present invention is described in detail below in conjunction with specific embodiments.

[0047] Example 1

[0048] A method for analyzing the influence of grains on device performance in scanning electron microscope images of thin films based on artificial intelligence technology, the implementation process is as follows Figure 1 shown.

[0049] The following steps are included: construction and preprocessing of thin film grain dataset, feature engineering of dataset, and optimization of different machine learning model parameters.

[0050] 1. Construction and preprocessing of thin film grain dataset, including the following steps:

[0051] (1) Segmentation of thin film grains and background. First, obtain a thin film grain dataset for model segmentation, and perform thin film grain segmentation using the Cellpose software developed by CarsenStringer in 2020 based on the U-Net convolutional network model. The perovskite layer in the perovskite FAPbI3 light-emitting diode is sampled using a scanning electron microscope. The collected thin film images are parameterized by the Avg cell diameter function in the Cellpose software to segment the grains from the background and generate a mask grayscale image.

[0052] (2) Import the film grain mask grayscale image successfully segmented by Cellpose into ImageJ software to generate grain morphology features. Specifically, after importing the film grain mask grayscale image into ImageJ software, first convert the film grain mask grayscale image into a 16-bit format image through Image-type, and then generate grain features through Plugins-MorphoLibJ-Analyze-AnalyzeRegions (the data is in pixels at this time).

[0053] 2. Feature engineering of the dataset, including the following steps:

[0054] (1) Compare the pixels of the original image and the generated mask image through Photoshop to calculate the real length of the grain feature in length units, that is, convert the grain morphology feature unit from pixels to parameters in length units through Photoshop software for the scanning electron microscope image of the film;

[0055] (2) Manual screening, standardization and feature engineering of grain morphology using the Pearson correlation coefficient matrix algorithm. For the entire film, 42 grain morphology features are generated using Cellpose software and ImageJ software. If the feature has a low variance value, it will contain less physical information. If there is a strong correlation between the features, deleting these redundant features will effectively reduce information redundancy. The Pearson correlation coefficient matrix algorithm is used to delete features with coefficients greater than 0.95. After feature engineering, 24 features are first retained for overall film training. The SHAP value is used to analyze the overall film features and the film grain area has the most direct impact on the device efficiency results. The area of ​​the film grain is then used as the label of the data set. The standard deviation and Pearson correlation coefficient matrix algorithm are used again to perform feature processing on the data set from the retained grain morphology feature data set. Finally, 8 features are retained for training the machine learning model. The Pearson correlation coefficient matrix is ​​as follows: Figure 4 As shown;

[0056] (3) The scanning electron microscope overall thin film grain data set is divided into a training set: test set ratio of 8:2, that is, when training the model, 80% of the data is used for training and 20% of the data is used for testing.

[0057] 3. Build and train the machine learning model, which includes the following steps:

[0058] (1) When constructing the machine learning model, because the grain dataset in the scanning electron microscope image of the film is not suitable for the nonlinear machine learning model, five nonlinear machine learning models in Scikit-learn were selected to train and verify the dataset.

[0059] (2) When training a machine learning model, in each round of training, the sample sizes of the training set and the test set are fixed, and grid search is used to adjust the model parameters to find the parameters that give the machine learning model the highest test set score.

[0060] Figure 5 It is the highest score of the model for the training set and test set of the grain data set in the overall scanning electron microscope image of the thin film after grid search and parameter adjustment.

[0061] 4. SHAP value feature importance analysis, specifically including the following steps:

[0062] Based on the construction and training of the machine learning model in step 3, after multiple parameter adjustment training, the model parameters with a test set accuracy of more than 94.5% under the five classification models were retained. Then, the feature importance analysis of the five model results was performed using the SHAP value. It can be seen that the film grain area has the most direct impact on the device efficiency. The prediction results are as follows: Figure 6 shown.

[0063] Example 2

[0064] When constructing the XGB Classifier model, the grain features in the scanning electron microscope image of the thin film are used as the data set, and the specific construction and training of the machine learning model are described in Example 1.

[0065] At this time, the verification accuracy of the XGB Classifier model reached up to 94.5%. The prediction results are as follows: Figure 5 shown.

[0066] Example 3

[0067] When constructing the Gaussian NB model, the grain features in the scanning electron microscope image of the thin film are used as the data set, and the specific construction and training of the machine learning model are described in Example 1.

[0068] At this time, the verification accuracy of the Gaussian NB model reaches up to 100%, and the prediction results are as follows: Figure 5 shown.

[0069] Example 4

[0070] When constructing the Decision Tree Classifier model, the grain features in the scanning electron microscope image of the film are used as the data set, and the specific construction and training of the machine learning model are described in Example 1.

[0071] At this time, the verification accuracy of the Decision Tree Classifier model reached 98.2%. The prediction results are as follows: Figure 5 shown.

[0072] Example 5

[0073] When constructing the Random Forest Classifier model, the grain features in the scanning electron microscope image of the film are used as the data set, and the specific construction and training of the machine learning model are described in Example 1.

[0074] At this time, the verification accuracy of the Random Forest Classifier model reached up to 98.6%. The prediction results are as follows Figure 5 shown.

[0075] Example 6

[0076] When constructing five machine learning models, a single grain in the thin film scanning electron microscope image is used as a data set, and the specific construction and training of the machine learning model is described in Example 1.

[0077] At this time, the verification accuracy of the Random Forest Classifier model reached 97.2%, and the prediction results are as follows Figure 7 By analyzing the grain characteristics through the SHAP value, it can be seen that the device efficiency is high when the grain shape is close to a circle, and the grain area size is between 20000-60000nm 2 The device efficiency is high when the device efficiency is high. The prediction results are as follows Figure 8 shown.

[0078] The source of the perovskite layer in the perovskite FAPbI3 light-emitting diode used in the above embodiment is recorded in https: / / doi.org / 10.1038 / s41586-018-0576-2, Perovskite light-emitting diodes based on spontaneously formed submicrometre-scale structures. It should be understood that for those of ordinary skill in the art, improvements or changes can be made according to the above description, and all these improvements and changes should fall within the scope of protection of the claims of the present invention.

Claims

1. A method for analyzing thin film grains based on artificial intelligence technology, characterized in that The method includes the construction and preprocessing of thin film grain datasets, feature engineering of datasets, construction and training of machine learning models, inputting training and test datasets into the machine learning model for scoring judgment, and analyzing the output results of the model through SHAP values ​​to identify the grain morphology features that are most critical to device efficiency.

2. The method for analyzing thin film grains based on artificial intelligence technology according to claim 1, characterized in that The thin film grains used in the data set are perovskite layers.

3. The method for analyzing thin film grains based on artificial intelligence technology according to claim 1, characterized in that The construction and preprocessing of the thin film grain data set is to segment the grains and the background of the collected perovskite layer scanning electron microscope image, and then generate morphological features of the obtained thin film grains through ImageJ software.

4. The method for analyzing thin film grains based on artificial intelligence technology according to claim 2, characterized in that The grain and background segmentation is performed by using Cellpose software based on the U-Net network model to segment the thin film grain image by setting the grain diameter parameter.

5. The method for analyzing thin film grains based on artificial intelligence technology according to claim 1, characterized in that The feature engineering of the data set is to construct and preprocess the thin film grain data set to generate thin film grain features by converting pixel units into length units through Photoshop, and then use manual screening, standardization and Pearson correlation coefficient matrix algorithm on the thin film grains to divide the thin film grain data set into a ratio of training set: test set = 8:

2.

6. The method for analyzing thin film grains based on artificial intelligence technology according to claim 1, characterized in that The method for constructing and training a machine learning model is specifically as follows: according to the input data processed by feature engineering of the data set, the machine learning model is trained by grid search, and the accuracy of the test set under the Random Forest classification model is as high as over 97.2%.

7. The method for analyzing thin film grains based on artificial intelligence technology according to claim 1, characterized in that The method specifically comprises the following steps: Step a: First, the collected scanning electron microscope thin film images are preprocessed to separate the grains from the background and generate a thin film grain mask grayscale image; Step b: The generated thin film grain mask grayscale image is used to generate a grain morphology feature data set through ImageJ software, and then the grain morphology feature unit is converted from pixel to parameter in length unit using Photoshop software; Step c: The 42 feature data sets generated from the entire film were preprocessed using standardization and Pearson correlation coefficient matrix algorithms, 18 features were deleted, and 24 features were retained for the entire film; the retained grain morphology feature data sets were scored and verified on the test set in the data set using five machine learning classification models, namely, machine learning support vector machine, random forest, and decision tree. The feature importance of the entire film features was analyzed using the SHAP value, and the features with the most direct impact were selected as labels for exploring the impact of a single film grain on device efficiency. The standard deviation and Pearson correlation coefficient matrix algorithms were used again to perform feature processing on the data sets from the retained 24 grain morphology feature data sets, and finally 8 features were retained for training the machine learning model to determine the impact of grain morphology changes on device efficiency.

Images