In-situ electron diffraction data processing analysis method based on machine learning and application

Abstract

The application provides an in-situ electron diffraction data processing and analysis method based on machine learning and application, and the method comprises the following steps: automatic calibration of the center of a single frame electron diffraction pattern, automatic detection and tracking of diffraction spots, calculation and calibration of a diffraction ring, and data storage and analysis method for radial radius, light and dark contrast and tangential angle change of a single diffraction spot. The application combines machine learning, image processing and other methods, and realizes qualitative analysis and quantitative calculation of each diffraction spot in each frame of in-situ electron diffraction pattern.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of electron microscopy structure characterization, the technical field of machine learning, the technical field of image processing, and in particular to a machine learning-based in-situ electron diffraction data processing and analysis method and application. BACKGROUND

[0002] Electron diffraction refers to the diffraction phenomenon of electrons with certain energy when passing through certain samples or obstacles. In the field of electron microscopy structure characterization, electron diffraction is a powerful tool for microstructure analysis of thin samples, which expands the analysis capability of transmission electron microscopy in reciprocal space. The microstructure inside the sample is different, and the diffraction pattern is also different. For example, the diffraction pattern of amorphous material is a diffuse scattering ring; while polycrystalline or single crystal samples can get sharp diffraction rings or spots. From the radial distribution, angular arrangement, number and intensity of diffraction spots and rings, the size, orientation, crystallinity and atomic arrangement in the corresponding unit cell of the crystal grains in the sample can be deduced. However, due to the dynamic diffraction effect of electron diffraction and the possible defects inside the crystal, the electron diffraction intensity is not simply linearly related to the microstructure, and it is generally difficult to exclude the influence factors such as dynamic effect and defects during analysis. In addition, relying on traditional manual methods to calibrate, calculate and analyze the diffraction spots and diffraction rings in the electron diffraction pattern is not only subjective, but also very time-consuming. Especially for in-situ electron diffraction data, a single data package often contains hundreds of electron diffraction pictures, if still relying on manual methods to calibrate and analyze each picture, the calculation and analysis process will become difficult to practice due to the large amount of data. In addition, the traditional electron diffraction pattern processing method usually does ring integration on the diffraction spots and diffraction rings, and analyzes the microstructure characteristics of the material according to the one-dimensional contrast information after ring integration, which loses the two-dimensional distribution characteristics of the diffraction intensity and ignores the specific information carried by a single diffraction spot.

[0003] Machine learning, as a branch of artificial intelligence, is a kind of algorithm that extracts rules from data. At present, machine learning has been widely applied in computer vision, natural language processing, biometric identification and other fields. Traditional machine learning includes clustering analysis and pattern recognition methods, which have been used in the analysis of X-ray diffraction (XRD) data. By clustering and recognizing XRD data, material structure and attribute information can be obtained. The application of machine learning in XRD data confirms the feasibility of machine learning in electron diffraction data analysis tasks. How to use machine learning and combine image processing methods to calibrate and analyze in-situ electron diffraction data is a problem to be solved. Summary of the Invention

[0004] Therefore, the purpose of the present invention is to overcome the defects in the prior art and provide an in-situ electron diffraction data processing and analysis method and application based on machine learning.

[0005] To achieve the above objectives, the first aspect of the present invention provides an in-situ electron diffraction data processing and analysis method based on machine learning, the method comprising the following steps:

[0006] (1) Diffraction pattern center calibration;

[0007] (2) Diffraction spot detection, which includes: detecting a single diffraction spot and storing its position and adjacent rectangular frame data;

[0008] (3) Diffraction spot tracking, which includes: calculating the displacement vector of each diffraction spot in adjacent frames, tracking the diffraction spot, and storing its position and adjacent rectangular frame data in each video frame;

[0009] (4) detecting and tracking data storage, which includes: storing the position of each diffraction spot in each video frame and adjacent rectangular frame data;

[0010] (5) Diffraction ring calibration and analysis, which includes: clustering the radial radii of the diffraction spots obtained in step (3), and using the contrast information of the corresponding diffraction spots as the weight of the clustering algorithm to obtain the possible diffraction ring radius of the in-situ electron diffraction video;

[0011] (6) Screening and analyzing the diffraction spots. This step includes: counting and analyzing the changes in the radius, angle, and contrast of each diffraction spot over time, and analyzing the structural features contained therein based on the physical and chemical properties.

[0012] According to the method of the first aspect of the present invention, in step (1), the calibration method includes the following steps:

[0013] (A) Get the mean image from the input video;

[0014] (B) Calculate the Gaussian mixture model based on the mean image;

[0015] (C) The data were fitted using the Gaussian model with the largest mean among the Gaussian mixture model results to obtain the approximate outline of the transmission spot;

[0016] (D) Perform minimum adjacent circle detection on the transmission spot to obtain the center point of the diffraction pattern.

[0017] According to the method of the first aspect of the present invention, in step (B), the Gaussian mixture model is calculated using the scikit-learn package based on the Python language.

[0018] According to the method of the first aspect of the present invention, in step (C), fitting is performed using the scikit-learn package based on the Python language.

[0019] According to the method of the first aspect of the present invention, in step (2), a single diffraction spot is detected using the Laplacian of Gaussian method using the scikit-image package based on the Python language.

[0020] According to the method of the first aspect of the present invention, in step (3), the displacement vector of each diffraction spot in adjacent frames is calculated using the OpenCV package based on Python language and the Gunnar-Farneback dense optical flow method.

[0021] According to the method of the first aspect of the present invention, in step (4), the position of each diffraction spot in each video frame and the adjacent rectangular frame data are stored using the xlsxwriter package based on the Python language.

[0022] According to the method of the first aspect of the present invention, in step (5), the radial radii of the diffraction spots obtained in step (3) are clustered using the K-means clustering method in scikit-learn based on the Python language.

[0023] The second aspect of the present invention provides an electron microscopy structure characterization method, which includes performing in-situ electron diffraction data processing and analysis according to the method of the first aspect.

[0024] The third aspect of the present invention provides an electron diffraction device, which performs in-situ electron diffraction data processing and analysis according to the method described in the first aspect.

[0025] The present invention provides a new method for processing and analyzing in-situ electron diffraction data based on machine learning, combining machine learning, image processing and other methods to achieve automatic calibration of the center of the electron diffraction pattern, automatic detection and tracking of diffraction spots, calculation and calibration of diffraction rings, and data storage and analysis of the radial radius, bright and dark contrast and tangential angle of each diffraction spot in the in-situ electron diffraction over time. Compared with traditional analysis methods, the new method for processing and analyzing in-situ electron diffraction data provided by the present invention can achieve quantitative analysis of a single diffraction spot and more accurate calibration and analysis of material microstructure information. Figure 1 As shown in the figure, (a) shows the traditional annular integral diffraction analysis method, and (b) shows the diffraction spot calibration method based on machine learning.

[0026] The method of the present invention may have but is not limited to the following beneficial effects:

[0027] The present application provides a new method for processing and analyzing in-situ electron diffraction data based on machine learning, which combines machine learning, image processing and other methods to realize automatic calibration of electron diffraction pattern center, automatic detection and tracking of diffraction spots, calculation and calibration of diffraction rings, and data storage and analysis means for the change of radial radius, light and dark contrast and tangential angle of each diffraction spot in in-situ electron diffraction with time. Compared with the traditional analysis method, the new method for processing and analyzing in-situ electron diffraction data provided by the present application can realize quantitative analysis of single diffraction spot and more accurate calibration and analysis of material microstructure information. BRIEF DESCRIPTION OF DRAWINGS

[0028] Hereinafter, the embodiments of the present application will be described in detail with reference to the accompanying drawings, in which:

[0029] Figure 1 The analysis results of high-purity Co3O4 nanoparticle powder by traditional and machine learning-based diffraction analysis methods are shown, wherein, Figure 1 (a) shows the results obtained by the traditional ring integral diffraction analysis method, Figure 1 (b) shows the results obtained by the machine learning-based diffraction spot calibration method.

[0030] Figure 2 A flowchart of the in-situ electron diffraction data processing and analysis method based on machine learning of the present application is shown, which is divided into 6 sub-modules, namely diffraction pattern center calibration, diffraction spot detection, diffraction spot tracking, detection and tracking data storage, diffraction ring calibration and analysis, and screening and analysis of diffraction spots.

[0031] Figure 3 A schematic diagram of the analysis data results of a single diffraction spot obtained by the algorithm of the present application is shown.

[0032] Figure 4 A schematic diagram of the analysis data results of diffraction rings obtained by the algorithm of the present application and artificial calibration based on the traditional analysis method is shown, wherein, Figure 4 (a) shows a schematic diagram of the analysis data results of diffraction rings obtained by the algorithm disclosed in the present application, Figure 4 (b) shows the artificial calibration results based on the traditional analysis method. DETAILED DESCRIPTION

[0033] The present application will be further described by specific examples below, but it should be understood that these examples are only for more detailed and specific description, and should not be understood as limiting the present application in any form.

[0034] This section provides a general description of the materials and experimental methods used in the experiments of the present invention. Although many of the materials and procedures used to achieve the purposes of the present invention are well known in the art, the present invention is described herein in as much detail as possible. It will be understood by those skilled in the art that, unless otherwise specified, the materials and procedures used in the present invention are well known in the art.

[0035] The reagents and instruments used in the following examples are as follows:

[0036] Reagents:

[0037] High-purity Co3O4 nanoparticle powder was purchased from US Research Nanomaterials, with a purity higher than 99.5%.

[0038] instrument:

[0039] The transmission electron microscope was a field emission transmission electron microscope (TEM) model JEM2100F from JEOL Ltd., with an operating voltage of 200 kV.

[0040] Example 1

[0041] This example is used to illustrate the in-situ electron diffraction data processing and analysis method based on machine learning of the present invention.

[0042] like Figure 2 The flowchart of the in-situ electron diffraction data processing and analysis method based on machine learning is shown, which specifically includes 6 submodules:

[0043] Module 1 is a method for calibrating the center of the diffraction pattern. Typically, the information contained in diffraction spots close to the transmission spot is unreliable. Therefore, the first step in processing and analyzing in-situ electron diffraction video data is to calibrate the center of the diffraction pattern. The calibration results are particularly important for subsequent diffraction spot detection and tracking. Specifically, Module 1 mainly includes the following steps:

[0044] Step 1: Obtain a mean image from the input video;

[0045] Step 2: Based on the mean image, use the Python-based scikit-learn package to calculate its Gaussian Mixing Model.

[0046] Step 3: Using the Python-based scikit-learn package, the Gaussian model with the largest mean among the Gaussian mixture model results is used to fit the data to obtain the approximate outline of the transmission spot;

[0047] Step 4: Use the Python-based OpenCV package to perform minimum adjacent circle detection on the transmission spot to obtain the center point of the diffraction pattern.

[0048] Figure 1 Traditional and machine learning-based diffraction analysis methods are shown, where Figure 1 (a) shows the traditional annular integral diffraction analysis method. Figure 1 (b) shows the diffraction spot calibration method based on machine learning.

[0049] Module 2 is the detection of diffraction spots. Using the Python-based scikit-image package, the Laplacian of Gaussian method is used to detect individual diffraction spots and store their positions and adjacent rectangular box data.

[0050] Module 3 is about tracking diffraction spots. Using the Python-based OpenCV package, the Gunnar-Farneback dense optical flow method is used to calculate the displacement vector of each diffraction spot in adjacent frames. This allows tracking of the diffraction spot and stores its position and adjacent rectangle data in each video frame.

[0051] Module 4 stores detection and tracking data. Using the Python-based xlsxwriter package, the position of each diffraction spot in each video frame and the adjacent rectangle data are stored in an Excel spreadsheet for subsequent analysis. The center coordinates of the diffraction pattern calculated in Module 1 are stored in a text document.

[0052] Module 5 involves diffraction ring calibration and analysis. Using the K-means clustering method in scikit-learn (Python), the radial radii of the diffraction spots obtained in Module 3 are clustered. The contrast information of the corresponding diffraction spots is used as the weight of the clustering algorithm to determine the possible diffraction ring radii in the in situ electron diffraction video.

[0053] Module 6 is to screen and analyze the diffraction spots. The radius, angle, and contrast of each diffraction spot are counted and analyzed over time, and the structural features contained therein are analyzed based on the physical and chemical properties.

[0054] like Figure 3 The figure shows a schematic diagram of the analysis results of the radius, angle, and contrast of a single diffraction spot. The right side of the figure shows a schematic diagram of the detection of a single diffraction spot, with the box showing the diffraction spot numbered 2294. The left side of the figure shows the curve of the radius, angle, and contrast of this diffraction spot over time. Experimental results show that the algorithm disclosed in this invention achieves quantitative analysis of independent diffraction spots.

[0055] Figure 4(a) The analysis data results of the diffraction ring obtained by the algorithm disclosed in the present application are shown. The left of the figure shows a frame of image and the corresponding calibration results of the diffraction ring in the in-situ electron diffraction data of Co3O4; the right of the figure shows the relationship between the tangential angle and the radial radius of all diffraction spots in the frame of image, and each scatter point represents a diffraction spot. Figure 4 (b) The artificial calibration results based on the traditional analysis method are shown. The artificial calibration results can only determine the phase information of the polycrystalline material, while the algorithm of the present application can extract the change information of the lattice constant, size and orientation of a single material particle from the in-situ electron diffraction image. The experimental results show that the algorithm disclosed in the present application can obtain more abundant physical information compared with the artificial calibration algorithm. The algorithm proposed in the present patent is not dependent on the sample and the instrument, and has universality.

[0056] Although the present application has been described to a certain extent, obviously, appropriate changes can be made to each condition without departing from the spirit and scope of the present application. It can be understood that the present application is not limited to the described embodiments, but is subject to the scope of the claims, which includes equivalent replacements of each factor described.

Claims

1. A method for processing and analyzing in-situ electron diffraction data based on machine learning, characterized in that: The method comprises the following steps: (1) Diffraction pattern center calibration; (2) Diffraction spot detection, which includes: using the Python-based scikit-image package to detect a single diffraction spot using the Laplacian of Gaussian method and storing its position and adjacent rectangular box data; (3) Diffraction spot tracking, which includes: using the OpenCV package based on Python language and the Gunnar-Farneback dense optical flow method to calculate the displacement vector of each diffraction spot in adjacent frames, tracking the diffraction spot, and storing its position and adjacent rectangular frame data in each video frame; (4) Detection and tracking data storage, which includes: using the Python language-based xlsxwriter package to store the position of each diffraction spot in each video frame and the adjacent rectangular box data; (5) Diffraction ring calibration and analysis, which includes: clustering the radial radii of the diffraction spots obtained in step (3) using the K-means clustering method in scikit-learn based on the Python language, and using the contrast information of the corresponding diffraction spots as the weight of the clustering algorithm to obtain the diffraction ring radius that may exist in the in-situ electron diffraction video; (6) screening and analyzing the diffraction spots, which includes: counting and analyzing the changes in the radius, angle, and contrast of each diffraction spot over time, and analyzing the structural features contained therein based on the physical and chemical properties; And in said step (1), said calibration method comprises the following steps: (A) Get the mean image from the input video; (B) Based on the mean image, the Gaussian mixture model was calculated using the scikit-learn package in Python. (C) Using the Gaussian model with the largest mean among the Gaussian mixture model results, the data were fitted using the Python-based scikit-learn package to obtain the approximate outline of the transmission spot. (D) Perform minimum adjacent circle detection on the transmission spot to obtain the center point of the diffraction pattern.

2. An electron microscopy structure characterization method, characterized in that: The method comprises performing in-situ electron diffraction data processing and analysis according to the method of claim 1.

3. An electron diffraction device, characterized in that The device performs in-situ electron diffraction data processing and analysis according to the method of claim 1.

Images