Method for measuring the number of layers and spacing of a two-dimensional material lattice in a transmission electron microscope
Patent Information
- Application Number
- CN202311075648.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-08-25
- Publication Date
- 2026-09-18
- Estimated Expiration
- 2043-08-25
AI Technical Summary
[0003]透射电子显微镜(TEM)能够从原子尺度准确分析二维材料的结构信息，但该方法会产生大量的TEM图像数据
[0031] By adopting the above-mentioned technical solution, the present invention has the following advantages and positive effects compared with the prior art: The present invention obtains a binary image marked with layered structures through an image segmentation model, determines the pixels of the scale through image processing, obtains a skeletonized image by skeletonizing the binary image, and finally determines the position and number of pixels perpendicular to the layered structure through traversal, and obtains the number of layers and interlayer spacing in the transmission electron microscope by combining the pixels of the scale, thereby realizing the automated measurement of the number of layers and spacing of two-dimensional materials, breaking through the limitations of traditional manual analysis which is complex and time-consuming.
Smart Images
Figure CN117269540B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of material property measurement technology, and in particular to a method for measuring the number and spacing of lattice layers in a two-dimensional material using a transmission electron microscope. Background Technology
[0002] With the rapid development of the integrated circuit industry, chip manufacturing processes have reached the nanoscale. Further reduction in feature size faces the challenge of limiting power consumption, particularly due to the theoretical limit of subthreshold swing, which makes it difficult to further reduce the chip's driving voltage. Two-dimensional materials, with a thickness of only one atom or a few atoms, are ideal for realizing ultra-thin, ultra-small devices. By utilizing the unit cell structure of two-dimensional materials, the size of electronic devices can be significantly reduced, achieving higher integration density and faster response speeds. Therefore, understanding and studying the properties of two-dimensional materials is a hot topic in integrated circuit development.
[0003] Transmission electron microscopy (TEM) can accurately analyze the structural information of two-dimensional materials at the atomic scale, but this method generates a large amount of TEM image data. Furthermore, accurate analysis of TEM structural images requires researchers to possess strong professional knowledge and experimental experience, and is also very time-consuming. Therefore, improving the efficiency of TEM structural image analysis is an urgent problem to be solved. Summary of the Invention
[0004] The technical problem to be solved by the present invention is to provide a method for measuring the number of lattice layers and spacing of two-dimensional materials in a transmission electron microscope, so as to realize the automated measurement of the number of layers and spacing of two-dimensional materials.
[0005] The technical solution adopted by this invention to solve its technical problem is: to provide a method for measuring the number and spacing of lattice layers in a two-dimensional material using a transmission electron microscope, comprising the following steps:
[0006] Receive the transmission electron microscope image to be measured;
[0007] The transmission electron microscope image to be measured is input into the target image segmentation model to obtain a binarized image marked with layered structures;
[0008] Image processing is performed on the transmission electron microscope image to be measured to determine the number of pixels occupied by the scale in the transmission electron microscope image to be measured.
[0009] A skeletonized image is obtained by processing a binary image marked with a layered structure using a skeleton extraction algorithm.
[0010] The pixels in the skeletonized image are traversed in the direction of the vertical layered structure.
[0011] Determine the position and number of directional pixels in the vertical layered structure, and calculate the number of layers and interlayer spacing in the transmission electron microscope by combining the pixels occupied by the scale.
[0012] The target image segmentation model is obtained in the following way:
[0013] Collect transmission electron microscope images used for training;
[0014] The layered and non-layered regions in the transmission electron microscope image used for training are marked, and the non-layered regions are set as the background color to obtain a binarized image.
[0015] An image segmentation model is constructed, and the image segmentation model is trained using the training transmission electron microscope (TEM) image and the corresponding binarized image as the training TEM image as the dataset, thereby obtaining the target image segmentation model.
[0016] The image segmentation model is constructed by:
[0017] Construct an initial image segmentation model;
[0018] Adjust the structure of the initial image segmentation model;
[0019] Set the loss function for the image segmentation model.
[0020] The initial image segmentation model is built based on FCN, U-Net, or Deep-Lab.
[0021] The adjustment of the initial image segmentation model structure specifically involves: replacing the downsampling convolution block of the initial image segmentation model; adding a regularization layer; and changing the base number of the downsampling convolution block.
[0022] The loss function of the image segmentation model is l = l c_w +0.5×l d Where l is the loss function of the image segmentation model, l c_w Let l be the cross-entropy loss function. d Let be the Dess loss function.
[0023] The step of image processing on the transmission electron microscope (TEM) image to be measured, to determine the pixel information of the scale bar in the TEM image to be measured, includes:
[0024] The transmission electron microscope image to be measured is subjected to Gaussian blurring.
[0025] Determine the pixel threshold, and perform binarization processing on the Gaussian blurred transmission electron microscope image to be measured based on the pixel threshold to obtain a binarized image;
[0026] The binarized image was detected using Hough change detection to obtain a binarized image containing only the scale.
[0027] Determine the number of pixels occupied by the scale in the binarized image containing only the scale.
[0028] The pixel threshold is 200-255.
[0029] The process of using a skeleton extraction algorithm to process a binarized image marked with a layered structure refers to replacing the layered structure in the binarized image with a line of single-pixel width at the center point of the layered structure.
[0030] Beneficial effects
[0031] By adopting the above-mentioned technical solution, the present invention has the following advantages and positive effects compared with the prior art: The present invention obtains a binary image marked with layered structures through an image segmentation model, determines the pixels of the scale through image processing, obtains a skeletonized image by skeletonizing the binary image, and finally determines the position and number of pixels perpendicular to the layered structure through traversal, and obtains the number of layers and interlayer spacing in the transmission electron microscope by combining the pixels of the scale, thereby realizing the automated measurement of the number of layers and spacing of two-dimensional materials, breaking through the limitations of traditional manual analysis which is complex and time-consuming. Attached Figure Description
[0032] Figure 1 This is a flowchart of a method for measuring the number and spacing of lattice layers in a two-dimensional material using a transmission electron microscope, according to an embodiment of the present invention.
[0033] Figure 2 This is a schematic diagram of the U-Net structure;
[0034] Figure 3 This is a diagram illustrating the changes to the U-Net structure;
[0035] Figure 4 It is a map showing the distribution of marked pixels and all pixels;
[0036] Figure 5 This is a schematic diagram of the prediction results after changing the U-Net structure;
[0037] Figure 6 This is a flowchart illustrating the calibration method in an embodiment of the present invention;
[0038] Figure 7 This is a schematic diagram of the skeletonization process in an embodiment of the present invention. Detailed Implementation
[0039] The present invention will be further illustrated below with reference to specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, it should be understood that after reading the teachings of this invention, those skilled in the art can make various alterations or modifications to the invention, and these equivalent forms also fall within the scope defined by the appended claims.
[0040] The embodiments of the present invention relate to a method for measuring the number and spacing of lattice layers in a two-dimensional material using a transmission electron microscope, such as... Figure 1 As shown, it includes the following steps:
[0041] Step 1: Receive the transmission electron microscope image to be measured.
[0042] Step 2: Input the transmission electron microscope image to be measured into the target image segmentation model to obtain a binarized image marked with layered structures.
[0043] In this step, the target image segmentation model can be obtained in the following way:
[0044] Collect transmission electron microscope images used for training;
[0045] The layered and non-layered regions in the transmission electron microscope image used for training are marked, and the non-layered regions are set as the background color to obtain a binarized image.
[0046] An image segmentation model is constructed, and the image segmentation model is trained using the training transmission electron microscope (TEM) image and the corresponding binarized image as the training TEM image as the dataset, thereby obtaining the target image segmentation model.
[0047] This embodiment of the image segmentation model includes: constructing an initial image segmentation model; adjusting the structure of the initial image segmentation model; and setting the loss function of the image segmentation model.
[0048] The initial image segmentation model is built based on FCN, U-Net, or Deep-Lab. Taking U-Net as an example... Figure 2 The structure of the classic U-net network model is shown. Overall, the network has a "U" structure. The most important structural features of U-net are downsampling, upsampling, and long-hop connections. Therefore, it is possible to retain this topology and adjust the backbone network.
[0049] In this embodiment, adjusting the structure of the initial image segmentation model specifically involves: replacing the downsampling convolutional block of the initial image segmentation model; adding a regularization layer; and changing the base number of channels in the downsampling convolutional block. For example... Figure 3 As shown, referencing VggNet and MobileNet, this implementation designs several alternative U-net downsampling convolutional blocks. All three structures incorporate a batch normalization layer. The advantage of this is that the data passing through this layer is normalized, which helps with gradient control during network training. The mean of the data is set to zero and the standard deviation is normalized, which helps prevent gradient explosion or vanishing phenomena. Figure 3(a) shows a modified convolutional block with two convolutions after referencing VggNet and adding a BN layer, with the ReLU layer placed after the BN layer. The number in the upper right corner represents the number of channels in the feature map, corresponding to the parameters of the first downsampled convolutional block. The base number of channels is 64, and this number is doubled with each subsequent downsampling up to 1024. Then, it is gradually reduced back to 64 during the upsampling stage. Finally, a 1×1 convolution is used to convert the number of image channels into the number of classification categories. Figure 3 (b) shows a modified convolutional block with two convolutions, based on MobileNet and with the addition of a BN layer. It still follows the "convolutional layer-BN layer-ReLU" structure. The convolutional structure here uses the depthwise separable convolution from MobileNet, dividing the convolution into depthwise convolution and pointwise convolution, reducing computation while maintaining approximate recognition accuracy. The base number of channels is still 64; the subsequent changes in the number of channels are similar to... Figure 3 The same description is given in (a). Figure 3 (c) illustrates a modified convolutional block with two convolutions after reducing the base number of channels and adding a BN layer. The ReLU layer is positioned the same as in the previous two structures. This convolutional block is similar to the original Unet structure, the main difference being that the base number of channels is reduced from 64 to 32. Each subsequent downsampling also doubles the number of channels, reaching 512 before upsampling begins. During upsampling, the number of channels is restored to 32, and finally, the number of channels becomes the number of classification categories. In later experiments, the Unet networks using these three convolutional blocks were named Vgg-Unet, Mobile-Unet, and bc32-Unet, respectively. In the original Unet scheme, the feature map size gradually decreases with each convolution operation, and during the upsampling skip connections, cropping is required to address the issue of different sizes. In this implementation, all convolution operations include padding, which helps maintain the feature map size before and after convolution.
[0050] The target image segmentation model in this implementation belongs to binary image segmentation. Specifically, it labels each pixel of the image as either a layered region or the background. When training the network model, a loss function is used to compare the difference between the network's predictions and the actual values. First, we consider the most commonly used cross-entropy loss function, which is as follows:
[0051]
[0052] This formula shows the cross-entropy loss function for N categories, y n and p n These correspond to the true value and the predicted label, respectively. In the code implementation of the formula, y is often replaced by a hot-coded vector. n That is, except for the position of its own category which is 1, all other positions are 0, p n Use yn The probability of the corresponding category is used instead. However, if the number of categories is unbalanced, the cross-entropy loss function will cause the network to favor the majority category.
[0053] like Figure 4 As shown, the layered structures and background regions of interest in both the molybdenum disulfide and black phosphorus datasets exhibit an imbalanced class relationship, meaning that the number of background pixels is greater than the number of pixels in the layered regions. In the molybdenum disulfide dataset, layered region pixels account for 7.2% of all pixels, while in the black phosphorus dataset, layered region pixels account for 30.3% of all pixels.
[0054] For datasets with uneven class distributions, a weighted cross-entropy loss function can be used. This function assigns weights to different classes based on their frequency of occurrence. The weighted cross-entropy loss function satisfies the following formula:
[0055]
[0056] The above formula adds a weight w before the corresponding category. n The advantage of using this loss function is that:
[0057] 1. By assigning high weights to low-frequency categories, the model can be effectively prevented from ultimately favoring high-frequency categories, thus achieving a better balance between low-frequency and high-frequency categories.
[0058] 2. By assigning higher weights to these hard-to-classify regions, the network can more accurately predict certain hard-to-classify regions.
[0059] 3. By assigning higher weights to more important categories, the network can perform better segmentation on important categories.
[0060] This weight can be determined based on the frequency of the category in the entire dataset, or it can be modified based on the importance of the category. In the 2D material transmission electron microscopy dataset, there are cases where the pixel values of layered regions and the background differ significantly, and the layered regions are considered more important than the background. In other words, the sparse but more important 2D material layered regions in this dataset should receive a larger weight.
[0061] Observing the characteristics of this type of dataset, it's easy to see that the foreground regions are small and elongated. Therefore, the network's ability to accurately predict foreground and background edges is crucial for this segmentation task. Thus, the Dyess loss function can be considered. The Dyess loss function evaluates the difference between the true and predicted values by calculating the similarity between two sets. The Dyess loss function satisfies the formula:
[0062]
[0063] In the above formula, the numerator calculates twice the intersection of the two sets, and the denominator calculates the sum of the two sets. The advantage of using the Desce loss function is that, compared to the cross-entropy loss function, the Desce loss function penalizes false negatives more severely than false positives. This makes the model try to avoid identifying foreground regions as background, which is very useful when dealing with two-dimensional material datasets with long, narrow foregrounds. However, during training, excessive gradients may occur, affecting training. Therefore, considering all factors, a new loss function can be constructed by combining a weighted cross-entropy loss function with a certain proportion of the Desce loss function. The relationship between the two satisfies the formula: l = l c_w +0.5×l d , where l c_w Let l be the cross-entropy loss function. d Let be the Dess loss function.
[0064] like Figure 5 As shown, the above section constructs three models: Vgg-Unet, Mobile-Unet, and bc32-Unet. Figure 5 This shows the trend of the dice coefficient on the test set of two datasets as a function of the training epochs. The horizontal axis represents the training epoch, and the vertical axis represents the dice coefficient of the network on the test set after each training epoch. Figure 5 (a) Corresponding to the MoS2 dataset. Figure 5 (b) Performance of the three networks on the BP dataset. On the MoS2 dataset, Vgg-Unet and bc32-Unet quickly reached a high score, while Mobile-Unet took several epochs longer to reach a similar stable score. This was even more pronounced on the Black Phosphorus dataset; although all three networks eventually reached a relatively stable score, Mobile-Unet required more epochs of training. Vgg-Unet and bc32-Unet achieved similar stable scores, while Mobile-Unet's scores were slightly lower. The table shows the performance of the three networks on the MoS2 and BP datasets. bc32-Unet and Vgg-Unet performed similarly in the dice coefficient, IoU, and mean IoU metrics, while Mobile-Unet lagged behind these two networks in each metric, with a difference of approximately 2%-4%. However, Mobile-Unet uses depthwise separable convolutions, saving on the number of parameters required for computation; it requires the fewest parameters among the three networks. Although bc32-Unet and Vgg-Unet are similar in performance, Vgg-Unet has twice as many channels in its convolutional blocks as bc32-Unet, so the number of parameters that need to be calculated during training and prediction is much greater for Vgg-Unet.
[0065] Table 1 shows the performance of the three network models on two datasets.
[0066]
[0067] When computational resources are limited, Mobile-Unet, which sacrifices a small amount of accuracy for reduced computation, is worth considering. Compared to bc32-Unet with 32 basic channels, Vgg-Unet, with 64 basic convolutional layer channels, does not offer a performance improvement but instead consumes more computation. Generally, increasing the number of channels in a convolutional layer will increase the amount of features extracted by the network. On these two datasets, very deep convolutional layers are sufficient to extract the feature information needed for foreground segmentation.
[0068] Step 3: Perform image processing on the transmission electron microscope image to be measured to determine the number of pixels occupied by the scale in the transmission electron microscope image to be measured.
[0069] like Figure 6 As shown, this step specifically includes:
[0070] (1) The transmission electron microscope image to be measured is subjected to Gaussian blurring.
[0071] Gaussian blur is a commonly used image processing technique for smoothing images and reducing noise. It can effectively remove noise from images while preserving the overall structure and edge information. Gaussian blur can reduce details in an image, extracting features that emphasize overall shape and structure. It makes object shapes clearer, allowing for better boundary extraction and shape analysis.
[0072] (2) Determine the pixel threshold, and perform binarization processing on the Gaussian blurred transmission electron microscope image to be measured based on the pixel threshold to obtain a binarized image.
[0073] Binarization is an image processing technique that converts an image into a binary image containing only two pixel values, often black and white. The purpose of binarization is to segment pixels in a grayscale image according to a set threshold, setting pixels with brightness above the threshold to white and pixels with brightness below the threshold to black, thus obtaining a binary image with only two grayscale values. In this embodiment, the pixel threshold can be selected from 200-255. In the binarized image, foreground and background pixels are clearly distinguishable, and the contour and shape information of objects can be extracted by the differences in pixel values. Binarization converts the grayscale information in the image into logical values 0 and 1, making it easier to extract and describe the features of the image.
[0074] (3) Use Hough change detection to detect the binarized image to obtain a binarized image containing only the scale.
[0075] The Hough transform is a commonly used image processing technique for detecting specific shapes such as lines and circles in images. It is widely applied in computer vision, image processing, and pattern recognition. The basic idea of the Hough transform is to represent specific shapes in an image as curves or surfaces in parameter space, thus transforming shape detection into finding specific curves or surfaces in parameter space. The Hough transform can be used to detect the presence of straight lines in images, without being limited by the length, direction, or position of the line. In this embodiment, the Hough transform is used to perform parameter space voting on the edges in the image, thereby finding the best-fitting straight line. The Hough transform also makes the scale bar in the image clearer.
[0076] (4) Determine the number of pixels occupied by the scale in the binarized image containing only the scale.
[0077] Step 4: Use a skeleton extraction algorithm to process the binarized image marked with layered structures to obtain a skeletonized image.
[0078] Image skeletonization is an image processing technique that simplifies elongated structures in an image to their finest representation by extracting the main skeleton or central axis. The goal of image skeletonization is to preserve the structural and shape features of objects in an image while removing redundant details and edges. It is widely used for tasks such as object extraction, shape matching, shape analysis, and image compression.
[0079] The process of using the skeleton extraction algorithm to process the binarized image marked with layered structures in this step means replacing the layered structures in the binarized image with a line of single-pixel width at the center point of the layered structure. Figure 7 It is a skeletonized image obtained after the above skeletonization process.
[0080] Step 5: Traverse the pixels in the skeletonized image along the direction of the vertical layer structure. By traversing the effectively identifiable layer structure in the skeletonized image, the distance between each line in the skeletonized image and the number of lines can be obtained.
[0081] Step 6: Determine the position and number of directional pixels of the vertical layered structure, and calculate the number of layers and interlayer spacing in the transmission electron microscope by combining the pixels occupied by the scale.
[0082] It is easy to see that this invention obtains a binary image marked with layered structures through an image segmentation model, determines the pixels of the scale through image processing, obtains a skeletonized image by skeletonizing the binary image, and finally determines the position and number of pixels perpendicular to the layered structure through traversal. Combined with the pixels of the scale, the number of layers and the interlayer spacing in the transmission electron microscope are obtained, thereby realizing the automated measurement of the number of layers and the interlayer spacing of two-dimensional materials, breaking through the limitations of traditional manual analysis which is complex and time-consuming.
Claims
1. A method for measuring the number and spacing of lattice layers in a two-dimensional material using a transmission electron microscope, characterized in that, Includes the following steps: Receive the transmission electron microscope image to be measured; The transmission electron microscope image to be measured is input into the target image segmentation model to obtain a binarized image marked with layered structures; wherein, the target image segmentation model is obtained in the following way: Collect transmission electron microscope images used for training; The layered and non-layered regions in the transmission electron microscope image used for training are marked, and the non-layered regions are set as the background color to obtain a binarized image. An image segmentation model is constructed, and the image segmentation model is trained using the training transmission electron microscope image and the corresponding binarized image as the dataset to obtain the target image segmentation model. The process of constructing the image segmentation model includes: constructing an initial image segmentation model; adjusting the structure of the initial image segmentation model; setting the loss function of the image segmentation model; the initial image segmentation model is constructed based on FCN, U-Net, or Deep-Lab; Image processing is performed on the transmission electron microscope image to be measured to determine the number of pixels occupied by the scale in the transmission electron microscope image to be measured. A skeletonized image is obtained by processing a binary image marked with a layered structure using a skeleton extraction algorithm. The pixels in the skeletonized image are traversed in the direction of the vertical layered structure. Determine the position and number of directional pixels in the vertical layered structure, and calculate the number of layers and interlayer spacing in the transmission electron microscope by combining the pixels occupied by the scale.
2. The method for measuring the number and spacing of lattice layers in a two-dimensional material using a transmission electron microscope according to claim 1, characterized in that, The adjustment of the initial image segmentation model structure specifically involves: replacing the downsampling convolution block of the initial image segmentation model; adding a regularization layer; and changing the base number of the downsampling convolution block.
3. The method for measuring the number and spacing of lattice layers in a two-dimensional material using a transmission electron microscope according to claim 1, characterized in that, The loss function of the image segmentation model is ,in, The loss function for the image segmentation model is... Let cross-entropy be the loss function. Let be the Dess loss function.
4. The method for measuring the number and spacing of lattice layers in a two-dimensional material using a transmission electron microscope according to claim 1, characterized in that, The step of image processing on the transmission electron microscope (TEM) image to be measured, to determine the pixel information of the scale bar in the TEM image to be measured, includes: The transmission electron microscope image to be measured is subjected to Gaussian blurring. Determine the pixel threshold, and perform binarization processing on the Gaussian blurred transmission electron microscope image to be measured based on the pixel threshold to obtain a binarized image; The binarized image was detected using Hough change detection to obtain a binarized image containing only the scale. Determine the number of pixels occupied by the scale in the binarized image containing only the scale.
5. The method for measuring the number and spacing of lattice layers in a two-dimensional material using a transmission electron microscope according to claim 4, characterized in that, The pixel threshold is 200-255.
6. The method for measuring the number and spacing of lattice layers in a two-dimensional material using a transmission electron microscope according to claim 1, characterized in that, The process of using a skeleton extraction algorithm to process a binarized image marked with a layered structure refers to replacing the layered structure in the binarized image with a line of single-pixel width at the center point of the layered structure.