Electron microscope catalyst image segmentation method based on deep learning

Abstract

The invention belongs to the technical field of image processing, and discloses an electron microscope catalyst image segmentation method based on deep learning, comprising the following steps: S1, feature extraction of an encoder; s2, restoring and segmenting the image of the decoder; s3, feature fusion of jump connection; s4, introduction of an attention module; and S5, generating a final segmentation result. According to the SGS-U-Net model, the SGS-U-Net model is used for training, data obtained through training is used for verification, the network parameter quantity and the calculated quantity are greatly reduced and the detection speed is increased under the condition that the detection precision is slightly increased finally, and facts prove that the SGS-U-Net model has excellent performance in the aspect of improving the working efficiency of the A1 2O3 representation information and has actual engineering application value.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to an electron microscope catalyst image segmentation method based on deep learning. Background Art

[0002] With the development and widespread application of information technology such as personal computers, the Internet and other communication technologies, human society has entered the information age, and the digitization of information has become the most effective method for information transmission, processing and storage. Digital images have also made great progress in the field of electron microscopy. It is an inevitable development trend of modern electron microscope image processing to convert various images obtained by electron microscope into digital signals for processing and analysis in real time. For metals / metal-oxides, their characterization structure and chemical properties are crucial to many catalytic processes, and transmission electron microscopy and scanning transmission electron microscopy can determine the distribution, morphology and orientation of aluminum oxide. Therefore, mature characterization technology is very important for the catalytic study of metal compounds. The most suitable phase for deep oxidation reaction of hydrocarbons is metal oxide phase, such as aluminum oxide.

[0003] For chemistry and materials researchers, using image processing tools to study the characterization of metallization and materials is a common technique. 2 O 3 Image information of metal compounds is very important for researchers studying Al 2 O 3 The characterization information is of great significance. Dividing the image of a metal compound into non-overlapping regions based on intensity or texture information is called image segmentation. However, it is very cumbersome and error-prone to annotate a large number of data sets by manual means. Therefore, in the field of image segmentation, automated image segmentation has been widely studied. Automated image segmentation can greatly improve work efficiency and automatically extract quantitative measurements to support decision-making. In order to solve the complex and tedious manual annotation, image segmentation algorithms based on deep learning are necessary for studying electron microscope images of catalysts. Image segmentation algorithms based on deep learning can extract key information from a large number of complex and redundant data samples and obtain relevant parameters of nanoscale particle microscope images.

[0004] At present, the traditional method for determining the activity of catalysts has the above-mentioned shortcomings. How to better preserve the information of the catalyst in the image is the focus of the present invention. Summary of the invention

[0005] In order to overcome the above technical problems, the present invention provides an electron microscope catalyst image segmentation method based on deep learning. In order to realize intelligent recognition and statistics of electron microscope image information of aluminum oxide catalyst, a model called SGS-U-Net is proposed in the method. The model is lightweight and improved on the basis of U-Net network structure. A new feature extraction network is designed by using VGG16, SE attention mechanism and ghostbottleneck module to enhance the feature extraction capability. In the decoding part of traditional U-Net network, SE attention mechanism is introduced to improve the network detection accuracy. Ghostconv is used to replace traditional convolution to solve the limitations of traditional image methods such as complex annotation, difficult measurement and low accuracy.

[0006] The present invention adopts the following technical solutions:

[0007] A deep learning-based electron microscope catalyst image segmentation method comprises the following steps:

[0008] S1. Feature extraction of encoder: The encoder effectively extracts features by downsampling four times, each downsampling halves the spatial size of the image and doubles the number of channels of the network;

[0009] Based on the VGG backbone network, the original combination of two 3x3 convolutional layers and one 2x2 maximum pooling layer is expanded to three 3x3 convolutional layers followed by a 2x2 maximum pooling layer after the second downsampling module, and the ReLU activation function is introduced.

[0010] Before the fourth downsampling module, the GhostBottleneck module is introduced to further improve the extraction accuracy of deep features;

[0011] S2. Image restoration and segmentation at the decoder: This involves four upsampling steps, each of which doubles the spatial size of the image. As upsampling proceeds, the number of network channels is halved, and the feature representation is further refined through convolutional layers.

[0012] S3. Feature fusion of skip connection: The shallow feature map rich in local details and edge information captured in the downsampling stage is effectively combined with the deep feature map containing broader context information generated in the upsampling stage;

[0013] S4. Introduction of attention module: A Squeeze-and-Excitation (SE) attention module is added between the input and output of U-Net.

[0014] S5. Generation of final segmentation results: The network performs dimensionality reduction through a 1x1 convolutional layer to reduce the number of channels to a specific number to generate the final recognition image.

[0015] Compared with the prior art, the present invention has the following beneficial effects:

[0016] The catalyst image segmentation method based on deep learning was verified using the data obtained from training. Ultimately, the network parameters and calculation amount were significantly reduced and the detection speed was improved while ensuring a slight increase in detection accuracy. 2 O 3 It performs well in characterizing information work efficiency and has practical engineering application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a network structure diagram of the catalyst image segmentation method based on deep learning provided by the present invention;

[0018] Figure 2 It is an aluminum oxide catalyst image data set designed to verify the method provided by the present invention;

[0019] Figure 3 It is the segmentation experiment result of the test set image. DETAILED DESCRIPTION

[0020] The embodiments of the present invention are described in detail below. Examples of the embodiments are shown in the accompanying drawings. Unless otherwise specified, the raw materials and equipment used can be purchased from the market or are commonly used in the art. The methods in the embodiments are conventional methods in the art unless otherwise specified. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0021] Electron microscope catalyst image segmentation method based on deep learning:

[0022] S1. Feature extraction of encoder: The encoder effectively extracts features by downsampling four times, each time halving the spatial size of the image. Correspondingly, after each downsampling, the number of channels of the network doubles to enhance the representation of image content. To further enhance the strength of feature extraction, we optimized the VGG backbone network. In particular, after the second downsampling module, we expanded the original combination of two 3x3 convolutional layers and one 2x2 max pooling layer to three 3x3 convolutional layers, followed by a 2x2 max pooling layer, and introduced the ReLU activation function. This design aims to deepen the network's understanding of image features, especially in capturing deep features of images.

[0023] In addition, before the fourth downsampling module, we introduced the GhostBottleneck module to further improve the extraction accuracy of deep features and ensure that the network can capture finer image details.

[0024] Ghostconv can achieve similar accuracy to regular convolution in some cases, but requires fewer parameters and can be calculated faster. The size of each layer of feature map of Ghostconv remains unchanged, so the step size s remains 1, and the calculation result does not need to consider the influence of the step size, so the parameter size of Ghostconv is Ghost. params The calculation of the amount of calculation is shown in formula (1) and formula (2):

[0025]

[0026]

[0027] In the formula, r is the channel expansion multiple of the second step, and r is much smaller than the number of input channels C in , the ratio of the number of parameters of Ghostconv to the number of parameters of conventional convolution is shown in equations (3) and (4):

[0028]

[0029] From the above formula, we can see that using Ghostconv can effectively reduce the number of network parameters and the amount of calculation by adjusting the size of r.

[0030] S2. Image Restoration and Segmentation in the Decoder: The purpose of the decoder is to map the high-level features extracted by the encoder back to the original image size and generate an accurate segmentation map. This process involves four upsamplings, each doubling the spatial size of the image. As the upsampling proceeds, the number of network channels is halved, and the feature representation is further refined through convolutional layers. These operations correspond to those in the encoder and help restore the detailed information of the image.

[0031] S3. Feature fusion via skip connections: The deconvolution output of the decoder is precisely aligned and integrated with the output of the corresponding level of the encoder through skip connections, achieving feature complementarity. In this process, the shallow feature maps rich in local details and edge information captured in the downsampling stage are effectively combined with the deep feature maps containing broader contextual information generated in the upsampling stage. This fusion strategy aims to retain the comprehensive information captured by the network while strengthening the perception of local features of the image.

[0032] The design of skip connections effectively alleviates the problems of gradient diffusion and gradient vanishing in deep network training. The symmetry between the encoder and decoder not only enhances the performance of skip connections, but also ensures the consistency of the spatial dimensions of the network input and output, thereby supporting an end-to-end direct classification process.

[0033] S4. Introduction of attention module: To further improve the accuracy of features, we added a SE (Squeeze-and-Excitation) attention module between the input and output of U-Net. This module enhances the network's response to image features by emphasizing the most important parts of the feature map.

[0034] S5. Generation of the final segmentation result: Finally, the network performs dimensionality reduction through a 1x1 convolutional layer to reduce the number of channels to a specific number to generate the final recognition image. This step integrates the feature information extracted by the encoder and decoder to provide accurate segmentation results for the catalyst segmentation task.

[0035] The catalyst image segmentation method based on deep learning provided by the present invention is verified below.

[0036] exist Figure 2 In order to make the experimental results of the samples observed by TEM representative, 404 images of aluminum oxide catalysts were obtained by transmission electron microscope equipment as a data set. The data set used for network training was randomly divided into training set and test set in a ratio of 8.5:1.5, as shown in the figure.

[0037] Figure 3 The visualization results of the model segmentation algorithm are given. From the figure, it can be seen that the model can accurately identify the contour information of aluminum oxide. It can be seen that the small contours detected by different models are obvious. There are certain differences in different details, but they all basically achieve the effect of image segmentation in a meaningful way, which has certain engineering application value.

[0038] Although the embodiments of the present invention have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions and variations may be made to the above embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

Claims

1. A method for electron microscope catalyst image segmentation based on deep learning, characterized in that: The following steps are involved: S1. Feature extraction of encoder: The encoder effectively extracts features by downsampling four times, each downsampling halves the spatial size of the image and doubles the number of channels of the network; Based on the VGG backbone network, the original combination of two 3x3 convolutional layers and one 2x2 maximum pooling layer is expanded to three 3x3 convolutional layers followed by a 2x2 maximum pooling layer after the second downsampling module, and the ReLU activation function is introduced. Before the fourth downsampling module, the GhostBottleneck module is introduced to further improve the extraction accuracy of deep features; S2. Image restoration and segmentation at the decoder: This involves four upsampling steps, each of which doubles the spatial size of the image. As upsampling proceeds, the number of network channels is halved, and the feature representation is further refined through convolutional layers. S3. Feature fusion of skip connection: The shallow feature map rich in local details and edge information captured in the downsampling stage is effectively combined with the deep feature map containing broader context information generated in the upsampling stage; S4. Introduction of attention module: A Squeeze-and-Excitation (SE) attention module is added between the input and output of U-Net. S5. Generation of final segmentation results: The network performs dimensionality reduction through a 1x1 convolutional layer to reduce the number of channels to a specific number to generate the final recognition image.

Images