Catalytic metal particle electron microscope image recognition method based on density estimation

Abstract

The invention belongs to the field of application of artificial intelligence in physical and chemical research, and provides a catalytic metal particle electron microscope image recognition method based on density estimation, which comprises the following steps: sending an electron microscope image into a backbone network for feature extraction to generate feature maps with different sampling multiplying power; inputting the feature maps with different scales into an expansion feature pyramid network, and carrying out multi-level feature fusion; sending the data to a self-attention expansion pyramid pooling module, and performing up-sampling to obtain a predicted density map; performing reverse distance transformation on the input original image to obtain a density map true value; using the predicted density map and the true value density map to calculate strong and weak loss, and weighting the two to jointly supervise the training process; performing local maximum positioning on the density map predicted by the final model through a prediction density map target positioning algorithm to obtain a target position coordinate; and finally, calculating to obtain a density and distance distribution curve of the monatomic, and marking the monatomic target in an original image. According to the invention, high-precision identification of the electron microscope image of the catalytic metal particles can be realized.

Description

Technical Field

[0001] The present invention belongs to the field of application of artificial intelligence in physical chemistry research, involves deep learning and computer vision content, and particularly relates to a catalytic metal particle electron microscope image recognition method based on density estimation. Background Art

[0002] The performance of catalytic materials depends on their microstructures, which reflect their physical and chemical properties. When studying the effectiveness of catalytic materials, characterization methods can be used as important tools to evaluate the activity of catalysts. Microstructural properties such as the number of single atoms of catalytic metal particles, carrier crystal form, spatial distribution, and density have a significant impact on the catalytic process and effect. At present, researchers mainly use transmission electron microscopy to obtain microscopic images of catalytic materials at the nanoscale and perform statistical analysis on nanoparticles in nanometal catalysts. However, this not only consumes a lot of human resources but is also inefficient. In addition, due to the subjectivity of visual identification by statisticians, the statistical results of different personnel may vary greatly, which can easily lead to significant quantitative deviations.

[0003] Target detection is the identification of objects in images. The purpose is to classify and locate objects, and obtain the category and positioning coordinates of the objects. However, small targets occupy less pixel area, have unclear features, and are easily disturbed by the background or blocked by larger targets. At the same time, their detailed features are often lost in the low-resolution layer of the network, which makes it difficult for traditional detectors to extract their effective features and difficult to detect. The single-atom electron microscope image is identified by a positioning method based on density estimation to obtain the accurate position coordinates of catalytic metal particles, and then the image processing capabilities of the computer are used to perform statistical analysis on its characterization. It can not only save a lot of manpower and financial costs, but also use massive data to quantify structural features and large sample statistics, significantly improving the quantitative accuracy and quantitative dimension of microstructure information analysis. At the same time, it can also avoid the differences in data quality obtained by personnel operating electron microscopes, effectively overcome the subjective bias of manual analysis, and the lack of representativeness of electron microscope micro-area analysis, making the acquired data more comprehensive and extensive. Summary of the invention

[0004] The technical problem to be solved by the present invention is to make up for the insufficient application of current deep learning technology in the field of intelligent physical chemistry research, and propose a density estimation-based electron microscope image recognition method for catalytic metal particles, so as to achieve high-precision recognition of electron microscope images of catalytic metal particles under specific conditions.

[0005] The technical solution of the present invention is as follows:

[0006] A method for electron microscope image recognition of catalytic metal particles based on density estimation, the steps are as follows:

[0007] Step (1) using the true value point annotation to perform reverse distance transformation on the input original image to obtain the true value of the density map;

[0008] For the input electron microscope image of catalytic material, the inverse distance transform is used to generate the density map, and the inverse of the distance from the pixel to the nearest adjacent target is used as the probability distribution of the target, and the density map is obtained by the true value.

[0009] In the widely used process of generating Gaussian kernel density maps, the size of the target is usually estimated based on the average distance of neighboring targets, and the width of the Gaussian kernel is adjusted to generate the probability distribution of the corresponding area. However, the distribution of targets (such as dense or sparse) may lead to errors in size estimation, which in turn cause the problem of overlapping probability distribution areas. Based on this, the inverse distance transform is used to replace the traditional Gaussian kernel to generate density maps. The inverse of the distance from the pixel to the nearest neighboring target is used as the probability distribution of the target. The closer to the center point of the target, the larger the value; while the farther away from the center point, the value decreases rapidly. Due to the characteristics of the inverse distance transform, the density map can better separate the targets, so that the local maximum point directly corresponds to the center of the target.

[0010] Step (2) inputs the electron microscope image of the catalytic material and sends it to the feature extraction network for feature extraction to generate feature maps with different sampling magnifications; the details are as follows:

[0011] The electron microscope image of the catalytic material is sent to the feature extraction network for feature extraction to generate feature maps with different sampling rates. The feature extraction network uses the Swin-Transformer network structure as the backbone. After the electron microscope image of the catalytic material is input into the Swin-Transformer network, the preliminary feature extraction is first performed through the block division module to reduce the size of the feature map and expand the number of feature channels. Then, deep feature extraction is performed through layer-by-layer convolution and Transformer modules. The Swin-Transformer network draws on the multi-scale feature extraction structure of the classic CNN network. At each feature scale, the feature map first passes through the block merging module, which is similar to the pooling operation in the convolution, and the width and height of the feature map are downsampled by 2 times, while the number of channels is doubled. The feature map is then input into a Transformer module consisting of grouped window attention W-MSA and shifted window attention SW-MSA to focus on global feature information; in grouped window attention W-MSA, the feature map is divided into non-overlapping windows of the same size, and self-attention is calculated separately in each window; while in shifted window attention SW-MSA, the window is translated and the self-attention of the features in the translated window is calculated, thereby capturing feature associations across windows.

[0012] Step (3) inputs the feature maps of different scales output by the feature extraction network into the dilated feature pyramid network to perform multi-level feature fusion; the details are as follows:

[0013] First, the feature map pos n of the feature extraction network after the maximum downsampling is input into the dilated feature pyramid network, where n is related to the downsampling multiple of the selected feature extraction network. The feature map pos n passes through the dilated feature fusion module (which consists of two parallel branches, one is two stacked dilated convolutions with different dilation rates, and the other is a normal convolution. The features output by the two branches are spliced ​​on the channel and fused through convolution), and then upsampled to expand its feature size to 2 times the original size. The upsampled feature map and the feature map of the same size output by the feature extraction network are weighted summed. The output feature map is pos(n-1), and the weights are optimized as the model is trained. Repeat the above steps, the output feature map is continuously expanded feature fusion, upsampled, and weighted summed with the corresponding size output of the feature extraction network to fuse the target features at different sizes and improve the network information capture capability. Finally, the feature map pos1 after the weighted summation of the last layer is output.

[0014] The dilated convolution increases the network's receptive field, enhances the network's understanding of contextual information, and is more conducive to capturing the positional relationship of single-atom targets in the entire image. The feature pyramid network fuses features of different scales output by the feature extraction network to improve the detection effect of targets of different sizes. In addition, the extraction of discrete feature points also reduces the feature loss of small targets and retains more details for dense target prediction.

[0015] Step (4) After layer-by-layer upsampling, the feature map finally output by the expanded feature pyramid network is sent to the self-attention expanded pyramid pooling module, and the predicted density map is obtained after upsampling; the details are as follows:

[0016] The fused feature map pos1 is divided into three branches, which are parallelized by three dilated convolutions with different dilation coefficients, and then spliced ​​to obtain the feature map pos(n+1). The three-branch pyramid pooling module has filters with multiple complementary fields of view to detect the input feature map, which not only captures different group characteristics at different scales, effectively makes up for the representation differences under different spatial scales, but also enriches the target information. Then the feature map pos1 is introduced into the channel self-attention branch, and the channel weight is obtained through global average pooling, fully connected layer and sigmoid activation function. The spliced ​​feature map pos(n+1) is weighted to obtain the feature map pos(n+2) to adaptively strengthen the focus on the key channels and reduce the information redundancy caused by multi-channel aggregation. The feature map pos1 is then introduced to directly add the feature map pos(n+2) to obtain the feature map pos(n+3), which maintains the original features of the input and enhances the training stability. Finally, the feature map pos(n+3) is passed through two consecutive upsampling layers to obtain the predicted density map pos(n+4) of the same size as the original image.

[0017] Step (5) locating the target in the predicted density map by using a predicted density map target positioning algorithm;

[0018] In order to obtain the single-atom position information from the density map, the target in the predicted density map is located. Since there is no Gaussian spot overlap problem in the traditional density map, the local maximum point is the predicted single-atom center. Through scale-invariant maximum pooling, all global candidate positions can be obtained. Then, an adaptive threshold is designed to remove false positive targets that appear in the prediction and obtain the true target position. At the same time, for samples without targets, false detection caused by background noise can be easily avoided.

[0019] Step (6) uses the predicted density map and the true value density map to calculate the strong and weak losses, and the two are weighted to jointly supervise the training process of the density estimation recognition model network constructed by steps (2) to (4);

[0020] The loss function consists of two parts: strong supervision and weak supervision. For strong supervision loss, the true density map is directly used to constrain the density map predicted by the network at the pixel level. First, the MSE loss is used to calculate the square of the pixel difference between the predicted density map and the true density map pixel by pixel, and average it over the entire image. Because as a strong supervision method, on the one hand, the MSE loss is continuous and smooth, which ensures the stability of training. On the other hand, the MSE loss has a larger penalty and is more sensitive to background noise, making the model more generalizable. In addition, in the strong supervision part, the SSIM loss, which is commonly used in image quality assessment, is also used to further improve the quality of the predicted image. Compared with the MSE loss, the SSIM loss pays more attention to the spatial distribution of the target rather than the color difference. The combination of the two losses can make the model pay attention to both pixel-level matching and structural similarity, thereby obtaining better performance. In the weak supervision part, the counting loss is used to supervise the training. Because the number is not the direct feature information in the density map, it is a form of weak supervision and plays an auxiliary role in training.

[0021] Step (7) locates the local maximum of the final predicted density map to obtain the target position coordinates;

[0022] After the joint training of strong and weak supervision loss in step (6), the network parameters of steps (2) to (4) are updated, and the output is a predicted density map that is closer to the true value of the annotation. Through the positioning method of step (5), all local maximum points in the predicted density map are obtained by using local maximum pooling, that is, the initial screening target points. Then, the screening threshold is designed according to the maximum pixel value of the whole image to remove the false positive targets that appear in the prediction and obtain the true target position coordinates.

[0023] Step (8) constructing a post-processing module, sending the target position coordinates into the post-processing module, calculating the density and distance distribution curve of the single atom, and marking the single atom target in the original image and saving it;

[0024] The post-processing module is responsible for processing the positioning results; marking the target position coordinates into the original image, and using the distance matrix to calculate the density and distance distribution curve of the single atom according to the carrier area and scale ratio, as the basis for the characterization and analysis of the catalytic performance.

[0025] The beneficial effects of the present invention are as follows: the method uses a density estimation method to solve the problem that it is difficult to efficiently and accurately identify catalytic metal single-atom particles, which are small in size and densely distributed, through the target detection model; the density map generation method based on the inverse distance transform solves the problem that the Gaussian density kernels overlap each other due to the uneven distribution of the target and can only be counted but not located; in addition, an expansion feature pyramid network is designed in the network decoding process, and useful image context information is captured through multi-scale adaptive capture, which can maintain long-range dependence while paying attention to fine-grained details of the edge, thereby effectively coping with scale changes under different observation scales in the electron microscope image. The expansion convolution improves the network's understanding of context information by expanding the network's receptive field, thereby more effectively capturing the positional relationship of single-atom targets in the entire image. The designed self-attention expansion pyramid pooling not only maintains the transmission of the original features, but also modulates the features to focus on important channels and strengthen feature aggregation. In this way, tiny single-atom targets can be effectively detected, which greatly improves the problem of poor recognition effect and omission in complex scenes; using the strong and weak joint loss, the quantitative relationship and spatial relationship of the targets are considered, and they complement each other to improve the prediction effect of the density map, thereby improving the recognition effect of the final single-atom target; BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is an overall flow chart of the method of the present invention.

[0027] Figure 2 This is the framework diagram of the expanded feature pyramid.

[0028] Figure 3 Schematic diagram of the expansion polymerization module.

[0029] Figure 4 Diagram of the self-attention dilated pyramid pooling framework.

[0030] Figure 5 Schematic diagram of image recognition effect in the embodiment. DETAILED DESCRIPTION

[0031] The specific implementation of the present invention is further described below in conjunction with the accompanying drawings and technical solutions.

[0032] The basic process of the electron microscope image recognition method for catalytic metal particles based on density estimation provided by the present invention is as follows: Figure 1 This example uses the example segmentation of an electron microscope image of a catalytic material, and the specific segmentation process is as follows:

[0033] Step (1) performing a reverse distance transformation on the input original image to obtain the true value of the density map;

[0034] For the input electron microscope image of catalytic material, the inverse distance transform is used to generate the density map, and the inverse of the distance from the pixel to the nearest adjacent target is used as the probability distribution of the target, and the density map is obtained by the true value.

[0035] Step (2) inputting an electron microscope image of the catalytic material and sending it to a backbone network for feature extraction to generate feature maps with different sampling magnifications;

[0036] The feature extraction network uses the Swin-Transformer network structure as the backbone. After the electron microscope image of the catalytic material is input into the network, the preliminary feature extraction is first performed through the block division module, and the output feature map size is 1 / 4 of the original image, and the number of channels becomes 48. Then, deep feature extraction is performed through layer-by-layer convolution and Transformer modules; the Swin-Transformer network draws on the multi-scale feature extraction structure of the classic CNN network. In the first feature scale, the feature map first passes through the linear mosaic module to downsample the width and height of the feature map by 2 times, and the number of channels becomes C. The feature map is then input into the Transformer self-attention module composed of grouped window attention and shifted window attention to focus on global feature information; in the grouped window attention, the feature map is divided into non-overlapping windows of the same size, and the self-attention is calculated separately in each window; in the shifted window attention, the window is translated, and the self-attention of the features in the translated window is calculated, thereby capturing the feature association across windows. In each layer of the module, the feature map is first input into the block merging module, which is similar to the pooling operation in convolution. The width and height of the feature map are downsampled by 2 times, and the number of channels is doubled, and then sent to the Transformer self-attention module. After multiple layers are stacked, the feature map is continuously downsampled and the number of channels is doubled.

[0037] In this way, local and global features are effectively extracted while improving computational efficiency. Each layer of attention module is followed by a multi-layer perceptron (MLP) module, and the two are stacked as the basic structure to form the entire network. 1 / 4, 1 / 8, 1 / 16 and 1 / 32 features of the input image size are extracted from the 2nd, 4th, 6th and 9th layers of the Swin-Transformer network as outputs, and the corresponding number of channels is 128, 256, 512 and 1024 channels respectively.

[0038] Step (3) Input the different scale feature maps output by the feature extraction network into the expanded feature pyramid network (such as Figure 2 As shown), perform multi-level feature fusion;

[0039] In the feature extraction stage, the backbone network outputs four feature maps of different scales from each image, with sampling steps of 4, 8, 16, and 32. First, the high-level features with a step size of 32 are input into an expansion aggregation module. Figure 3 As shown in the figure, in the expansion aggregation module, the feature map first passes through a 1x1 convolution, and then is divided into two branches, one passes through two stacked 3x3 convolutions with expansion rates of 1 and 2 respectively, and the other passes through a 1x1 convolution. Finally, the two output features are concatenated on the channel and fused through a 1x1 convolution. The fused feature map is then upsampled and summed with the feature map with a stride of 16. This process is repeated, and feature details are learned from multiple scales from 16 to 4 times the sampling step, and finally a 4-fold downsampled high-resolution low-level feature map is obtained.

[0040] Step (4) The feature pyramid outputs the maximum scale feature map, which is sent to the self-attention expansion pyramid pooling module (such as Figure 4 As shown), the predicted density map is obtained after upsampling;

[0041] The fused low-level features are parallelized through 3×3 convolutions with dilation rates of 3, 6, and 9, and then concatenated and fused. In addition, the concatenated feature map is compressed through global average pooling to obtain a 1×1×C output. Next, the input passes through two consecutive fully connected layers, first reducing the number of channels to C / 16 and then restoring it to C, thereby reducing the number of parameters. Finally, the Sigmoid function is used for activation, converting the values ​​of each channel to a range of 0 to 1, and weighting the concatenated features. Finally, the original low-level features of the input are directly connected to the residual and added to the weighted features to retain the original information.

[0042] Step (5) Prediction of density map target positioning algorithm

[0043] First, the feature map F is calculated using the maximum filter k,l The local maximum under the window size w, the local maximum map is recorded as L i,j .

[0044]

[0045] Where (i, j) is the coordinate point in the feature map, and (k, l) is the coordinate of the feature point in the local window w.

[0046] Then, the local maximum points are filtered using a preset threshold, where the threshold P is the maximum value of all local maximum values ​​multiplied by the scale factor α, that is,

[0047] P = α × max (L i,j )

[0048] Among them, the optimal effect is achieved by taking the proportional factor α as 0.4 through experiments.

[0049] Finally, the local maximum point greater than the threshold is taken as the true predicted single-atom target, and the coordinates (i, j) are saved.

[0050] Step (6) uses the predicted density map and the true value density map to calculate the strong and weak losses, and the two are weighted to jointly supervise the training process of the overall network;

[0051] For the strong supervision loss, on the one hand, the mean square error (MSE) loss is used to calculate the square of the pixel difference between the predicted density map and the true density map pixel by pixel and take the average over the entire image.

[0052]

[0053] In the formula, represents the mean square error (MSE) loss between the true density map and the predicted density map, z(i,j) represents the pixel value of the true density map, Represents the predicted density map pixel value, m and n represent the density map width and height.

[0054] On the other hand, the structural similarity index (SSIM) loss is used, which compares the differences between images in terms of brightness (l), contrast (c), and image structure (s):

[0055]

[0056] In the formula, represents the structural similarity index (SSIM) loss of the ground truth density map and the predicted density map, represents the brightness loss of the true density map and the predicted density map, represents the contrast loss between the true density map and the predicted density map, Represents the structural loss of the true value density map and the predicted density map; [] indicates that the three parts are a whole, which together represent the difference in the structural similarity index of the two density maps.

[0057] In the weakly supervised part, the counting loss l is used C Supervised training. Because the quantity is not a direct feature information in the density map, it is a form of weak supervision and plays an auxiliary role in training.

[0058]

[0059] In the formula, Represents the count loss of the true value density map and the predicted density map, represents the mean error loss, c(z) represents the number of targets in the true density map, Represents the number of targets for the prediction density map.

[0060] Therefore, the total loss function is the weighted sum of strong supervision and weak supervision:

[0061]

[0062] In the formula, represents the total loss function, and λ1, λ2, and λ3 are weights.

[0063] The weight λ1 of the mean square error loss during training is The weight λ2 of SSIM loss is 1, and the weight λ3 of counting loss is 0.01.

[0064] Step (7) locates the local maximum of the final predicted density map to obtain the target position coordinates;

[0065] After the joint training of strong and weak supervision loss in step (6), the network parameters of steps (2) to (4) are updated, and the input is a predicted density map that is closer to the true value of the annotation. Through the positioning method of step (5), all local maximum points in the predicted density map are obtained by using local maximum pooling, that is, the initial screening target points. Then, the screening threshold is designed according to the maximum pixel value of the whole image to remove the false positive targets that appear in the prediction and obtain the true target position coordinates.

[0066] Step (8) constructs a post-processing module, feeds the target positioning prediction results into the algorithm, obtains the density and distance distribution curves of the single atom, and marks the single atom target in the original image and saves it.

[0067] The post-processing module is responsible for processing the positioning results. After the recognition model reasoning is completed, the predicted coordinates are annotated to the original image, and the density and distance distribution curve of the single atom are calculated based on the carrier area and scale ratio using the distance matrix as the basis for characterization and analysis of the catalytic performance.

[0068] Step (9) Training method

[0069] The density estimation recognition model uses the Swin-Transformer network as the backbone network, which is pre-trained on the general dataset ImageNet 1k, and the network pre-training parameters are saved. Transfer learning is performed on the electron microscope images of catalytic materials. The image training size is 384x384, the training optimizer uses AdamW, and the initial learning rate is 3×10 -4 , Weight Decay is set to 0.05, and the Warmup method is used to adjust the learning rate according to the training rounds. The training time is 3000 rounds, the batch size is 16, and convergence is achieved after 2000 rounds. During the training process, the self-attention weights of the expanded pyramid pooling are optimized; at the same time, during the training process, a variety of online data enhancements are used on the training data, such as Crop, Rotate, and Flip operations, to increase sample diversity, improve model robustness and detection effect.

[0070] Step (10) Evaluation indicators

[0071] In this task, we should not only focus on the density estimation of metal single atoms, but also on the recognition accuracy of single atoms. Therefore, the evaluation index is set to F1-socre, which is the harmonic mean of precision and recall, and its calculation formula is:

[0072]

[0073] Among them, TP is the number of correctly detected positive samples, and FP is the number of falsely detected positive samples. Precision calculates the proportion of correct positive samples among all detected positive samples. FN is the number of missed targets in the image. In other words, Recall calculates the proportion of correctly detected targets among all true value targets.

[0074] The test results obtained in this embodiment are as follows Figure 5 As shown, two classic metal single-atom electron microscope images and their prediction results are shown. It can be seen that the model accurately identifies the position of the single atom, and the predicted density map and the true density map are highly consistent. The background noise of the electron microscope image can be effectively judged, avoiding the occurrence of false detection, showing strong robustness. At the same time, the densely distributed areas of single-atom particles can also be well separated and accurately located, and accurate counting can be achieved. The validation set was evaluated using evaluation indicators. The single-atom recognition model had an F1-score of 0.85, a MAE of 22, an accuracy of 94%, and an accurate recognition effect. These results not only demonstrate the theoretical effectiveness of the method of the present invention, but also prove its application potential in complex environments through practical application results.

Claims

1. A method for electron microscope image recognition of catalytic metal particles based on density estimation, characterized in that: The steps are as follows: Step (1) perform reverse distance transformation on the input original image using point annotation to obtain the true value of the density map; For the input electron microscope image of the catalytic material, the inverse distance transformation is used to generate the density map, the inverse of the distance from the pixel to the nearest adjacent target is used as the probability distribution of the target, and the true value density map is obtained using the true value of the coordinates; Step (2) inputs the electron microscope image of the catalytic material and sends it to the feature extraction network for feature extraction to generate feature maps with different sampling magnifications; the details are as follows: The electron microscope image of the catalytic material is sent to the feature extraction network for feature extraction to generate feature maps with different sampling magnifications. The feature extraction network uses the Swin-Transformer network structure as the backbone. After the electron microscope image of the catalytic material is input into the Swin-Transformer network, the preliminary feature extraction is first performed through the block division module to reduce the size of the feature map and expand the number of feature channels. Then, deep feature extraction is performed through layer-by-layer block merging and the Transformer module. Step (3) inputs the feature maps of different scales output by the feature extraction network into the dilated feature pyramid network to perform multi-level feature fusion; the details are as follows: First, the feature map pos n of the feature extraction network after the maximum downsampling is input into the dilated feature pyramid network, where n is related to the downsampling multiple of the selected feature extraction network; the feature map pos n passes through the dilated feature fusion module, and then is upsampled to expand its feature size to twice the original, and then the upsampled feature map and the feature map of the same size output by the feature extraction network are weighted summed, and the output feature map is pos(n-1), and the weight is optimized with the model training; repeat the above steps, the output feature map is continuously expanded feature fusion, upsampled, and weighted summed with the corresponding size output of the feature extraction network to fuse the target features at different sizes and improve the network information capture capability, and finally the feature map pos1 after the weighted summation of the last layer is output; The dilation feature fusion module is composed of two parallel branches, one is two stacked dilation convolutions with different dilation rates, and the other is a normal convolution. The features output by the two branches are spliced ​​on the channel and fused through convolution; Step (4) After layer-by-layer upsampling, the feature map finally output by the expanded feature pyramid network is sent to the self-attention expanded pyramid pooling module, and the predicted density map is obtained after upsampling; the details are as follows: The fused feature map pos1 is divided into three branches, which are parallelized through three dilated convolutions with different dilation coefficients and concatenated to obtain the feature map pos(n+1); then the feature map pos1 is introduced into the channel self-attention branch, and the channel weight is obtained through global average pooling, fully connected layer and sigmoid activation function, and the concatenated feature map pos(n+1) is weighted to obtain the feature map pos(n+2); the feature map pos1 is introduced again to directly add the feature map pos(n+2) to obtain the feature map pos(n+3), and the original input features are maintained; finally, the feature map pos(n+3) is passed through two consecutive upsampling layers to obtain the predicted density map pos(n+4) of the same size as the original image; Step (5) locating the target in the predicted density map by using a predicted density map target positioning algorithm; First, the feature map F is calculated using the maximum filter k，l The local maximum under the window size w, the local maximum map is recorded as L i，j ; Where (i, j) is the coordinate point in the feature map, and (k, l) is the coordinate of the feature point in the local window w; Then, the local maximum points are filtered using a preset threshold, where the threshold P is the maximum value of all local maximum values ​​multiplied by the scale factor α, that is, P=α×max(L i，j ) Finally, the local maximum point greater than the threshold is taken as the real predicted single atom target, and the coordinates (i, j) are saved; Step (6) uses the predicted density map and the true value density map to calculate the strong and weak losses, and the two are weighted to jointly supervise the training process of the density estimation recognition model network constructed by steps (2) to (4); For strong supervision loss, on the one hand, the mean square error (MSE) loss is used to calculate the square of the pixel difference between the predicted density map and the true value density map pixel by pixel, and then average it over the entire image; In the formula, represents the mean square error loss between the true density map and the predicted density map, z(i, j) represents the pixel value of the true density map, Represents the predicted density map pixel value, m and n represent the density map width and height; On the other hand, the structural similarity index SSIM loss is used to compare the differences between images in terms of brightness, contrast and image structure: In the formula, Represents the structural similarity index loss of the true value density map and the predicted density map, represents the brightness loss of the true density map and the predicted density map, represents the contrast loss between the true density map and the predicted density map, The structural loss representing the true value density map and the predicted density map; In the weak supervision part, the counting loss l is used C supervised training; In the formula, represents the count loss of the true value density map and the predicted density map, represents the mean error loss, c(z) represents the number of targets in the true density map, Represents the number of targets in the predicted density map; Therefore, the total loss function is the weighted sum of strong supervision and weak supervision: In the formula, represents the total loss function, λ1, λ2 and λ3 are weights; Step (7) locates the local maximum of the final predicted density map to obtain the target position coordinates; After the joint training of strong and weak supervision losses in step (6), the network parameters of steps (2) to (4) are updated, and a predicted density map that is closer to the true value of the annotation is output; through the positioning method of step (5), all local maximum points in the predicted density map are obtained by using local maximum pooling, that is, the initial screening target points; then, the screening threshold is designed according to the maximum pixel value of the whole image to remove the false positive targets that appear in the prediction and obtain the true target position coordinates; Step (8) constructing a post-processing module, sending the target position coordinates into the post-processing module, calculating the density and distance distribution curve of the single atom, and marking the single atom target in the original image and saving it; The post-processing module is responsible for processing the positioning results; marking the target position coordinates into the original image, and using the distance matrix to calculate the density and distance distribution curve of the single atom according to the carrier area and scale ratio, as the basis for the characterization and analysis of the catalytic performance.

Images