Atomic spatial distribution analysis method and device based on ADF-STEM graph

Abstract

The embodiment of the invention relates to an atomic spatial distribution analysis method and device based on an ADF-STEM graph. The method comprises the following steps: constructing a first GAN model, a second GAN model and a first prediction model; constructing a cross-system and cross-experiment style first data set; performing joint training on the two GAN models according to the first data set according to a training mechanism of a cyclic generative adversarial network; after the training is finished, constructing a second data set by using a first generator of the first GAN model to train a first prediction model; after training is finished, atomic-scale three-dimensional space distribution analysis is conducted on the sample ADF-STEM graph of any system through the first prediction model, and monatomic density analysis, dimer distribution analysis and trimer distribution analysis are conducted on the basis of an analyzed atom set. According to the method, atomic-scale three-dimensional space distribution analysis can be carried out, cross-system cross-experiment style analysis can be carried out, and the method can adapt to a high-throughput calculation scene.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method and apparatus for analyzing the spatial distribution of atoms based on ADF-STEM images. Background Technology

[0002] Single-atom catalysts (SACs), as cutting-edge materials in heterogeneous catalysis, have attracted widespread attention due to their maximized metal utilization, atomic-scale coordination control capabilities, and excellent performance in thermocatalytic and electrocatalytic reactions. Their atomic structure and spatial distribution determine reaction selectivity and stability; therefore, accurately identifying their true configuration on the support material is crucial for establishing the structure-performance correlation of catalysts.

[0003] Currently, annular dark-field scanning transmission electron microscopy (ADF-STEM) imaging is the mainstream method for characterizing SACs, achieving single-atom-level resolution due to its strong dependence on the signal intensity and atomic number of heavy elements. However, in practice, factors such as local thickness variations in the support, limited depth of focus, atomic deviation from the focal plane, and probe noise weaken the signals of some real atoms in the ADF-STEM image, causing these real atoms to be ignored during identification (such as non-focal layer atoms more than 6 nm off the focal plane), thus leading to statistical and configurational identification biases. In other words, improving the resolution accuracy of atomic spatial distribution in ADF-STEM images is a pressing technical problem in the current field of catalyst characterization.

[0004] Currently, the mainstream methods for solving the above-mentioned technical problems can be roughly divided into two categories: 1) Image classification model based on Convolutional Neural Network (CNN): This model identifies atomic bright spots in images by constructing a shallow CNN model and uses a sliding window strategy to perform pixel-by-pixel judgment. The drawbacks of this method are: it cannot provide three-dimensional spatial distribution, and its model generalization ability across different systems and experimental styles is very poor.

[0005] 2) A dual recognition method based on image fitting and spectroscopic verification: This method uses two-dimensional Gaussian fitting to extract atomic brightness features, and then combines this with electron energy loss spectroscopy for atomic-level three-dimensional spatial distribution analysis. The drawbacks of this method are: reliance on spectroscopic calculations, high experimental costs, and long computation cycles, making it unsuitable for high-throughput computing scenarios.

[0006] In summary, current conventional methods for improving the analytical accuracy of ADF-STEM images either fail to provide a three-dimensional spatial distribution and cannot maintain analytical accuracy across different systems and experimental styles; or they increase processing costs and lose processing efficiency while maintaining high analytical accuracy through a large amount of basic computation, making them completely unsuitable for high-throughput computing scenarios. Summary of the Invention

[0007] The purpose of this invention is to address the shortcomings of existing technologies by providing a method, apparatus, electronic device, and computer-readable storage medium for analyzing atomic spatial distribution based on ADF-STEM images. This invention constructs two GAN models (a first GAN model and a second GAN model) and one prediction model (a first prediction model). The first GAN model performs experimental style transfer on the image style of the ADF-STEM image (referred to as the simulated image) generated by a simulation tool; the experimental style refers to the image style of the ADF-STEM image (referred to as the experimental image) obtained in a real experimental scenario. The second GAN model performs simulated style transfer on the image style of the experimental image; the simulated style refers to the image style of the ADF-STEM image generated by the simulation tool. The first prediction model performs atomic-level depth prediction based on the ADF-STEM image of the experimental style, that is, predicts the atomic depth semantic map of the experimental image. The system constructs an experimental graph sample set (first sample set) by collecting experimental graph samples from multiple systems (the range of material systems specified in the first material system set), and a simulation graph sample set (second sample set) by generating simulation graphs from multiple systems. An unsupervised dataset (first dataset) is then constructed by pairwise combining the experimental graphs and simulation graphs from the first and second sample sets. Two GAN models are jointly trained using an unsupervised game-theoretic training method within the Cycle Generative Adversarial Networks (CycleGAN) framework. After joint training, a cross-system, cross-experimental style labeled dataset (second dataset) is constructed based on the first generator of the first GAN model and the second sample set. The first prediction model is then trained using a supervised training method on the second dataset. After the prediction model training is complete, the first prediction model is used to perform three-dimensional atomic space distribution analysis on the ADF-STEM graphs of any material system sample satisfying the first material system set. After obtaining the atomic spatial distribution analysis results (atomic set), this invention further analyzes the single-atom density, dimer distribution, and trimer distribution of the current material sample based on these results. On the one hand, this invention can perform atomic-level three-dimensional spatial distribution analysis through a predictive model; on the other hand, by constructing and training models across systems and experimental styles, it can maintain analytical accuracy and improve model generalization ability in cross-system and cross-experimental style scenarios; furthermore, it eliminates the need for extensive and complex basic calculations, thus shortening analysis time, improving processing efficiency, and adapting to high-throughput computing scenarios.

[0008] To achieve the above objectives, a first aspect of the present invention provides a method for analyzing the spatial distribution of atoms based on ADF-STEM images, the method comprising: A first GAN model, a second GAN model, and a first prediction model are constructed. The first GAN model is used to perform experimental style transfer on the image style of the ADF-STEM image. The second GAN model is used to perform simulated style transfer on the image style of the ADF-STEM image. The first prediction model is used to perform atomic-level depth prediction based on the experimental style of the ADF-STEM image. Based on a preset first material system set, real samples of multi-system ADF-STEM images are collected to generate a corresponding first sample set; and based on the first material system set, multi-system ADF-STEM images are simulated to generate a corresponding second sample set; and a first dataset is constructed based on the first and second sample sets. The training mechanism of recurrent generative adversarial networks is used to jointly train the first and second GAN models based on the first dataset; after the joint training is completed, the second dataset is constructed based on the first generator of the first GAN model and the second sample set. The first prediction model is trained based on the second dataset; After the prediction model training is completed, the first sample information input by the user is received; the first sample information corresponds to a material sample, and the material system of the current material sample satisfies the first material system set; the first sample information consists of the sample ADF-STEM image and the pixel distance scale △p and depth interlayer spacing △d of the current ADF-STEM image; Based on the first prediction model and the first sample information, the atomic spatial distribution of the current material sample is analyzed to obtain the corresponding atom set; and based on the atom set, the single atom density, dimer distribution and trimer distribution of the current material sample are analyzed to obtain the corresponding first density, dimer set and trimer set; A spatial distribution analysis report of atoms, consisting of the set of atoms, the first density, the set of dimers, and the set of trimers, is fed back to the current user.

[0009] Preferably, the first GAN model is formed by connecting the first generator and the first discriminator; The model input of the first GAN model is the first simulation graph M. s1 The model output is the first probability p1; The first simulation diagram M s1 ADF-STEM image; The first probability p1 is the first experimental plot M generated by the first generator. e1 The probability of truth or falsehood; the first probability p1 takes a value between 0 and 1, the larger the value, the higher the probability of truth or falsehood in the first experimental diagram M. e1 The higher the probability of the true image and the smaller the value, the better the first experimental image M. e1 The higher the probability of a fake image; The first generator is used to process the first simulation graph M s1 The image style was subjected to experimental style transfer processing to obtain the corresponding first experimental image M. e1 ; in, , G S-E () represents the model inference expression of the first generator, and θ1 represents the model parameters of the first generator; The first discriminator is used to analyze the first experimental diagram M. e1 The probability of a true or false image is used to determine the corresponding first probability p1; in, , D E () represents the model inference expression of the first discriminator, and θ2 represents the model parameters of the first discriminator; The second GAN model is composed of a second generator and a second discriminator connected together; The input to the second GAN model is the second experimental graph M. e2 The model output is the second probability p2; Second experimental diagram M e2 ADF-STEM image; The second probability p2 is the second simulation graph M generated by the second generator. s2 The true or false probability; the second probability p2 takes a value between 0 and 1, the larger the value, the higher the probability of the second simulation graph M. s2 The higher the probability of the true image and the smaller the value, the better the second simulated image M. s2 The higher the probability of a fake image; The second generator is used for the second experimental graph M e2 The image style is subjected to simulated style transfer processing to obtain the corresponding second simulated image M. s2 ; in, , G E-S () represents the model inference expression of the second generator, and θ3 represents the model parameters of the second generator; The second discriminator is used to analyze the second simulation image M. s2 The probability of a true or false image is used to determine the corresponding second probability p2; in, , D S () represents the model inference expression of the second discriminator, and θ4 represents the model parameters of the second discriminator; The first prediction model consists of a first feature extraction network and a depth prediction head; the input of the first feature extraction network is connected to the model input of the first prediction model, and the output is connected to the input of the depth prediction head; the output of the depth prediction head is connected to the model output of the first prediction model. The model input of the first prediction model is the first experimental graph M. e1 The model output is a deep semantic graph M. d ; First experimental diagram M e1 The shape is denoted as H0×W0×C0, where H0, W0, and C0 are respectively the first experimental diagram M. e1 Image height, image width, and pixel feature dimensions; The deep semantic graph M d The shape is H0×W0×Z, and the preset total number of depth levels Z is a positive integer greater than 1; the depth semantic graph M d It consists of H0×W0 first pixels, and the pixel semantic feature of each first pixel consists of Z depth level probabilities; each depth level probability corresponds to a depth level i, 1≤i≤Z; the value of each depth level probability is between 0 and 1; the larger the depth level probability, the greater the probability that there is an atom at the three-dimensional spatial position corresponding to the current depth level in the depth direction of the current pixel; the smaller the depth level probability, the smaller the probability that there is an atom at the three-dimensional spatial position corresponding to the current depth level in the depth direction of the current pixel. The first feature extraction network is implemented based on the U-Net model; the first feature extraction network is used to extract features from the first experimental image M. e1 Feature extraction is performed to obtain the corresponding first feature map H; the shape of the first feature map H is H0×W0×C. H C H Let C be the pixel feature dimension of the first feature map H. H >Z; The depth prediction head is implemented based on a convolutional network and a sigmoid function. The depth prediction head uses a built-in convolutional network to perform feature mapping on the first feature map H to obtain a depth feature map of shape H0×W0×Z. The depth feature map consists of H0×W0 first pixel features, each of which consists of Z depth feature values. The sigmoid function is used to calculate the depth level probability corresponding to each depth feature value of each first pixel feature. The Z depth level probabilities corresponding to each first pixel feature are used to form a pixel semantic feature for the corresponding first pixel. The resulting H0×W0 pixel semantic features of the first pixels form the corresponding depth semantic map M.d ; The first set of material systems includes multiple first material systems; all of the first material systems can be used as single-atom catalyst supports; The first sample set includes multiple first sample images; all first sample images are real ADF-STEM images collected from the experimental environment; each first sample image corresponds to a class of the first material system; the set of material systems in the first sample set is consistent with the set of first material systems. The second sample set includes multiple second sample plots; all second sample plots are simulated ADF-STEM plots generated by simulation tools; each second sample plot corresponds to a class of the first material system; the set of material systems corresponding to the second sample set is consistent with the set of the first material systems; the simulation tools include abTEM, xHREM, and Multem tools; The first dataset includes multiple first data records; the first data records include a first training simulation graph and a first training experiment graph; The second dataset includes multiple second data records; the second data records include a second training experiment graph and a first label semantic graph; the graph data structure of the first label semantic graph is the same as that of the deep semantic graph M. d Maintain consistency; The set of atoms includes multiple first atoms; the atomic attributes of the first atoms include atomic identifiers and three-dimensional atomic coordinates; The dimer set includes multiple first dimers; the dimer attribute of the first dimer consists of a dimer identifier and two atomic identifiers; The set of trimers includes multiple first trimers; the trimer attribute of the first trimer consists of a trimer identifier and three atom identifiers.

[0010] Preferably, constructing the first dataset based on the first and second sample sets specifically includes: Each of the first sample images in the first sample set is taken as a corresponding first training experiment image, and each of the second sample images in the second sample set is taken as a corresponding first training simulation image; each of the first training simulation images is taken as the current simulation image; and the current simulation image and each of the first training experiment images are combined to form a corresponding first data record; and all the obtained first data records are combined to form a corresponding first dataset.

[0011] Preferably, the training mechanism of the cyclic generative adversarial network jointly trains the first and second GAN models based on the first dataset, specifically including: Step 41: Calculate the total number of records in the first dataset to obtain the corresponding total number N. tr1 And the first training simulation graph and the first training experiment graph of each of the first data records are recorded as the corresponding simulation graph M. S(j) Experimental diagram M E(j) ; Where 1 ≤ index j ≤ N tr1 ; Step 42, N tr1 The simulation diagram M described in the group S(j) and the experimental diagram M E(j) Substitute the preset first model loss function L M1 The corresponding first loss value is obtained through calculation; Wherein, the first model loss function L M1 The joint loss implementation based on adversarial loss and cycle consistency loss is as follows: , , , ; L adv1 Let L be the adversarial loss function of the first GAN model. adv2 Let L be the adversarial loss function of the second GAN model. cyc Let λ be the cycle consistency loss function; cyc The preset weighting coefficients for cycle consistency loss; Step 43: Identify whether the first loss value meets the preset first loss value range; if the first loss value does not meet the first loss value range, then optimize the first model loss function L according to the preset objective function O1 and the preset first model optimizer. M1 The model parameters θ1 and θ3 that achieve the minimum value, and the loss function L of the first model that makes the first model reach its minimum value. M1 The model parameters θ2 and θ4 that have reached their maximum values ​​are subjected to one round of parameter modulation, and the process returns to step 42 when the parameter modulation ends; if the first loss value meets the first loss value range, then training is stopped and the joint training is confirmed to be over. Wherein, the preset objective function O1 is: ; The first model optimizer includes the Adam optimizer.

[0012] Preferably, the construction of the second dataset based on the first generator of the first GAN model and the second sample set specifically includes: Use each of the second sample graphs in the second sample set as the current simulation graph; And the current simulation graph is used as the current first simulation graph M. s1 The first generator is input for processing, and the first experimental diagram M obtained from this processing is... e1 As the corresponding second training experiment diagram; The system identifies the atomic center points of all atoms in the current simulation graph and the depth level of each atomic center point; and based on the depth semantic graph M... d The image data format is the first label semantic map corresponding to the initialization of the current simulation image; and according to the atomic center point identification result, the Z depth level probabilities corresponding to each first pixel are labeled on the first label semantic map; on the first label semantic map, among the Z depth level probabilities of the first pixel that matches each atomic center point, there is only one probability of 1 and the remaining Z-1 probabilities are all 0, and the depth level i corresponding to the depth level probability of the depth level that is 1 matches the center point depth level identification result of the current atomic center point; on the first label semantic map, the Z depth level probabilities of each first pixel that does not match any atomic center point are all 0. The second data record is composed of the second training experiment graph corresponding to the current simulation graph and the first label semantic graph; and the second dataset is composed of all the obtained second data records.

[0013] Preferably, training the first prediction model based on the second dataset specifically includes: Step 61: Calculate the total number of records in the second dataset to obtain the corresponding total number N. tr2 And the second training experimental graph and the first label semantic graph of each second data record in the second dataset are denoted as the corresponding experimental graph M. E(k) Tag semantic graph M tag(k) ; Among them, the experimental diagram M E(k) The image height and image width are denoted as H. k W k The tag semantic graph M tag(k) By H k ×W k label vector mt k,r Composition; the label vector mt k,r Given the probability mt of Z labels k,r,q Composition; 1 ≤ index k ≤ N tr2 1 ≤ index r ≤ H k ×W k, 1 ≤ index q ≤ Z; Step 62, transfer each of the experimental diagrams M E(k) As the corresponding first experimental diagram M e1 The first prediction model is input for processing, and the deep semantic map M obtained from this processing is... d As the corresponding predictive semantic graph M pre(k) ; Wherein, the predicted semantic graph M pre(k) By H k ×W k The prediction vector mk,r is composed of Z prediction probabilities mk,r,q. Step 63, obtain N tr2 The group's predicted semantic graph M pre(k) and the tag semantic graph M tag(k) Substitute the preset second model loss function L M2 The corresponding second loss value is obtained through calculation; Wherein, the second model loss function L M2 The implementation is based on the joint loss of weighted cross-entropy loss and Dice loss, specifically as follows: , , ; L wce For the preset weighted cross-entropy loss function, L dice α is the preset Dice loss function, and α is the preset balancing parameter; w q The weights are for the q-th class. Step 64: Identify whether the second loss value meets the preset second loss value range; if the second loss value does not meet the second loss value range, then based on the preset second model optimizer, move towards making the second model loss function L... M2 The model parameters of the first prediction model are modulated once in the direction that reaches the minimum value, and the process returns to step 62 when the parameter modulation ends; if the second loss value meets the range of the second loss value, then training is stopped and the prediction model training is confirmed to be completed. The second model optimizer includes the Adam optimizer and the SGD optimizer.

[0014] Preferably, the step of analyzing the atomic spatial distribution of the current material sample based on the first prediction model and the first sample information to obtain the corresponding atom set specifically includes: The sample ADF-STEM image of the first sample information is used as the current first experimental image M. e1The first prediction model is input for processing; and the deep semantic map M obtained from this processing is... d As the corresponding current depth semantic graph; Based on the image height H0, image width W0, total number of depth levels Z, pixel distance scale △p, and depth layer spacing △d of the first sample information, a Cartesian three-dimensional space is constructed to obtain the corresponding first three-dimensional space; the XYZ axes of the first three-dimensional space only have positive axes, where the maximum value of the width X axis is W0×△p, the maximum value of the height Y axis is H0×△p, and the maximum value of the depth Z axis is Z×△d; the XY plane of the first three-dimensional space is parallel to the imaging plane of the sample ADF-STEM image; Among the H0×W0×Z depth level probabilities of the current depth semantic graph, the depth level probabilities whose probability values ​​exceed a preset first probability threshold are all recorded as the corresponding third probabilities. Each of the third probabilities is taken as the corresponding current probability; and based on the width pixel coordinate u, height pixel coordinate v, and depth level i of the first pixel corresponding to the current probability, a set of corresponding width coordinates x, height coordinates y, and depth coordinates z in the first three-dimensional space are calculated, x=u×△p, y=v×△p, z=i×△d; and the width coordinate x, height coordinate y, and depth coordinate z obtained this time form a corresponding three-dimensional atom coordinate (x,y,z); and a unique identifier is assigned to the current probability as the corresponding atom identifier; and the atom identifier corresponding to the current probability and the three-dimensional atom coordinate (x,y,z) form a corresponding first atom; The corresponding set of atoms is composed of all the first atoms obtained.

[0015] Preferably, the step of analyzing the single-atom density, dimer distribution, and trimer distribution of the current material sample based on the atomic set to obtain the corresponding first density, dimer set, and trimer set specifically includes: Based on the image height H0, image width W0, total number of depth levels Z, pixel distance scale Δp, and depth interlayer spacing Δd corresponding to the sample ADF-STEM image, the corresponding spatial volume V = (W0 × Δp) × (H0 × Δp) × (Z × Δd) is calculated; and the total number of atoms N of the atom set is calculated. atom Perform statistics; and based on the total number of atoms N atom The first density is obtained by calculating the single-atom density of the spatial volume V, where the first density = N. atom / V; The coordinate spacing between any two first atoms in the atom set is calculated to obtain the corresponding first spacing; and based on the first spacing and a preset first atomic spacing threshold, all first atoms are clustered to obtain multiple corresponding first atom clusters; and the first atom cluster containing only one first atom is recorded as a single atom cluster, and the first atom cluster containing two or more first atoms is recorded as a multi-atom cluster; and the multi-atom cluster with a total number of atoms of 2 or 3 is recorded as the corresponding dimer atom cluster or trimer atom cluster; the first atom cluster is composed of one or more first atoms; the first spacing between any two first atoms in the multi-atom cluster is less than the first atomic spacing threshold. The system identifies whether the total number of the dimer atom clusters is zero; if so, the corresponding dimer set is set to empty; if not, each dimer atom cluster is taken as the corresponding current atom cluster, and a unique identifier is assigned to the current atom cluster as the corresponding dimer identifier. The dimer identifier and the atom identifiers of the two first atoms in the current atom cluster are combined to form a corresponding first dimer, and all the obtained first dimers are combined to form the corresponding dimer set. The system identifies whether the total number of the trimer atom clusters is zero; if so, the corresponding trimer set is set to empty; if not, each trimer atom cluster is taken as the corresponding current atom cluster, and a unique identifier is assigned to the current atom cluster as the corresponding trimer identifier. The trimer identifier and the atom identifiers of the three first atoms in the current atom cluster are combined to form a corresponding first trimer, and all the obtained first trimers are combined to form the corresponding trimer set. The first density, the dimer set, and the trimer set obtained in this analysis will be output as the results of this analysis.

[0016] A second aspect of the present invention provides an apparatus for implementing the atomic spatial distribution analysis method based on ADF-STEM graphs as described in the first aspect above. The apparatus includes: a model building module, a dataset preparation module, a cyclic generation adversarial training module, a prediction model training module, a sample receiving module, a sample parsing module, and a parsing feedback module. The model building module is used to build a first GAN model, a second GAN model, and a first prediction model; the first GAN model is used to perform experimental style transfer on the image style of the ADF-STEM image; the second GAN model is used to perform simulated style transfer on the image style of the ADF-STEM image; and the first prediction model is used to perform atomic-level depth prediction based on the experimental style of the ADF-STEM image. The dataset preparation module generates a corresponding first sample set by collecting real samples of multi-system ADF-STEM images based on a preset first material system set; and generates a corresponding second sample set by simulating multi-system ADF-STEM images based on the first material system set; and constructs a first dataset based on the first and second sample sets. The recurrent generative adversarial training module is used to jointly train the first and second GAN models according to the first dataset based on the training mechanism of the recurrent generative adversarial network; after the joint training is completed, a second dataset is constructed based on the first generator of the first GAN model and the second sample set. The prediction model training module trains the first prediction model based on the second dataset; The sample receiving module is used to receive the first sample information input by the user after the prediction model training is completed; the first sample information corresponds to a material sample, and the material system of the current material sample satisfies the first material system set; the first sample information consists of the sample ADF-STEM image and the pixel distance scale △p and depth interlayer spacing △d of the current ADF-STEM image; The sample analysis module analyzes the atomic spatial distribution of the current material sample based on the first prediction model and the first sample information to obtain the corresponding atom set; and analyzes the single atom density, dimer distribution and trimer distribution of the current material sample based on the atom set to obtain the corresponding first density, dimer set and trimer set; The analysis feedback module provides the current user with an atomic spatial distribution analysis report composed of the atom set, the first density, the dimer set, and the trimer set.

[0017] A third aspect of the present invention provides an electronic device, including: a memory, a processor, and a transceiver; The processor is used to couple with the memory, read and execute instructions in the memory to implement the steps of the method described in the first aspect above; The transceiver is coupled to the processor, and the processor controls the transceiver to send and receive messages.

[0018] A fourth aspect of the present invention provides a computer-readable storage medium storing computer instructions that, when executed by a computer, cause the computer to perform the instructions described in the first aspect.

[0019] This invention provides a method, apparatus, electronic device, and computer-readable storage medium for analyzing atomic spatial distribution based on ADF-STEM images. As described above, this invention constructs two GAN models (a first GAN model and a second GAN model) and one prediction model (a first prediction model). The first GAN model performs experimental style transfer on the image style of the ADF-STEM image (referred to as the simulated image) generated by a simulation tool. The experimental style refers to the image style of the ADF-STEM image (referred to as the experimental image) obtained in a real experimental scenario. The second GAN model performs simulated style transfer on the image style of the experimental image. The simulated style refers to the image style of the ADF-STEM image generated by the simulation tool. The first prediction model performs atomic-level depth prediction based on the ADF-STEM image with the experimental style, that is, predicts the atomic depth semantic map of the experimental image. The invention constructs an experimental graph sample set (first sample set) by collecting experimental graph samples from multiple systems (the range of material systems specified in the first material system set), and a simulation graph sample set (second sample set) by generating simulation graphs from multiple systems. An unsupervised dataset (first dataset) is then constructed by pairwise combining the experimental graphs and simulation graphs from the first and second sample sets. Following the unsupervised game-theoretic training method of the CycleGAN framework, the two GAN models are jointly trained using the unsupervised dataset. After joint training, a cross-system, cross-experimental style labeled dataset (second dataset) is constructed based on the first generator of the first GAN model and the second sample set. The first prediction model is then trained using a supervised training method based on the second dataset. After the prediction model training is complete, the first prediction model is used to perform three-dimensional atomic spatial distribution analysis on the ADF-STEM graphs of any material system sample satisfying the first material system set. In this embodiment of the invention, after obtaining the atomic spatial distribution analysis result (atomic set), further analysis is performed on the single-atom density, dimer distribution, and trimer distribution of the current material sample based on this analysis result. The embodiments of the present invention achieve atomic-level three-dimensional spatial distribution analysis through a prediction model; on the other hand, by constructing and training datasets across systems and experimental styles, the analytical accuracy can be maintained and the model generalization ability can be improved in cross-system and cross-experimental styles; furthermore, without the need for a large number of complex basic calculations, the analysis time is shortened, the processing efficiency is improved, and it is adaptable to high-throughput computing scenarios. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of an atomic spatial distribution analysis method based on ADF-STEM images provided in Embodiment 1 of the present invention; Figure 2 A module structure diagram of the first GAN model and the second GAN model provided in Embodiment 1 of the present invention; Figure 3 This is a block diagram of the first prediction model provided in Embodiment 1 of the present invention; Figure 4 This is a module structure diagram of an atomic spatial distribution analysis device based on an ADF-STEM image provided in Embodiment 2 of the present invention; Figure 5 This is a schematic diagram of the structure of an electronic device provided in Embodiment 3 of the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0022] Embodiment 1 of the present invention provides a method for analyzing the spatial distribution of atoms based on ADF-STEM images, such as... Figure 1 The schematic diagram shows a method for analyzing the spatial distribution of atoms based on ADF-STEM images provided in Embodiment 1 of the present invention. The method mainly includes the following steps: Step 1: Construct the first GAN model, the second GAN model, and the first prediction model.

[0023] Here, the first GAN model in this embodiment of the invention is used to perform experimental style transfer on the image style of the ADF-STEM image; the second GAN model is used to perform simulated style transfer on the image style of the ADF-STEM image; and the first prediction model is used to perform atomic-level depth prediction based on the experimental style of the ADF-STEM image.

[0024] (1) First GAN model: The first GAN model in this embodiment of the invention is composed of a first generator and a first discriminator connected together, such as... Figure 2 The module structure diagram of the first GAN model and the second GAN model provided in Embodiment 1 of the present invention is shown.

[0025] The input to the first GAN model is the first simulation graph M. s1 The model output is the first probability p1. The first simulation graph M... s1 This is an ADF-STEM plot. The first probability p1 is the first experimental plot M generated by the first generator. e1 The true or false probability; the first probability p1 takes a value between 0 and 1, the larger the value, the higher the probability of the first experimental plot M. e1 The higher the probability of the true graph and the smaller the value, the better the first experimental graph M. e1 The higher the probability of a fake image.

[0026] The first generator is used to process the first simulation graph M. s1 The image style was subjected to experimental style transfer processing to obtain the corresponding first experimental image M. e1 The reasoning process is as follows: ; Among them, G S-E () represents the model inference expression of the first generator, and θ1 represents the model parameters of the first generator.

[0027] The first discriminator is used to analyze the first experimental diagram M. e1 The probability of a true or false image is used to determine the corresponding first probability p1; the reasoning process is as follows: ; Among them, D E () represents the model inference expression of the first discriminator, and θ2 represents the model parameters of the first discriminator.

[0028] (2) Second GAN model: The second GAN model in this embodiment of the invention is composed of a second generator and a second discriminator connected together, such as... Figure 2 As shown.

[0029] The input to the second GAN model is the second experimental graph M. e2 The model output is the second probability p2. The second experimental diagram M... e2 This is the ADF-STEM plot. The second probability p2 is the second simulation plot M generated by the second generator. s2 The true or false probability of the second simulation plot M; the second probability p2 takes a value between 0 and 1, the larger the value, the higher the probability of the second simulation plot M. s2 The higher the probability of the true graph and the smaller the value, the better the second simulation graph M. s2 The higher the probability of a fake image.

[0030] The second generator is used to process the second experimental graph M. e2 The image style is subjected to simulated style transfer processing to obtain the corresponding second simulated image M. s2 The reasoning process is as follows: ; Among them, G E-S () represents the model inference expression of the second generator, and θ3 represents the model parameters of the second generator.

[0031] The second discriminator is used to analyze the second simulation image M. s2 The probability of a true or false image is used to determine the corresponding second probability p2; the reasoning process is as follows: ; Among them, D S() represents the model inference expression of the second discriminator, and θ4 represents the model parameters of the second discriminator.

[0032] (3) First prediction model: The first prediction model in this embodiment of the invention consists of a first feature extraction network and a deep prediction head, such as... Figure 3 The diagram shows the module structure of the first prediction model provided in Embodiment 1 of the present invention. The input of the first feature extraction network is connected to the model input of the first prediction model, and its output is connected to the input of the depth prediction head; the output of the depth prediction head is connected to the model output of the first prediction model.

[0033] The model input for the first prediction model is the first experimental graph M. e1 The model output is a deep semantic graph M. d .

[0034] First Experimental Diagram M e1 The shape is denoted as H0×W0×C0, where H0, W0, and C0 are the shapes of the first experimental figure M. e1 Image height, image width, and pixel feature dimensions.

[0035] Deep semantic graph M d The shape is H0×W0×Z, and the preset total number of depth levels Z is a positive integer greater than 1; the depth semantic graph M d It consists of H0×W0 first pixels, and the pixel semantic feature of each first pixel consists of Z depth level probabilities; each depth level probability corresponds to a depth level i, 1≤i≤Z; the value of each depth level probability is between 0 and 1; the larger the depth level probability, the greater the probability that there is an atom at the three-dimensional spatial position corresponding to the current depth level in the depth direction of the current pixel; the smaller the depth level probability, the smaller the probability that there is an atom at the three-dimensional spatial position corresponding to the current depth level in the depth direction of the current pixel.

[0036] The first feature extraction network is implemented based on the U-Net model; the first feature extraction network is used to extract features from the first experimental image M. e1 Feature extraction is performed to obtain the corresponding first feature map H; the shape of the first feature map H is H0×W0×C. H C H Let C be the pixel feature dimension of the first feature map H. H >Z.

[0037] The depth prediction head is implemented based on convolutional networks and the sigmoid function.

[0038] The depth prediction head uses a built-in convolutional network to perform feature mapping on the first feature map H to obtain a depth feature map of shape H0×W0×Z. This depth feature map consists of H0×W0 first pixel features, and each first pixel feature consists of Z depth feature values. The sigmoid function is used to calculate the depth level probability corresponding to each depth feature value of each first pixel feature on the depth feature map H0. The Z depth level probabilities corresponding to each first pixel feature are then used to form a pixel semantic feature for the corresponding first pixel. Finally, the resulting H0×W0 pixel semantic features of the first pixels form the corresponding depth semantic map M. d .

[0039] Step 2: Based on the preset first material system set, collect real samples of multi-system ADF-STEM images to generate the corresponding first sample set; and based on the first material system set, simulate multi-system ADF-STEM images to generate the corresponding second sample set; and construct the first dataset based on the first and second sample sets. Specifically, it includes: Step 21, collecting real samples of multi-system ADF-STEM images based on a preset first material system set to generate the corresponding first sample set; Here, the first material system set in the embodiments of the present invention includes multiple first material systems; all first material systems can be used as single-atom catalyst supports, such as graphene, tungsten disulfide WS2, tungsten diselenide WSe2, etc. The first sample set of this invention is a cross-system, cross-experimental style experimental image sample set; the first sample set includes multiple first sample images; all first sample images are real ADF-STEM images collected from the experimental environment; each first sample image corresponds to a class of first material systems; the material system set of the first sample set is consistent with the first material system set; Step 22, and generate the corresponding second sample set by performing multi-system ADF-STEM plot simulation based on the first material system set; Here, the second sample set in this embodiment of the invention is a simulation plot sample set that spans systems and experimental styles; the second sample set includes multiple second sample plots; all second sample plots are simulated ADF-STEM plots generated by simulation tools; each second sample plot corresponds to a class of first material systems; the set of material systems corresponding to the second sample set is consistent with the set of first material systems; the simulation tools in this embodiment of the invention include at least abTEM, xHREM, and Multem tools; Step 23, and construct the first dataset based on the first and second sample sets; In this embodiment of the invention, the first dataset is an unsupervised dataset, constructed by pairwise combining experimental graphs and simulation graphs from the first and second sample sets; the first dataset includes multiple first data records; each first data record includes a first training simulation graph and a first training experimental graph. Specifically, this includes: taking each first sample image of the first sample set as a corresponding first training experiment image, and taking each second sample image of the second sample set as a corresponding first training simulation image; taking each first training simulation image as the current simulation image; and forming a corresponding first data record by combining the current simulation image with each first training experiment image; and forming a corresponding first dataset by combining all the obtained first data records.

[0040] Step 3: According to the training mechanism of the recurrent generative adversarial network, the first and second GAN models are jointly trained based on the first dataset; after the joint training is completed, the second dataset is constructed based on the first generator of the first GAN model and the second sample set. Specifically, this includes: Step 31, training the first and second GAN models jointly based on the first dataset according to the training mechanism of the recurrent generative adversarial network; Specifically, this includes: Step 311, calculating the total number of records in the first dataset to obtain the corresponding total number N. tr1 And record the first training simulation graph and the first training experiment graph of each first data record as the corresponding simulation graph M. S(j) Experimental diagram M E(j) ; Where 1 ≤ index j ≤ N tr1 ; Step 312, N tr1 Group simulation diagram M S(j) And experimental diagram M E(j) Substitute the preset first model loss function L M1 The corresponding first loss value is obtained through calculation; Here, the first model loss function L in this embodiment of the invention M1 The joint loss implementation based on adversarial loss and cycle consistency loss is as follows: , , , ; Among them, L adv1 L is the adversarial loss function of the first GAN model. adv2 L is the adversarial loss function of the second GAN model. cyc Let λ be the cycle consistency loss function; cycThe preset weighting coefficients for cycle consistency loss; Step 313: Identify whether the first loss value meets the preset first loss value range; if the first loss value does not meet the first loss value range, then optimize the first model loss function L according to the preset objective function O1 and the preset first model optimizer. M1 The model parameters θ1, θ3 that achieve the minimum value and the loss function L of the first model that achieves the minimum value M1 The model parameters θ2 and θ4 that have reached their maximum values ​​are subjected to one round of parameter modulation, and the process returns to step 312 when the parameter modulation ends; if the first loss value meets the first loss value range, then training is stopped and the joint training is confirmed to be over. The first loss value range is a pre-set numerical range; The preset objective function O1 is: ; The first model optimizer includes the Adam optimizer; Step 32: After joint training, construct the second dataset based on the first generator of the first GAN model and the second sample set; The second dataset includes multiple second data records; each second data record includes a second training experiment graph and a first label semantic graph; the graph data structure of the first label semantic graph is related to the deep semantic graph M. d Maintain consistency; Specifically, this includes: step 321, using each second sample graph of the second sample set as the current simulation graph; Step 322, and use the current simulation graph as the current first simulation graph M. s1 The first generator is input for processing, and the first experimental image M obtained from this processing is generated. e1 As the corresponding second training experiment diagram; Step 323 involves identifying the atomic center points of all atoms in the current simulation graph and determining the depth level of each atomic center point; and then basing this on the deep semantic graph M. d The image data format is the first label semantic map corresponding to the initialization of the current simulation image; and based on the atomic center point identification results, the Z depth level probabilities corresponding to each first pixel point are labeled on the first label semantic map; Here, in the first label semantic map of this embodiment, among the Z depth level probabilities of the first pixel that matches each atomic center point, only one probability is 1, and the remaining Z-1 probabilities are all 0. The depth level corresponding to the depth level probability of 1 matches the center point depth level recognition result of the current atomic center point. In the first label semantic map, all Z depth level probabilities of the first pixel that does not match any atomic center points are 0. Step 324, and a corresponding second data record is formed by the second training experimental graph corresponding to the current simulation graph and the first label semantic graph; and the corresponding second dataset is formed by all the obtained second data records.

[0041] Step 4: Train the first prediction model based on the second dataset; Specifically, this includes: Step 41, calculating the total number of records in the second dataset to obtain the corresponding total number N. tr2 And denote the second training experimental graph and the first label semantic graph of each second data record in the second dataset as the corresponding experimental graph M. E(k) Tag semantic graph M tag(k) ; Here, experimental diagram M of the embodiment of the present invention. E(k) The image height and image width are denoted as H. k W k Tag semantic graph M tag(k) By H k ×W k label vector mt k,r Composition; label vector mt k,r Given the probability mt of Z labels k,r,q Composition; 1 ≤ index k ≤ N tr2 1 ≤ index r ≤ H k ×W k , 1 ≤ index q ≤ Z; Step 42, transfer each experimental diagram M E(k) As the corresponding first experimental diagram M e1 The input is processed by the first prediction model, and the resulting deep semantic map M is output. d As the corresponding predictive semantic graph M pre(k) ; Here, the predicted semantic graph M in this embodiment of the invention pre(k) By H k ×W k The prediction vector mk,r is composed of Z prediction probabilities mk,r,q. Step 43, obtain N tr2 Group Predictive Semantic Graph M pre(k) and label semantic graph M tag(k) Substitute the preset second model loss function L M2 The corresponding second loss value is obtained through calculation; Here, the second model loss function L in this embodiment of the invention M2 The implementation is based on the joint loss of weighted cross-entropy loss and Dice loss, specifically as follows: , , ; Among them, L wce For the preset weighted cross-entropy loss function, L dice α is the preset Dice loss function, and α is the preset balancing parameter; w q The weights are for the q-th class. Step 44: Identify whether the second loss value meets the preset range of the second loss value; if the second loss value does not meet the range of the second loss value, then based on the preset second model optimizer, move towards making the second model loss function L... M2 The direction that reaches the minimum value modulates the model parameters of the first prediction model in one round, and returns to step 42 when the parameter modulation ends; if the second loss value meets the range of the second loss value, then stop training and confirm that the prediction model training is over. The second loss value range is a pre-set numerical range; the second model optimizer includes the Adam optimizer and the SGD optimizer.

[0042] Step 5: After the prediction model training is completed, receive the first sample information input by the user.

[0043] Here, the first sample information of this embodiment corresponds to a material sample, and the material system of the current material sample satisfies the first material system set; the first sample information consists of the sample ADF-STEM image and the pixel distance scale △p and depth layer spacing △d of the current ADF-STEM image; the pixel distance scale △p is used to set the corresponding real spatial distance between every two horizontal or vertical adjacent pixels, and the depth layer spacing △d is used to set the fixed real spatial distance between every two adjacent depth levels i, i+1.

[0044] Step 6: Based on the first prediction model and the first sample information, analyze the atomic spatial distribution of the current material sample to obtain the corresponding atom set; and based on the atom set, analyze the single atom density, dimer distribution and trimer distribution of the current material sample to obtain the corresponding first density, dimer set and trimer set; Specifically, this includes: Step 61, analyzing the atomic spatial distribution of the current material sample based on the first prediction model and the first sample information to obtain the corresponding atomic set; The atom set includes multiple first atoms; the atomic properties of the first atoms include atomic identifier and three-dimensional atomic coordinates. Specifically, this includes: Step 611, using the sample ADF-STEM image of the first sample information as the current first experimental image M. e1 The input is processed into the first prediction model; and the resulting deep semantic map M is processed. d As the corresponding current depth semantic graph; Step 612: Based on the image height H0, image width W0, total number of depth levels Z, pixel distance scale △p and depth layer spacing △d corresponding to the current depth semantic map, construct the corresponding first three-dimensional space using the Cartesian three-dimensional space; Here, in the first three-dimensional space of this embodiment, the XYZ axes only have positive axes, where the maximum value of the width X axis is W0×△p, the maximum value of the height Y axis is H0×△p, and the maximum value of the depth Z axis is Z×△d; the XY plane of the first three-dimensional space is parallel to the imaging plane of the sample ADF-STEM image; Step 613: Record the probability values ​​of the depth levels that exceed the preset first probability threshold among the H0×W0×Z depth level probabilities of the current depth semantic graph as the corresponding third probability. Here, the first probability threshold in this embodiment of the invention is a preset threshold parameter; Step 614: Take each third probability as the corresponding current probability; and based on the width pixel coordinate u, height pixel coordinate v, and depth level i of the first pixel corresponding to the current probability, calculate a set of corresponding width coordinates x, height coordinates y, and depth coordinates z in the first three-dimensional space: x = u × Δp, y = v × Δp, z = i × Δd; and form a corresponding three-dimensional atom coordinate (x, y, z) from the width coordinate x, height coordinate y, and depth coordinate z obtained this time; and assign a unique identifier to the current probability as the corresponding atom identifier; and form a corresponding first atom from the atom identifier corresponding to the current probability and the three-dimensional atom coordinate (x, y, z); Step 615: The obtained first atoms form the corresponding atom set; Step 62, and based on the atomic set, analyze the single atom density, dimer distribution and trimer distribution of the current material sample to obtain the corresponding first density, dimer set and trimer set; The dimer set includes multiple first dimers; the dimer attribute of the first dimer consists of a dimer identifier and two atom identifiers; the trimer set includes multiple first trimers; the trimer attribute of the first trimers consists of a trimer identifier and three atom identifiers. Specifically, this includes: Step 621, calculating the corresponding spatial volume V=(W0×△p)×(H0×△p)×(Z×△d) based on the image height H0, image width W0, total number of depth levels Z, pixel distance scale △p, and depth interlayer spacing △d corresponding to the sample ADF-STEM image; and calculating the total number of atoms N of the atom set. atom Statistical analysis was conducted; and based on the total number of atoms N atom The first density is obtained by calculating the single-atom density of the spatial volume V, where the first density = N.atom / V; Step 622: Calculate the coordinate spacing between every two first atoms in the atom set to obtain the corresponding first spacing; and based on the first spacing and the preset first atom spacing threshold, cluster all first atoms to obtain multiple corresponding first atom clusters; and record the first atom cluster containing only one first atom as a single atom cluster, and the first atom cluster containing two or more first atoms as a multi-atom cluster; and record the multi-atom cluster with a total number of atoms of 2 or 3 as the corresponding dimer atom cluster or trimer atom cluster; Here, the first atomic cluster in this embodiment of the invention is composed of one or more first atoms; the first distance between any two first atoms in the multi-atom cluster is less than the first atomic distance threshold; the first atomic distance threshold in this embodiment of the invention is a preset threshold parameter; In this invention, a single-atom cluster is composed of one first atom; a multi-atom cluster is composed of two or more first atoms; a dimer atom cluster is composed of two first atoms; and a trimer atom cluster is composed of three first atoms. Step 623: Identify whether the total number of dimer atom clusters is zero; if yes, set the corresponding dimer set to empty; if no, take each dimer atom cluster as the corresponding current atom cluster, assign a unique identifier to the current atom cluster as the corresponding dimer identifier, and form a corresponding first dimer by the dimer identifier and the atom identifiers of the two first atoms in the current atom cluster, and form the corresponding dimer set by all the obtained first dimers. Step 624: Identify whether the total number of trimer atom clusters is zero; if yes, set the corresponding trimer set to empty; if no, take each trimer atom cluster as the corresponding current atom cluster, assign a unique identifier to the current atom cluster as the corresponding trimer identifier, and form a corresponding first trimer by the trimer identifier and the atom identifiers of the three first atoms in the current atom cluster, and form the corresponding trimer set by all the obtained first trimers. Step 625, and output the first density, dimer set, and trimer set obtained this time as the analysis result.

[0045] Step 7: The corresponding atomic spatial distribution analysis report, composed of the atom set, the first density, the dimer set, and the trimer set, is fed back to the current user.

[0046] Figure 4This is a module structure diagram of an atomic spatial distribution analysis device based on an ADF-STEM image provided in Embodiment 2 of the present invention. This device can be a terminal device or server implementing the aforementioned method embodiments, or it can be a device that enables the aforementioned terminal device or server to implement the aforementioned method embodiments. For example, the device can be a device or chip system of the aforementioned terminal device or server. Figure 4 As shown, the device includes: a model building module 201, a dataset preparation module 202, a loop generation adversarial training module 203, a prediction model training module 204, a sample receiving module 205, a sample parsing module 206, and a parsing feedback module 207.

[0047] The model building module 201 is used to build a first GAN model, a second GAN model, and a first prediction model; the first GAN model is used to perform experimental style transfer on the image style of the ADF-STEM image; the second GAN model is used to perform simulated style transfer on the image style of the ADF-STEM image; and the first prediction model is used to perform atomic-level depth prediction based on the experimental style of the ADF-STEM image.

[0048] The dataset preparation module 202 generates a corresponding first sample set by collecting real samples of multi-system ADF-STEM images based on a preset first material system set; and generates a corresponding second sample set by simulating multi-system ADF-STEM images based on the first material system set; and constructs a first dataset based on the first and second sample sets.

[0049] The recurrent generative adversarial training module 203 is used to jointly train the first and second GAN models according to the first dataset based on the training mechanism of the recurrent generative adversarial network; after the joint training is completed, the second dataset is constructed based on the first generator of the first GAN model and the second sample set.

[0050] The prediction model training module 204 trains the first prediction model based on the second dataset.

[0051] The sample receiving module 205 is used to receive the first sample information input by the user after the prediction model training is completed; the first sample information corresponds to a material sample, and the material system of the current material sample satisfies the first material system set; the first sample information consists of the sample ADF-STEM image and the pixel distance scale △p and depth interlayer spacing △d of the current ADF-STEM image.

[0052] The sample analysis module 206 analyzes the atomic spatial distribution of the current material sample based on the first prediction model and the first sample information to obtain the corresponding atomic set; and analyzes the single atom density, dimer distribution and trimer distribution of the current material sample based on the atomic set to obtain the corresponding first density, dimer set and trimer set.

[0053] The analysis feedback module 207 provides the current user with an analysis report on the spatial distribution of atoms, consisting of the atomic set, the first density, the dimer set, and the trimer set.

[0054] The atomic spatial distribution analysis device based on ADF-STEM images provided in this embodiment of the invention can execute the method steps in the above method embodiment. Its implementation principle and technical effect are similar, and will not be repeated here.

[0055] It should be noted that the division of the various modules in the above device is merely a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, these modules can be implemented entirely in software via processing elements; they can be fully implemented in hardware; or some modules can be implemented by processing elements calling software, while others are implemented in hardware. For example, the model building module can be a separate processing element, or it can be integrated into a chip in the above device. Alternatively, it can be stored as program code in the memory of the above device, and its functions can be called and executed by a processing element. The implementation of other modules is similar. Moreover, these modules can be fully or partially integrated together, or they can be implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each of the above modules can be completed through integrated logic circuits in the hardware of the processor element or through software instructions.

[0056] For example, these modules can be one or more integrated circuits configured to implement the above methods, such as one or more Application Specific Integrated Circuits (ASICs), one or more Digital Signal Processors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs). As another example, when a module is implemented using processing element scheduler code, the processing element can be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. Furthermore, these modules can be integrated together as a System-on-a-Chip (SOC).

[0057] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. This computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the foregoing method embodiments are generated. The computer described above can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The aforementioned computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the aforementioned computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, Bluetooth, microwave, etc.) means. The aforementioned computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The aforementioned available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state disks (SSDs)).

[0058] Figure 5 This is a schematic diagram of an electronic device provided in Embodiment 3 of the present invention. This electronic device can be a terminal device or server implementing the methods of the aforementioned embodiments, or it can be a terminal device or server connected to the aforementioned terminal device or server implementing the methods of the aforementioned embodiments. Figure 5 As shown, the electronic device may include: a processor 301 (e.g., CPU), a memory 302, and a transceiver 303; the transceiver 303 is coupled to the processor 301, and the processor 301 controls the transmission and reception operations of the transceiver 303. The memory 302 may store various instructions for performing various processing functions and implementing the processing steps described in the foregoing embodiments. Preferably, the electronic device involved in the embodiments of the present invention further includes: a power supply 304, a system bus 305, and a communication port 306. The system bus 305 is used to realize communication connections between components. The communication port 306 is used for communication between the electronic device and other peripherals.

[0059] exist Figure 5The system bus 305 mentioned can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This system bus can be divided into address bus, data bus, control bus, etc. For ease of representation, it is represented by only one thick line in the figure, but this does not indicate that there is only one bus or one type of bus. The communication interface is used to enable communication between the database access device and other devices (e.g., clients, read-write libraries, and read-only libraries). Memory may include Random Access Memory (RAM) and may also include Non-Volatile Memory, such as at least one disk storage device.

[0060] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), graphics processing units (GPUs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0061] It should be noted that the embodiments of the present invention also provide a computer-readable storage medium storing instructions that, when run on a computer, cause the computer to perform the methods and processes provided in the above embodiments.

[0062] This invention provides a method, apparatus, electronic device, and computer-readable storage medium for analyzing atomic spatial distribution based on ADF-STEM images. As described above, this invention constructs two GAN models (a first GAN model and a second GAN model) and one prediction model (a first prediction model). The first GAN model performs experimental style transfer on the image style of the ADF-STEM image (referred to as the simulated image) generated by a simulation tool. The experimental style refers to the image style of the ADF-STEM image (referred to as the experimental image) obtained in a real experimental scenario. The second GAN model performs simulated style transfer on the image style of the experimental image. The simulated style refers to the image style of the ADF-STEM image generated by the simulation tool. The first prediction model performs atomic-level depth prediction based on the ADF-STEM image with the experimental style, that is, predicts the atomic depth semantic map of the experimental image. The invention constructs an experimental graph sample set (first sample set) by collecting experimental graph samples from multiple systems (the range of material systems specified in the first material system set), and a simulation graph sample set (second sample set) by generating simulation graphs from multiple systems. An unsupervised dataset (first dataset) is then constructed by pairwise combining the experimental graphs and simulation graphs from the first and second sample sets. Following the unsupervised game-theoretic training method of the CycleGAN framework, the two GAN models are jointly trained using the unsupervised dataset. After joint training, a cross-system, cross-experimental style labeled dataset (second dataset) is constructed based on the first generator of the first GAN model and the second sample set. The first prediction model is then trained using a supervised training method based on the second dataset. After the prediction model training is complete, the first prediction model is used to perform three-dimensional atomic spatial distribution analysis on the ADF-STEM graphs of any material system sample satisfying the first material system set. In this embodiment of the invention, after obtaining the atomic spatial distribution analysis result (atomic set), further analysis is performed on the single-atom density, dimer distribution, and trimer distribution of the current material sample based on this analysis result. The embodiments of the present invention achieve atomic-level three-dimensional spatial distribution analysis through a prediction model; on the other hand, by constructing and training datasets across systems and experimental styles, the analytical accuracy can be maintained and the model generalization ability can be improved in cross-system and cross-experimental styles; furthermore, without the need for a large number of complex basic calculations, the analysis time is shortened, the processing efficiency is improved, and it is adaptable to high-throughput computing scenarios.

[0063] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented in hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0064] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for analyzing the spatial distribution of atoms based on ADF-STEM images, characterized in that, The method includes: A first GAN model, a second GAN model, and a first prediction model are constructed. The first GAN model is used to perform experimental style transfer on the image style of the ADF-STEM image. The second GAN model is used to perform simulated style transfer on the image style of the ADF-STEM image. The first prediction model is used to perform atomic-level depth prediction based on the experimental style of the ADF-STEM image. Based on a preset first material system set, real samples of multi-system ADF-STEM images are collected to generate a corresponding first sample set; and based on the first material system set, multi-system ADF-STEM images are simulated to generate a corresponding second sample set; and a first dataset is constructed based on the first and second sample sets. The training mechanism of recurrent generative adversarial networks is used to jointly train the first and second GAN models based on the first dataset; after the joint training is completed, the second dataset is constructed based on the first generator of the first GAN model and the second sample set. The first prediction model is trained based on the second dataset; After the prediction model training is completed, the first sample information input by the user is received; the first sample information corresponds to a material sample, and the material system of the current material sample satisfies the first material system set; the first sample information consists of the sample ADF-STEM image and the pixel distance scale △p and depth interlayer spacing △d of the current ADF-STEM image; Based on the first prediction model and the first sample information, the atomic spatial distribution of the current material sample is analyzed to obtain the corresponding atom set; and based on the atom set, the single atom density, dimer distribution and trimer distribution of the current material sample are analyzed to obtain the corresponding first density, dimer set and trimer set; A spatial distribution analysis report of atoms, consisting of the set of atoms, the first density, the set of dimers, and the set of trimers, is fed back to the current user.

2. The method for analyzing the spatial distribution of atoms based on ADF-STEM images according to claim 1, characterized in that, The first GAN model is composed of the first generator and the first discriminator connected together; The model input of the first GAN model is the first simulation graph M. s1 The model output is the first probability p1; The first simulation diagram M s1 ADF-STEM image; The first probability p1 is the first experimental plot M generated by the first generator. e1 The probability of the first experimental plot M being true or false; the value of the first probability p1 is between 0 and 1, and the larger the value, the higher the probability of the first experimental plot M being true or false. e1 The higher the probability of the true image and the smaller the value, the better the first experimental image M. e1 The higher the probability of a fake image; The first generator is used to process the first simulation graph M s1 The image style was subjected to experimental style transfer processing to obtain the corresponding first experimental image M. e1 ; in, , G S-E () represents the model inference expression of the first generator, and θ1 represents the model parameters of the first generator; The first discriminator is used to analyze the first experimental diagram M. e1 The probability of a true or false image is used to determine the corresponding first probability p1; in, , D E () represents the model inference expression of the first discriminator, and θ2 represents the model parameters of the first discriminator; The second GAN model is composed of a second generator and a second discriminator connected together; The input to the second GAN model is the second experimental graph M. e2 The model output is the second probability p2; Second experimental diagram M e2 ADF-STEM image; The second probability p2 is the second simulation graph M generated by the second generator. s2 The true or false probability; the second probability p2 takes a value between 0 and 1, the larger the value, the higher the probability of the second simulation graph M. s2 The higher the probability of the true image and the smaller the value, the better the second simulated image M. s2 The higher the probability of a fake image; The second generator is used for the second experimental graph M e2 The image style is subjected to simulated style transfer processing to obtain the corresponding second simulated image M. s2 ; in, , G E-S () represents the model inference expression of the second generator, and θ3 represents the model parameters of the second generator; The second discriminator is used to analyze the second simulation image M. s2 The probability of a true or false image is used to determine the corresponding second probability p2; in, , D S () represents the model inference expression of the second discriminator, and θ4 represents the model parameters of the second discriminator; The first prediction model consists of a first feature extraction network and a depth prediction head; the input of the first feature extraction network is connected to the model input of the first prediction model, and the output is connected to the input of the depth prediction head; the output of the depth prediction head is connected to the model output of the first prediction model. The model input of the first prediction model is the first experimental graph M. e1 The model output is a deep semantic graph M. d ; First experimental diagram M e1 The shape is denoted as H0×W0×C0, where H0, W0, and C0 are respectively the first experimental diagram M. e1 Image height, image width, and pixel feature dimensions; The deep semantic graph M d The shape is H0×W0×Z, and the preset total number of depth levels Z is a positive integer greater than 1; the depth semantic graph M d It consists of H0×W0 first pixels, and the pixel semantic feature of each first pixel consists of Z depth level probabilities; each depth level probability corresponds to a depth level i, 1≤i≤Z; the value of each depth level probability is between 0 and 1; the larger the depth level probability, the greater the probability that there is an atom at the three-dimensional spatial position corresponding to the current depth level in the depth direction of the current pixel; the smaller the depth level probability, the smaller the probability that there is an atom at the three-dimensional spatial position corresponding to the current depth level in the depth direction of the current pixel. The first feature extraction network is implemented based on the U-Net model; the first feature extraction network is used to extract features from the first experimental image M. e1 Feature extraction is performed to obtain the corresponding first feature map H; the shape of the first feature map H is H0×W0×C. H C H Let C be the pixel feature dimension of the first feature map H. H >Z; The depth prediction head is implemented based on a convolutional network and a sigmoid function. The depth prediction head uses a built-in convolutional network to perform feature mapping on the first feature map H to obtain a depth feature map of shape H0×W0×Z. The depth feature map consists of H0×W0 first pixel features, each of which consists of Z depth feature values. The sigmoid function is used to calculate the depth level probability corresponding to each depth feature value of each first pixel feature. The Z depth level probabilities corresponding to each first pixel feature are used to form a pixel semantic feature for the corresponding first pixel. The resulting H0×W0 pixel semantic features of the first pixels form the corresponding depth semantic map M. d ; The first set of material systems includes multiple first material systems; all of the first material systems can be used as single-atom catalyst supports; The first sample set includes multiple first sample images; all first sample images are real ADF-STEM images collected from the experimental environment; each first sample image corresponds to a class of the first material system; the set of material systems in the first sample set is consistent with the set of first material systems. The second sample set includes multiple second sample plots; all second sample plots are simulated ADF-STEM plots generated by simulation tools; each second sample plot corresponds to a class of the first material system; the set of material systems corresponding to the second sample set is consistent with the set of the first material systems; the simulation tools include abTEM, xHREM, and Multem tools; The first dataset includes multiple first data records; the first data records include a first training simulation graph and a first training experiment graph; The second dataset includes multiple second data records; the second data records include a second training experiment graph and a first label semantic graph; the graph data structure of the first label semantic graph is the same as that of the deep semantic graph M. d Maintain consistency; The set of atoms includes multiple first atoms; the atomic attributes of the first atoms include atomic identifiers and three-dimensional atomic coordinates; The dimer set includes multiple first dimers; the dimer attribute of the first dimer consists of a dimer identifier and two atomic identifiers; The set of trimers includes multiple first trimers; the trimer attribute of the first trimer consists of a trimer identifier and three atom identifiers.

3. The method for analyzing the spatial distribution of atoms based on ADF-STEM images according to claim 2, characterized in that, The construction of the first dataset based on the first and second sample sets specifically includes: Each of the first sample images in the first sample set is taken as a corresponding first training experiment image, and each of the second sample images in the second sample set is taken as a corresponding first training simulation image; each of the first training simulation images is taken as the current simulation image; and the current simulation image and each of the first training experiment images are combined to form a corresponding first data record; and all the obtained first data records are combined to form a corresponding first dataset.

4. The method for analyzing the spatial distribution of atoms based on ADF-STEM images according to claim 2, characterized in that, The training mechanism of the cyclic generative adversarial network jointly trains the first and second GAN models based on the first dataset, specifically including: Step 41: Calculate the total number of records in the first dataset to obtain the corresponding total number N. tr1 And the first training simulation graph and the first training experiment graph of each of the first data records are recorded as the corresponding simulation graph M. S(j) Experimental diagram M E(j) ; Where 1 ≤ index j ≤ N tr1 ; Step 42, N tr1 The simulation diagram M described in the group S(j) and the experimental diagram M E(j) Substitute the preset first model loss function L M1 The corresponding first loss value is obtained through calculation; Wherein, the first model loss function L M1 The joint loss implementation based on adversarial loss and cycle consistency loss is as follows: ， ， ， ; L adv1 Let L be the adversarial loss function of the first GAN model. adv2 Let L be the adversarial loss function of the second GAN model. cyc Let λ be the cycle consistency loss function; cyc The preset weighting coefficients for cycle consistency loss; Step 43: Identify whether the first loss value meets the preset first loss value range; if the first loss value does not meet the first loss value range, then optimize the first model loss function L according to the preset objective function O1 and the preset first model optimizer. M1 The model parameters θ1 and θ3 that achieve the minimum value, and the loss function L of the first model that makes the first model reach its minimum value. M1 The model parameters θ2 and θ4 that have reached their maximum values ​​are subjected to one round of parameter modulation, and the process returns to step 42 when the parameter modulation ends; if the first loss value meets the first loss value range, then training is stopped and the joint training is confirmed to be over. Wherein, the preset objective function O1 is: ； The first model optimizer includes the Adam optimizer.

5. The method for analyzing the spatial distribution of atoms based on ADF-STEM images according to claim 2, characterized in that, The construction of the second dataset based on the first generator of the first GAN model and the second sample set specifically includes: Use each of the second sample graphs in the second sample set as the current simulation graph; And the current simulation graph is used as the current first simulation graph M. s1 The first generator is input for processing, and the first experimental diagram M obtained from this processing is... e1 As the corresponding second training experiment diagram; The system identifies the atomic center points of all atoms in the current simulation graph and the depth level of each atomic center point; and based on the depth semantic graph M... d The image data format is the first label semantic map corresponding to the initialization of the current simulation image; and according to the atomic center point identification result, the Z depth level probabilities corresponding to each first pixel are labeled on the first label semantic map; on the first label semantic map, among the Z depth level probabilities of the first pixel that matches each atomic center point, there is only one probability of 1 and the remaining Z-1 probabilities are all 0, and the depth level i corresponding to the depth level probability of the depth level that is 1 matches the center point depth level identification result of the current atomic center point; on the first label semantic map, the Z depth level probabilities of each first pixel that does not match any atomic center point are all 0. The second data record is composed of the second training experiment graph corresponding to the current simulation graph and the first label semantic graph; and the second dataset is composed of all the obtained second data records.

6. The method for analyzing the spatial distribution of atoms based on ADF-STEM images according to claim 2, characterized in that, Training the first prediction model based on the second dataset specifically includes: Step 61: Calculate the total number of records in the second dataset to obtain the corresponding total number N. tr2 And the second training experimental graph and the first label semantic graph of each second data record in the second dataset are denoted as the corresponding experimental graph M. E(k) Tag semantic graph M tag(k) ; Among them, the experimental diagram M E(k) The image height and image width are denoted as H. k W k The tag semantic graph M tag(k) By H k ×W k label vector mt k,r Composition; the label vector mt k,r Given the probability mt of Z labels k,r,q Composition; 1 ≤ index k ≤ N tr2 1 ≤ index r ≤ H k ×W k , 1 ≤ index q ≤ Z; Step 62, transfer each of the experimental diagrams M E(k) As the corresponding first experimental diagram M e1 The first prediction model is input for processing, and the deep semantic map M obtained from this processing is... d As the corresponding predictive semantic graph M pre(k) ; Wherein, the predicted semantic graph M pre(k) By H k ×W k The prediction vector mk,r is composed of Z prediction probabilities mk,r,q. Step 63, obtain N tr2 The group's predicted semantic graph M pre(k) and the tag semantic graph M tag(k) Substitute the preset second model loss function L M2 The corresponding second loss value is obtained through calculation; Wherein, the second model loss function L M2 The implementation is based on the joint loss of weighted cross-entropy loss and Dice loss, specifically as follows: ， , ； L wce For the preset weighted cross-entropy loss function, L dice α is the preset Dice loss function, and α is the preset balancing parameter; w q The weights are for the q-th class. Step 64: Identify whether the second loss value meets the preset second loss value range; if the second loss value does not meet the second loss value range, then based on the preset second model optimizer, move towards making the second model loss function L... M2 The model parameters of the first prediction model are modulated once in the direction that reaches the minimum value, and the process returns to step 62 when the parameter modulation ends; if the second loss value meets the range of the second loss value, then training is stopped and the prediction model training is confirmed to be completed. The second model optimizer includes the Adam optimizer and the SGD optimizer.

7. The method for analyzing the spatial distribution of atoms based on ADF-STEM images according to claim 2, characterized in that, The step of analyzing the atomic spatial distribution of the current material sample based on the first prediction model and the first sample information to obtain the corresponding atom set specifically includes: The sample ADF-STEM image of the first sample information is used as the current first experimental image M. e1 The first prediction model is input for processing; and the deep semantic map M obtained from this processing is... d As the corresponding current depth semantic graph; Based on the image height H0, image width W0, total number of depth levels Z, pixel distance scale △p, and depth layer spacing △d of the first sample information, a Cartesian three-dimensional space is constructed to obtain the corresponding first three-dimensional space; the XYZ axes of the first three-dimensional space only have positive axes, where the maximum value of the width X axis is W0×△p, the maximum value of the height Y axis is H0×△p, and the maximum value of the depth Z axis is Z×△d; the XY plane of the first three-dimensional space is parallel to the imaging plane of the sample ADF-STEM image; Among the H0×W0×Z depth level probabilities of the current depth semantic graph, the depth level probabilities whose probability values ​​exceed a preset first probability threshold are all recorded as the corresponding third probabilities. Each of the third probabilities is taken as the corresponding current probability; and based on the width pixel coordinate u, height pixel coordinate v, and depth level i of the first pixel corresponding to the current probability, a set of corresponding width coordinates x, height coordinates y, and depth coordinates z in the first three-dimensional space are calculated, x=u×△p, y=v×△p, z=i×△d; and the width coordinate x, height coordinate y, and depth coordinate z obtained this time form a corresponding three-dimensional atom coordinate (x,y,z); and a unique identifier is assigned to the current probability as the corresponding atom identifier; and the atom identifier corresponding to the current probability and the three-dimensional atom coordinate (x,y,z) form a corresponding first atom; The corresponding set of atoms is composed of all the first atoms obtained.

8. The method for analyzing the spatial distribution of atoms based on ADF-STEM images according to claim 7, characterized in that, The step of analyzing the single-atom density, dimer distribution, and trimer distribution of the current material sample based on the atomic set to obtain the corresponding first density, dimer set, and trimer set specifically includes: Based on the image height H0, image width W0, total number of depth levels Z, pixel distance scale Δp, and depth interlayer spacing Δd corresponding to the sample ADF-STEM image, the corresponding spatial volume V = (W0 × Δp) × (H0 × Δp) × (Z × Δd) is calculated; and the total number of atoms N of the atom set is calculated. atom Perform statistics; and based on the total number of atoms N atom The first density is obtained by calculating the single-atom density of the spatial volume V, where the first density = N. atom / V; The coordinate spacing between any two first atoms in the atom set is calculated to obtain the corresponding first spacing; and based on the first spacing and a preset first atomic spacing threshold, all first atoms are clustered to obtain multiple corresponding first atom clusters; and the first atom cluster containing only one first atom is recorded as a single atom cluster, and the first atom cluster containing two or more first atoms is recorded as a multi-atom cluster; and the multi-atom cluster with a total number of atoms of 2 or 3 is recorded as the corresponding dimer atom cluster or trimer atom cluster; the first atom cluster is composed of one or more first atoms; the first spacing between any two first atoms in the multi-atom cluster is less than the first atomic spacing threshold. The system identifies whether the total number of the dimer atom clusters is zero; if so, the corresponding dimer set is set to empty; if not, each dimer atom cluster is taken as the corresponding current atom cluster, and a unique identifier is assigned to the current atom cluster as the corresponding dimer identifier. The dimer identifier and the atom identifiers of the two first atoms in the current atom cluster are combined to form a corresponding first dimer, and all the obtained first dimers are combined to form the corresponding dimer set. The system identifies whether the total number of the trimer atom clusters is zero; if so, the corresponding trimer set is set to empty; if not, each trimer atom cluster is taken as the corresponding current atom cluster, and a unique identifier is assigned to the current atom cluster as the corresponding trimer identifier. The trimer identifier and the atom identifiers of the three first atoms in the current atom cluster are combined to form a corresponding first trimer, and all the obtained first trimers are combined to form the corresponding trimer set. The first density, the dimer set, and the trimer set obtained in this analysis will be output as the results of this analysis.

9. An apparatus for performing the atomic spatial distribution analysis method based on ADF-STEM images as described in any one of claims 1-8, characterized in that, The device includes: a model building module, a dataset preparation module, a loop generation adversarial training module, a prediction model training module, a sample receiving module, a sample parsing module, and a parsing feedback module; The model building module is used to build a first GAN model, a second GAN model, and a first prediction model; the first GAN model is used to perform experimental style transfer on the image style of the ADF-STEM image; the second GAN model is used to perform simulated style transfer on the image style of the ADF-STEM image; and the first prediction model is used to perform atomic-level depth prediction based on the experimental style of the ADF-STEM image. The dataset preparation module generates a corresponding first sample set by collecting real samples of multi-system ADF-STEM images based on a preset first material system set; and generates a corresponding second sample set by simulating multi-system ADF-STEM images based on the first material system set; and constructs a first dataset based on the first and second sample sets. The recurrent generative adversarial training module is used to jointly train the first and second GAN models according to the first dataset based on the training mechanism of the recurrent generative adversarial network; after the joint training is completed, a second dataset is constructed based on the first generator of the first GAN model and the second sample set. The prediction model training module trains the first prediction model based on the second dataset; The sample receiving module is used to receive the first sample information input by the user after the prediction model training is completed; the first sample information corresponds to a material sample, and the material system of the current material sample satisfies the first material system set; the first sample information consists of the sample ADF-STEM image and the pixel distance scale △p and depth interlayer spacing △d of the current ADF-STEM image; The sample analysis module analyzes the atomic spatial distribution of the current material sample based on the first prediction model and the first sample information to obtain the corresponding atom set; and analyzes the single atom density, dimer distribution and trimer distribution of the current material sample based on the atom set to obtain the corresponding first density, dimer set and trimer set; The analysis feedback module provides the current user with an atomic spatial distribution analysis report composed of the atom set, the first density, the dimer set, and the trimer set.

10. An electronic device, characterized in that, include: Memory, processor, and transceiver; The processor is configured to be coupled to the memory, read and execute instructions in the memory to implement the method according to any one of claims 1-8; The transceiver is coupled to the processor, and the processor controls the transceiver to send and receive messages.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a computer, cause the computer to perform the method described in any one of claims 1-8.

Images