Atomic level electron tomography three-dimensional reconstruction method, device and medium

Abstract

This invention provides an atomic-level electron tomography 3D reconstruction method. Its core is to match the simulated projection of an atomic model with experimental images through iterative optimization. The method first constructs a 3D space based on the sample contour and initializes the atomic model parameters. Then, it acquires experimental images from multiple angles and their imaging parameters. In each iteration, a simulated projection image is generated based on the current atomic model and imaging parameters, and the difference between it and the corresponding experimental image is calculated as the loss value. An optimization algorithm dynamically adjusts atomic parameters (position, size, etc.) and imaging parameters (such as focus distance and aberrations) to minimize the total loss. This process includes intelligent simplification steps, which can merge or delete atoms to optimize model complexity. Finally, using the converged high-precision atomic model, 3D reconstructed images with arbitrary viewpoints and depths can be generated. This method can achieve high-precision atomic-level 3D reconstruction without completely relying on 360-degree scan data, helping to reduce data acquisition costs.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional imaging, and in particular to a method, apparatus and medium for three-dimensional reconstruction by atomic-level electron tomography. Background Technology

[0002] After spherical aberrations in transmission electron microscopy (TEM) are corrected, materials can be imaged with sub-angstrom resolution, revealing atomic-level structural features. Typically, TEM images produce two-dimensional projection data, which cannot reflect the three-dimensional structural characteristics of materials. Atomic electron tomography (AET) utilizes a series of projection images at various tilt angles, and through inverse transformation, obtains the three-dimensional atomic spatial coordinates and atom types from multiple two-dimensional image data. In recent years, AET technology has achieved a series of important results in fields such as amorphous and metallic crystals.

[0003] However, standard reconstruction algorithms used in AET technology require 360-degree coverage to obtain a complete three-dimensional structure, but due to limitations in electron microscopy principles, they have the following drawbacks: 1. The inability to achieve 360° coverage leads to the missing wedge problem, resulting in a decrease in image quality. Therefore, the three-dimensional structure itself has defects. 2. Reconstruction at the voxel level incurs a very high computational cost.

[0004] To address the above problem 2, Chinese patent CN116959637A discloses a three-dimensional reconstruction method, apparatus, and computer device based on depth-dependent electron beams. This method obtains the internal structure by scattering points, then slices the data to obtain atomic projections, and then convolves the projections with a pre-stored electron beam matrix. The result of the convolution is used as the calculated image. In this way, it is no longer necessary to store every voxel point; simply convolving the slices with the pre-stored electron beam matrix is ​​sufficient to obtain an image with any viewpoint and any depth.

[0005] However, while the aforementioned Chinese patent CN116959637A alleviated the measurement and calculation issues during the reconstruction phase to some extent, the following problems still exist: 1) In the reconstruction process, it requires high-density, unconstrained spatial dot distribution. In practical applications, this will lead to the aggregation of atomic dots, causing stretching defects in the imaging effect. Therefore, it does not solve the problem of missing wedges caused by the inability to achieve 360-degree coverage.

[0006] 2) When the target object to be reconstructed is very large, since it uses the method of pre-storing electron beam matrices, if the target object is very large, there are many slices of different electron beams in the same direction, which means that a lot of electron beam matrices need to be stored, and the dimension of each electron beam matrix will also be very large. When the dimension of the electron beam matrix is ​​very large, the calculation speed of the convolution process will be very slow, and at the same time, it will cause problems such as low efficiency, decreased accuracy, and even failure of the optimization iterative algorithm. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of the prior art by providing an atomic-level electron tomography three-dimensional reconstruction method, apparatus, and medium.

[0008] The objective of this invention can be achieved through the following technical solutions: An atomic-level electron tomography three-dimensional reconstruction method includes: Step S1: Construct the object space based on the object's surface contour; Step S2: Initialize atomic parameters and arrange atoms in the object space based on atomic parameters to obtain an initial first atomic-level model, wherein the atomic parameters include the number of atoms, as well as the size, position and intensity distribution of each atom; Step S3: Acquire multiple actual scanned images, as well as the observation direction and nominal focus distance of each image, initialize aberrations, and use the nominal focus distance as the initialized focus distance; Step S4: Based on the current atomic-level model, and the observation direction, focusing distance and aberration corresponding to each scan image, generate projection images corresponding to each scan image respectively; Step S5: Calculate the difference between each projected image and the corresponding scanned image as the loss value of each projected image, and obtain the total loss value based on the loss values ​​of all projected images. Step S6: Dynamically optimize the atomic parameters based on the total loss value, and obtain the final atomic-level model corresponding to the focus distance and aberration of each scanned image; Step S7: Generate reconstructed images with arbitrary viewpoints and depths based on the final atomic-level model.

[0009] Step S6 includes: Step S6-1: Determine whether the current total loss value is better than the current optimal total loss. If yes, then take the current total loss value as the optimal total loss. Step S6-2: Determine whether the current optimal total loss is less than the pre-configured first loss threshold or meets the second exit condition. If yes, then take the first atomic-level model corresponding to the current optimal total loss as the final atomic-level model and execute step S7. Otherwise, execute step S6-3. Step S6-3: Determine whether the first exit condition is met. If yes, proceed to step S6-4; otherwise, proceed to step S6-5. Step S6-4: Reduce the number of atoms, update the size, position and intensity distribution of each atom based on the optimization algorithm, as well as the focus distance and aberration corresponding to each scan image, and return to step S4; Step S6-5: Update the atomic parameters, as well as the focus distance and aberration corresponding to each scanned image, based on the optimization algorithm, and return to step S4.

[0010] The methods for reducing the number of atoms in step S6-4 include: Merge multiple atoms whose pairwise distances are all less than a threshold into one atom; Atoms with intensity distributions below a certain threshold are deleted.

[0011] The process in step S4 of generating a projection image corresponding to a single scan image based on the current atomic-level model and the observation direction, focus distance, and aberrations corresponding to that scan image includes: Step S4-1: Determine the field of view based on the observation direction, focusing distance, and aberrations of the scanned image, and filter out atoms outside the field of view; Step S4-2: Based on the accelerating voltage, convergence angle, aberration, defocusing amount, and electron source size of the transmission electron microscope, the two-dimensional matrix of the electron beam at different depths is calculated analytically. Step S4-3: Slice the current first atomic level model to obtain slice images corresponding to different depths; Step S4-4: Convolve each slice image with a two-dimensional matrix of the same depth to obtain convolution images corresponding to different depths, and then superimpose all the convolution images to obtain the projection image.

[0012] All atomic parameters have been normalized.

[0013] The total loss value is the sum of the loss values ​​of all projected images.

[0014] The optimization algorithm is a combination of adaptive momentum gradient optimization algorithm and particle swarm optimization algorithm.

[0015] The atoms are spherical in shape.

[0016] An atomic-level electron tomography three-dimensional reconstruction device includes a memory, a processor, and a program stored in the memory, wherein the processor executes the program to implement the method described above.

[0017] A storage medium having a program stored thereon, which, when executed, implements the method described above.

[0018] Compared with the prior art, the present invention has the following beneficial effects: 1. By using a point-splitting method to simulate the microstructure within an object, and using the error between the simulated image and the real scanned image as the loss value, the method approximates the real structure by gradually reducing the number of atoms. This solves the problems of existing technologies that cannot perform omnidirectional scanning and that slow rendering speed is caused by too many voxels, while improving the accuracy of 3D microstructure modeling.

[0019] 2. By setting the "first loss threshold" and "first / second exit conditions", the system can automatically stop when the accuracy requirements are met, or intelligently switch to the "reduce the number of atoms" simplification mode when the optimization gets stuck in a bottleneck, thereby jumping out of the local optimum and finding a simpler model again, balancing the model complexity and fitting accuracy, and avoiding overfitting or invalid iteration.

[0020] 3. By simplifying the model through two physically meaningful methods—"merging neighboring atoms" and "removing weak atoms"—the physical interpretability and computational efficiency of the reconstruction results are improved. The merging operation is suitable for rationalizing an overly dense initial atomic distribution, while the deletion operation can remove redundant or erroneous atoms that contribute little to the image. This mechanism ensures that the final model not only fits the image but also conforms to the prior knowledge that "matter is composed of discrete atoms," making the reconstructed atomic model closer to reality. It also overcomes the missing wedge problem caused by the inability to collect data from 360 degrees.

[0021] 4. The scattering and interference effects that occur when the electron beam passes through the sample are simulated in detail, especially considering factors such as aberrations and defocus. Compared with a simple linear projection model, this can greatly improve the realism of the simulated image, thus making subsequent loss comparison and parameter optimization more accurate.

[0022] 5. An analytical method is used to calculate the electron beam matrix at different angles / layers, avoiding the direct use of high-dimensional electron beam matrices as optimization variables. The optimization parameters are simplified to a few parameters such as defocus, astigmatism, and coma. This greatly improves the algorithm's iteration efficiency and stability, and enhances the accuracy of the reconstruction model, while ensuring that the imaging process meets the physical principles.

[0023] 6. It ensures that parameters of different dimensions and orders of magnitude in the optimization algorithm are on the same scale, which can accelerate the convergence of the optimization process, improve the numerical stability and search efficiency of the optimization algorithm, and avoid the dominance of the entire optimization process due to the excessive variation range of some parameters.

[0024] 7. The optimized atomic-level model must be able to reproduce the experimental image well under all observation angles, rather than matching only a few specific angles. This ensures that the reconstructed 3D model has consistency under various viewpoints from the objective function level, and improves the global credibility of the reconstruction results. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of the main steps of the method of the present invention. Detailed Implementation

[0026] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0027] An atomic-level electron tomography 3D reconstruction method simulates the microstructure within an object by using a point-splitting method and uses the error between the simulated image and the real scanned image as a loss value. It approximates the real structure by gradually reducing the number of atoms, thereby solving the problems of the inability to scan in all directions and the slow rendering speed caused by too many voxels in the existing technology, while improving the accuracy of 3D microstructure modeling.

[0028] like Figure 1 As shown, it includes: Step S1: Construct the object space based on the object's surface contour; Step S2: Initialize atomic parameters and arrange atoms in the object space based on atomic parameters to obtain the initial first atomic level model. The atomic parameters include the number of atoms, as well as the size, position, and intensity distribution of each atom. Generally, in this embodiment, the atoms are spherical. The field strength distribution specifically refers to the distribution of the projected intensity, representing the intensity of the fiber penetration. In particular, the initial number of atoms should be as large as possible, and the atomic parameters are all normalized.

[0029] Step S3: Acquire multiple actual scanned images, as well as the observation direction and nominal focus distance of each image, initialize aberrations, and use the nominal focus distance as the initialized focus distance; The observation direction mentioned above generally refers to the observation direction relative to the sample.

[0030] Step S4: Based on the current atomic-level model, and the observation direction, focusing distance and aberration corresponding to each scan image, generate projection images corresponding to each scan image respectively; Specifically, step S4, which generates a projection image corresponding to the scanned image based on the current atomic-level model and the observation direction, focus distance, and aberrations corresponding to a single scanned image, includes: Step S4-1: Determine the field of view based on the observation direction, focusing distance, and aberrations of the scanned image, and filter out atoms outside the field of view; Step S4-2: Based on the accelerating voltage, convergence angle, aberration, defocusing amount, and electron source size of the transmission electron microscope, the two-dimensional matrix of the electron beam at different depths is calculated analytically. The specific analysis process can be found in some existing technologies. To avoid obscuring the purpose of this invention, it will not be elaborated further.

[0031] An analytical method is used to calculate the electron beam matrix at different angles / layers, avoiding the direct use of high-dimensional electron beam matrices as optimization variables. The optimization parameters are simplified to a few parameters such as defocus, astigmatism, and coma. This greatly improves the algorithm's iteration efficiency and stability, and enhances the accuracy of the reconstruction model, while ensuring that the imaging process meets the physical principles.

[0032] Step S4-3: Slice the current first atomic level model to obtain slice images corresponding to different depths; Step S4-4: Convolve each slice image with a two-dimensional matrix of the same depth to obtain convolution images corresponding to different depths, and then superimpose all the convolution images to obtain the projection image.

[0033] Step S5: Calculate the difference between each projected image and the corresponding scanned image as the loss value of each projected image, and obtain the total loss value based on the loss values ​​of all projected images. Specifically, the loss value of the projected image is calculated by taking the difference between each pixel of the projected image and the scanned image, and then summing the differences after taking the modulus.

[0034] In this embodiment, the total loss value is the sum of the loss values ​​of all projected images.

[0035] Step S6: Dynamically optimize the atomic parameters based on the total loss value, and obtain the final atomic-level model corresponding to the focus distance and aberration of each scanned image; Step S6 includes: Step S6-1: Determine whether the current total loss value is better than the current optimal total loss. If yes, then take the current total loss value as the optimal total loss. Step S6-2: Determine whether the current optimal total loss is less than the pre-configured first loss threshold or meets the second exit condition. If yes, then take the first atomic-level model corresponding to the current optimal total loss as the final atomic-level model and execute step S7. Otherwise, execute step S6-3. Step S6-3: Determine whether the first exit condition is met. If yes, proceed to step S6-4; otherwise, proceed to step S6-5. Step S6-4: Reduce the number of atoms, update the size, position and intensity distribution of each atom based on the optimization algorithm, as well as the focus distance and aberration corresponding to each scan image, and return to step S4; The methods for reducing the number of atoms include: Merge multiple atoms whose pairwise distances are all less than a threshold into one atom; Atoms with intensity distributions below a certain threshold are deleted.

[0036] In addition, the optimization algorithms include genetic algorithms or particle swarm optimization algorithms.

[0037] Step S6-5: Update the atomic parameters, as well as the focus distance and aberration corresponding to each scanned image, based on the optimization algorithm, and return to step S4.

[0038] By setting a "first loss threshold" and a "first / second exit condition", the system can automatically stop when the accuracy requirements are met, or intelligently switch to a simplified mode of "reducing the number of atoms" when the optimization gets stuck in a bottleneck. This allows it to break out of local optima and find a simpler model, balancing model complexity and fitting accuracy, and avoiding overfitting or invalid iterations.

[0039] Step S7: Generate reconstructed images with arbitrary viewpoints and depths based on the final atomic-level model.

[0040] This application solves the problem of reconstructing the microstructure of large objects in existing technologies by using a reverse solution or reconstruction process that relies only on a small number of real scanned images and optimizes atomic parameters to match the simulated projection with the real image. It can be applied in fields such as flaw detection in precision instruments.

[0041] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A method for three-dimensional reconstruction using atomic-level electron tomography, characterized in that, include: Step S1: Construct the object space based on the object's surface contour; Step S2: Initialize atomic parameters and arrange atoms in the object space based on atomic parameters to obtain an initial first atomic-level model, wherein the atomic parameters include the number of atoms, as well as the size, position and intensity distribution of each atom; Step S3: Acquire multiple actual scanned images, as well as the observation direction and nominal focus distance of each image, initialize aberrations, and use the nominal focus distance as the initialized focus distance; Step S4: Based on the current atomic-level model, and the observation direction, focusing distance and aberration corresponding to each scan image, generate projection images corresponding to each scan image respectively; Step S5: Calculate the difference between each projected image and the corresponding scanned image as the loss value of each projected image, and obtain the total loss value based on the loss values ​​of all projected images. Step S6: Dynamically optimize the atomic parameters based on the total loss value, and obtain the final atomic-level model corresponding to the focus distance and aberration of each scanned image; Step S7: Generate reconstructed images with arbitrary viewpoints and depths based on the final atomic-level model.

2. The atomic-level electron tomography three-dimensional reconstruction method according to claim 1, characterized in that, Step S6 includes: Step S6-1: Determine whether the current total loss value is better than the current optimal total loss. If yes, then take the current total loss value as the optimal total loss. Step S6-2: Determine whether the current optimal total loss is less than the pre-configured first loss threshold or meets the second exit condition. If yes, then take the first atomic-level model corresponding to the current optimal total loss as the final atomic-level model and execute step S7. Otherwise, execute step S6-3. Step S6-3: Determine whether the first exit condition is met. If yes, proceed to step S6-4; otherwise, proceed to step S6-5. Step S6-4: Reduce the number of atoms, update the size, position and intensity distribution of each atom based on the optimization algorithm, as well as the focus distance and aberration corresponding to each scan image, and return to step S4; Step S6-5: Update the atomic parameters, as well as the focus distance and aberration corresponding to each scanned image, based on the optimization algorithm, and return to step S4.

3. The atomic-level electron tomography three-dimensional reconstruction method according to claim 2, characterized in that, The methods for reducing the number of atoms in step S6-4 include: Merge multiple atoms whose pairwise distances are all less than a threshold into one atom; Atoms with intensity distributions below a certain threshold are deleted.

4. The atomic-level electron tomography three-dimensional reconstruction method according to claim 1, characterized in that, The process in step S4 of generating a projection image corresponding to a single scan image based on the current atomic-level model and the observation direction, focus distance, and aberrations corresponding to that scan image includes: Step S4-1: Determine the field of view based on the observation direction, focusing distance, and aberrations of the scanned image, and filter out atoms outside the field of view; Step S4-2: Based on the accelerating voltage, convergence angle, aberration, defocusing amount, and electron source size of the transmission electron microscope, the two-dimensional matrix of the electron beam at different depths is calculated analytically. Step S4-3: Slice the current first atomic level model to obtain slice images corresponding to different depths; Step S4-4: Convolve each slice image with a two-dimensional matrix of the same depth to obtain convolution images corresponding to different depths, and then superimpose all the convolution images to obtain the projection image.

5. The atomic-level electron tomography three-dimensional reconstruction method according to claim 1, characterized in that, All atomic parameters have been normalized.

6. The atomic-level electron tomography three-dimensional reconstruction method according to claim 1, characterized in that, The total loss value is the sum of the loss values ​​of all projected images.

7. The atomic-level electron tomography three-dimensional reconstruction method according to claim 2, characterized in that, The optimization algorithm is a combination of adaptive momentum gradient optimization algorithm and particle swarm optimization algorithm.

8. The atomic-level electron tomography three-dimensional reconstruction method according to claim 1, characterized in that, The atoms are spherical in shape.

9. An atomic-level electron tomography three-dimensional reconstruction device, comprising a memory, a processor, and a program stored in the memory, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1-8.

10. A storage medium having a program stored thereon, characterized in that, When the program is executed, it implements the method as described in any one of claims 1-8.

Images