Image imaging method and device, storage medium and computer program product

Abstract

The invention relates to the technical field of electron microscope imaging, in particular to an image imaging method and device, a storage medium and a computer program product. The image imaging method comprises the following steps: acquiring multiple frames of continuous high-angle annular dark field images of a target sample without irradiation damage; performing filtering processing and two-dimensional matrix conversion on each frame of high-angle annular dark field image to obtain a two-dimensional matrix data set; based on each two-dimensional matrix in the two-dimensional matrix data set, determining a first pixel point meeting a preset condition in each two-dimensional matrix; obtaining a first preprocessed image corresponding to each two-dimensional matrix based on the first pixel point corresponding to each two-dimensional matrix; on the basis of the first preprocessed image corresponding to each two-dimensional matrix, multiple frames of aligned second preprocessed images are obtained; and obtaining a target high-angle annular dark field image based on the multiple frames of second pre-processed images. According to the invention, filtering, three-dimensional stack reconstruction, alignment processing and the like are carried out on the high-angle annular dark field image, and the resolution and definition of imaging are improved.

Description

Technical Field

[0001] This disclosure relates to the field of electron microscope imaging technology, and in particular to an image imaging method, apparatus, storage medium, and computer program product. Background Technology

[0002] With the rapid development of science and technology, aberration-corrected transmission electron microscopy enables researchers to directly observe the crystal structure, defects (such as dislocations and vacancies), interface structure, and fine morphology of nanoparticles of materials, providing valuable experimental evidence for the optimization of material performance and the development of new materials.

[0003] However, the sphere of aberration-corrected transmission electron microscopes releases high-energy electron beams during imaging. When these high-energy electron beams interact with electron beam-sensitive materials, they can cause radiation damage to the materials, thereby limiting the resolution and sharpness of the imaging.

[0004] Therefore, how to avoid radiation damage to electron beam-sensitive materials while improving the resolution and clarity of imaging is a technical problem that urgently needs to be solved. Summary of the Invention

[0005] This disclosure is made in view of the above-mentioned problems. This disclosure provides an image imaging method, apparatus, storage medium, and computer program product.

[0006] According to one aspect of this disclosure, an image imaging method is provided, comprising: acquiring multiple consecutive high-angle annular dark-field images of a target sample without radiation damage; performing filtering and two-dimensional matrix transformation on each frame of the high-angle annular dark-field image to obtain a two-dimensional matrix dataset; determining a first pixel point in each two-dimensional matrix that satisfies a preset condition based on each two-dimensional matrix in the two-dimensional matrix dataset; acquiring a first preprocessed image corresponding to each two-dimensional matrix based on the first pixel point corresponding to each two-dimensional matrix; acquiring multiple aligned second preprocessed images based on the first preprocessed images corresponding to each two-dimensional matrix; and acquiring a target high-angle annular dark-field image based on the multiple second preprocessed images.

[0007] Furthermore, according to one aspect of the image imaging method of this disclosure, acquiring multiple consecutive high-angle annular dark-field images of a target sample without radiation damage includes: acquiring multiple consecutive high-angle annular dark-field images of the target sample without radiation damage by reducing the electron beam exposure time and exposure dose.

[0008] Furthermore, according to one aspect of the image imaging method of this disclosure, each frame of high-angle annular dark field image is filtered and transformed into a two-dimensional matrix to obtain a two-dimensional matrix dataset, including: Each frame of high-angle annular dark field image is filtered using Gaussian filtering or Wiener filtering. Each frame of the high-angle annular dark field image after filtering is converted into a two-dimensional matrix; the two-dimensional matrices corresponding to each frame of the high-angle annular dark field image are combined into a two-dimensional matrix dataset.

[0009] Furthermore, according to one aspect of the image imaging method of this disclosure, the preset conditions include: the value of the first pixel is greater than a preset threshold.

[0010] Furthermore, according to one aspect of the image imaging method of this disclosure, a first preprocessed image corresponding to each two-dimensional matrix is ​​obtained based on the first pixel point corresponding to each two-dimensional matrix, including: determining a second pixel point within a preset range in each two-dimensional matrix based on the first pixel point corresponding to each two-dimensional matrix; and obtaining the first preprocessed image based on the second pixel point of each two-dimensional matrix.

[0011] Furthermore, according to one aspect of the image imaging method of this disclosure, based on a first preprocessed image corresponding to each two-dimensional matrix, an aligned multi-frame second preprocessed image is obtained, comprising: determining the correlation between each two adjacent frames of the first preprocessed image based on each frame of the first preprocessed image; performing image alignment on each frame of the first preprocessed image based on the correlation, and obtaining an aligned multi-frame second preprocessed image.

[0012] Furthermore, according to one aspect of the image imaging method of this disclosure, a target high-angle annular dark field image is obtained based on multiple frames of second preprocessed images, including: superimposing multiple frames of second preprocessed images to determine the region with the largest signal-to-noise ratio as the target high-angle annular dark field image.

[0013] According to another aspect of this disclosure, an image imaging apparatus is provided, comprising: a memory for storing computer-readable instructions; and a processor for executing the computer-readable instructions, causing the image imaging apparatus to perform the image imaging method as described above.

[0014] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the image imaging method as described above.

[0015] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the image imaging method as described above.

[0016] The image imaging method, apparatus, storage medium, and computer program product according to embodiments of the present disclosure will be described in detail below. The present disclosure improves the resolution and clarity of the imaging by acquiring multiple consecutive high-angle annular dark-field images of a target sample without radiation damage, and by performing filtering, alignment, and other processing on these high-angle annular dark-field images, making the target high-angle annular dark-field images more closely resemble the true state of the target sample. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic flowchart illustrating an image imaging method provided according to an embodiment of the present disclosure.

[0019] Figure 2 This is a further schematic diagram illustrating the process of an image imaging method provided according to an embodiment of the present disclosure.

[0020] Figure 3 This is a schematic diagram illustrating the amount of drift in the x and y directions between consecutive frames of images provided according to embodiments of the present disclosure.

[0021] Figure 4 This is a schematic diagram illustrating the structure of an image imaging apparatus according to an embodiment of the present disclosure.

[0022] Figure 5 This is a schematic diagram illustrating a non-transitory computer storage medium according to an embodiment of the present disclosure.

[0023] Figure 6 This is a schematic diagram illustrating a computer program product according to an embodiment of the present disclosure. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this disclosure clearer, the technical solutions of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0025] Figure 1 This is a schematic flowchart illustrating an image imaging method according to an embodiment of the present disclosure.

[0026] like Figure 1 As shown, the image imaging method according to an embodiment of this disclosure specifically includes the following steps.

[0027] Step S101: Acquire multiple consecutive high-angle annular dark field images of the target sample without irradiation damage.

[0028] In one embodiment of this disclosure, the target sample can refer to an electron beam sensitive material that undergoes significant physical or chemical changes under electron beam irradiation. The high-angle annular dark-field image is obtained through high-angle annular dark-field imaging. Since electron beam sensitive materials are extremely sensitive to electron beam irradiation, by concatenating multiple frames of radiation-free high-angle annular dark-field images of the target sample, the fine atomic-scale structure of the electron beam sensitive material can be revealed at the atomic scale, thereby understanding the structure-property relationship between its microstructure and properties.

[0029] Specifically, to obtain radiation-damage-free high-angle annular dark-field images, the irradiation time and dose can be reduced. Here, irradiation time refers to the length of time the electron beam irradiates the sample. By reducing the exposure time per frame (i.e., shortening the dwell time of the electron beam at each scan point on the sample), the total irradiation received by the sample can be reduced. Irradiation dose refers to the total energy of the electron beam received by the sample per unit area. Reducing the irradiation dose can be achieved by decreasing the beam current density of the electron beam (i.e., the number of electrons passing through the sample per unit time). A lower beam current density means that the energy carried by each electron is distributed more widely throughout the sample, thereby reducing energy deposition in localized areas and potential radiation damage.

[0030] In practical applications, radiation damage can be reduced by combining strategies that decrease irradiation time and dose. For example, the efficiency and quality of image acquisition can be improved by optimizing the scanning speed and beam current density of the electron beam.

[0031] Step S102: Filter and perform two-dimensional matrix transformation on each frame of high-angle circular dark field image to obtain a two-dimensional matrix dataset.

[0032] In one embodiment of this disclosure, the main purpose of filtering is to reduce or eliminate noise in the image and improve image quality. During the continuous acquisition of high-angle annular dark-field images, noise may be introduced due to various factors (such as electron beam instability, detector noise, and minute movements of the sample itself). By filtering multiple consecutive frames of high-angle annular dark-field images, the noise in the multiple frames of high-angle annular dark-field images caused by continuous acquisition can be reduced. Each frame of filtered high-angle annular dark-field image is converted into a two-dimensional matrix, and each of the multiple frames of filtered high-angle annular dark-field images corresponds to a two-dimensional matrix, thereby forming a two-dimensional matrix dataset.

[0033] Preferably, the two-dimensional matrices are combined into a two-dimensional matrix dataset according to the order of the corresponding image frames. This dataset not only contains all the filtered image information, but also maintains the order relationship between the image frames, which is beneficial for subsequent image processing.

[0034] Step S103: Based on each two-dimensional matrix in the two-dimensional matrix dataset, determine the first pixel point in each two-dimensional matrix that satisfies the preset conditions.

[0035] In one embodiment of this disclosure, preset conditions refer to a series of standards or rules set according to research or application needs before pixel selection. These conditions can be based on various attributes of pixels, such as coordinate position, grayscale value, color, texture features, shape features, numerical value, etc. The specific content of the preset conditions depends on the research purpose and data processing needs. By traversing each two-dimensional matrix, the first pixel in each two-dimensional matrix that meets the preset conditions can be determined, thereby determining the position of the first pixel to achieve more accurate and effective image processing and analysis.

[0036] Step S104: Based on the first pixel point corresponding to each two-dimensional matrix, obtain the first preprocessed image corresponding to each two-dimensional matrix.

[0037] In one embodiment of this disclosure, after determining the first pixel in each two-dimensional matrix, the first pixel contained in each two-dimensional matrix can be extracted to obtain a first preprocessed image. For example, a rectangular, circular, or other shaped region of a fixed size can be extracted centered on the first pixel in each two-dimensional matrix, and these extracted regions can be used as the first preprocessed image. Each two-dimensional matrix corresponds to one first preprocessed image.

[0038] Step S105: Based on the first preprocessed image corresponding to each two-dimensional matrix, obtain the aligned multi-frame second preprocessed image.

[0039] In one embodiment of this disclosure, the two-dimensional matrix dataset contains multiple two-dimensional matrices, each corresponding to a first preprocessed image. After aligning all the first preprocessed images, a multi-frame aligned second preprocessed image can be obtained. The purpose of alignment is to ensure that the content of the second preprocessed images between different frames is spatially consistent, thereby eliminating image offset caused by electron beam scanning position deviation and facilitating subsequent analysis.

[0040] Step S106: Based on multiple frames of the second preprocessed image, obtain the target high-angle annular dark field image.

[0041] In one embodiment of this disclosure, by utilizing aligned multi-frame second preprocessed images, the high-resolution and high-clarity portions of the images can be identified. Using these high-resolution and high-clarity portions as the target high-angle annular dark-field image more closely approximates the true state of the target sample. Furthermore, this target high-angle annular dark-field image can be used to observe and analyze information such as the microstructure, elemental distribution, and chemical composition of the target sample, providing strong support for scientific research and technological development.

[0042] In summary, according to the technical solution provided in the embodiments of this disclosure, this disclosure improves the resolution and clarity of imaging by acquiring multiple consecutive high-angle annular dark-field images of the target sample without radiation damage, and performing filtering, alignment, and other processing on the high-angle annular dark-field images, making the target high-angle annular dark-field images closer to the true state of the target sample.

[0043] Furthermore, multiple consecutive high-angle annular dark-field images of the target sample without radiation damage were obtained using an electron beam exposure time of 1-3 μs and an exposure dose of 1-5 pA.

[0044] Specifically, shorter exposure times (1-3 μs) help reduce radiation damage to the target sample from the electron beam. In electron beam imaging, prolonged exposure causes changes to the sample surface due to electron beam bombardment, such as atomic shifts and chemical bond breaking. These changes interfere with the observation of the sample's true state. By shortening the exposure time, this radiation damage can be minimized while maintaining image quality. Shorter exposure times also mean higher temporal resolution. This is particularly important for observing dynamic processes or rapidly changing phenomena. When it is necessary to capture changes in the target sample within a very short time, shorter exposure times provide more accurate temporal information. Precise control of the exposure dose (1-5 pA) is crucial for image quality. Lower exposure doses can reduce electron beam damage to the sample, but may also lead to a decrease in the image signal-to-noise ratio. By selecting an appropriate exposure dose, damage to the sample can be reduced while maintaining image quality. Furthermore, controlling the exposure dose also helps optimize image contrast and resolution.

[0045] Furthermore, the resolution of the acquired high-angle annular dark field image is 512 pixels * 512 pixels or 1024 pixels * 1024 pixels, and the number of frames acquired is 5-20.

[0046] Specifically, the resolution of 512 pixels * 512 pixels provides sufficient detail to observe and analyze the microstructure of the target sample.

[0047] Furthermore, the resolution is 1024 pixels * 1024 pixels, suitable for scenarios requiring higher precision and detailed analysis. Higher resolution means that each pixel represents a smaller actual size, thus revealing more detailed information, such as finer structural features and more accurate dimensional measurements.

[0048] Figure 2 This is a further schematic flowchart illustrating a graphics processing method according to an embodiment of the present disclosure.

[0049] like Figure 2 As shown, the image processing method according to the embodiments of this disclosure performs filtering and two-dimensional matrix transformation on each frame of high-angle annular dark field image to obtain a two-dimensional matrix dataset, specifically including the following steps.

[0050] Step S201: Filter each frame of high-angle annular dark field image based on Gaussian filtering or Wiener filtering.

[0051] In one embodiment of this disclosure, in order to reduce image noise, Gaussian filtering or Wiener filtering is used to filter each frame of high-angle annular dark field image, which can effectively reduce the noise of multiple frames of high-angle annular dark field images caused by continuous acquisition.

[0052] Step S202: Convert each frame of the filtered high-angle circular dark field image into a two-dimensional matrix.

[0053] In one embodiment of this disclosure, each frame of the filtered high-angle annular dark field image is converted into a two-dimensional matrix form to simplify image information and facilitate subsequent processing.

[0054] Step S203: The two-dimensional matrices corresponding to each frame of high-angle annular dark field image are combined into a two-dimensional matrix dataset.

[0055] In one embodiment of this disclosure, each frame of high-angle annular dark field image corresponds to a two-dimensional matrix, and the two-dimensional matrices corresponding to multiple frames of high-angle annular dark field images are combined to form a two-dimensional matrix dataset.

[0056] In summary, according to the technical solution provided in the embodiments of this disclosure, this disclosure can effectively reduce image noise, simplify image information, and improve imaging resolution and clarity by performing two-dimensional matrix transformation on multiple consecutive high-angle annular dark field images of the target sample without radiation damage through filtering processing.

[0057] Furthermore, the preset conditions include: the value of the first pixel is greater than a preset threshold.

[0058] Specifically, each element of the two-dimensional matrix corresponds to a pixel in the image. By traversing each element in the matrix, we check whether its value is greater than a preset threshold. This preset threshold is set in advance based on the data characteristics and analysis objectives. The pixel in the two-dimensional matrix whose value is greater than the preset threshold is called the first pixel.

[0059] Further, based on the first pixel point corresponding to each two-dimensional matrix, a first preprocessed image corresponding to each two-dimensional matrix is ​​obtained, including: Based on the first pixel point corresponding to each two-dimensional matrix, determine the second pixel point within a preset range in each two-dimensional matrix; The first preprocessed image is obtained based on the second pixel of each second matrix.

[0060] Specifically, using the first pixel point determined in each two-dimensional matrix as the center, a second pixel point is found within a preset range. This "preset range" can be a circular, square, or other shaped region centered on the first pixel point; the size of this region is determined based on the analysis requirements and data characteristics. For example, using each first pixel point in the two-dimensional matrix as the center of an atomic column, an elliptic fitting method with multiple contour lines is used to determine the radius of the atomic column in the high-angle annular dark-field image. The radius of the atomic column is the size of the preset range. Using the center position of this atomic column and its radius, the second pixel point within the preset range can be determined. The second pixel point includes all pixels within this range. For each two-dimensional matrix, the second pixel point is used to generate a new image frame, i.e., the first preprocessed image. Here, an "atomic column" refers to a row of atoms arranged along the electron beam direction (usually the thickness direction of the sample) in the target sample. Because high-angle annular dark-field imaging is sensitive to atomic number, heavier atoms will appear as brighter points or areas in the image. When the electron beam passes through the target sample, a visible bright spot or bright area is formed; this bright spot or bright area is called an "atomic column."

[0061] In summary, according to the technical solution provided in this disclosure, the first preprocessed image is generated using second pixels within a preset range, which essentially focuses on a local region of the image. This method reduces noise that may be introduced when processing the entire image, because noise is often randomly distributed throughout the image, while its impact is relatively small in local regions, thus significantly reducing the influence of noise on the analysis results. Furthermore, since the size and shape of the preset range can be flexibly adjusted according to analysis needs and data characteristics, the accuracy and reliability of the analysis results can be improved. Therefore, this technology can support a variety of different analysis needs.

[0062] Furthermore, the first preprocessed image described above can also be obtained using the following method.

[0063] Specifically, each frame of filtered high-angle annular dark field image is converted into a two-dimensional matrix. Each of the multi-frame filtered high-angle annular dark field images corresponds to a two-dimensional matrix. All two-dimensional matrices are then converted into a single three-dimensional matrix, forming a 3D stack file. By traversing the pixels corresponding to each two-dimensional matrix in the three-dimensional matrix, pixels with values ​​greater than a preset threshold are identified as the first pixel. Using the first pixel in each two-dimensional matrix as the midpoint, a second pixel is identified within a circular, square, or other shaped region centered on that first pixel. The range of this region is a preset range, which can be determined based on analysis requirements and data characteristics.

[0064] In summary, according to the technical solutions provided in the embodiments of this disclosure, a three-dimensional matrix is ​​richer in data structure than a two-dimensional matrix. A two-dimensional matrix only represents the pixel distribution of an image on a two-dimensional plane, while a three-dimensional matrix can add a new dimension, which can represent time (such as the stacking of image frames). This allows image data to be preserved and expanded in both time and space. By traversing the two-dimensional matrices within the three-dimensional matrix, the changes in image features over time or spatial location can be observed, thereby more accurately identifying and extracting the first pixel, and subsequently obtaining the second pixel.

[0065] Further, based on the first preprocessed image corresponding to each two-dimensional matrix, aligned multi-frame second preprocessed images are obtained, including: Based on the first preprocessed image of each frame, determine the correlation between each two adjacent first preprocessed images. Based on correlation, image alignment is performed on the first preprocessed image of each frame to obtain the aligned multi-frame second preprocessed image.

[0066] Specifically, determining the correlation involves analyzing the similarity or difference between any two adjacent frames of the first preprocessed images. Based on the correlation between these frames, transformation parameters (such as translation, rotation, scaling, etc.) that maximize their similarity are calculated. These transformation parameters are then applied to each pair of adjacent frames within the same coordinate system, thus aligning them. For multiple frames of the first preprocessed images, this process is iteratively applied, ensuring each frame is aligned with its predecessor, resulting in a series of aligned images—the aligned multi-frame second preprocessed images. These aligned images will be more efficient and accurate in subsequent processing or analysis. For example, the crosscorrelation function can be used to calculate the cross-correlation between any two adjacent frames of the first preprocessed images to obtain their correlation, resulting in a cross-correlation matrix. Alternatively, by sliding one image over another and calculating the similarity at each position, a cross-correlation matrix can be obtained. The position with the maximum value in the cross-correlation matrix typically corresponds to the optimal alignment of the two images. This position represents the distance (i.e., the translation vector) that one image needs to be translated relative to another. The translation vector contains components in both the horizontal and vertical directions, corresponding to the distance the image moves along the x-axis and y-axis, respectively. Once the translation vector is determined, a translation matrix can be created. This matrix describes how to move one image to a position aligned with the other image. The above steps are repeated for every two adjacent frames of the first preprocessed images in the multi-frame first preprocessed image series, aligning each frame with its preceding frame (i.e., consecutive frames). In this way, by aligning frame by frame, a temporally continuous and spatially aligned image sequence can be constructed, thus obtaining a multi-frame second preprocessed image series.

[0067] Furthermore, based on multiple frames of the third preprocessed image, a high-angle annular dark field image of the target is obtained, including: Multiple frames of the second preprocessed image are superimposed to determine the region with the highest signal-to-noise ratio as the target high-angle annular dark field image.

[0068] Specifically, multiple frames of the second preprocessed image are superimposed, and the pixel values ​​at the same location are accumulated or averaged, which helps reduce the impact of random noise. After superimposing multiple frames, a new image is obtained. The entire image region of this new image is analyzed. Based on the superposition result, the region with the highest signal-to-noise ratio is extracted from the superimposed image as the target's high-angle annular dark field image. This target's high-angle annular dark field image has a higher signal-to-noise ratio and clearer microstructural features, which is helpful for subsequent image analysis or scientific research.

[0069] In summary, according to the technical solutions provided in the embodiments of this disclosure, to achieve higher spatial resolution, mainstream aberration-corrected transmission electron microscopes typically operate at a working voltage of 300 kV. However, irradiating target sample materials with high-energy electron beams can easily cause radiation damage, thereby destroying the intrinsic structure of the material. To address this problem, this disclosure provides an image imaging method specifically for electron beam-sensitive materials. By reducing the electron beam exposure time and dose, and continuously acquiring multiple frames of images, followed by image denoising, filtering, centering correction, and overlay processing steps, a high-angle annular dark-field image at the atomic scale with a high signal-to-noise ratio is finally obtained. This method not only effectively avoids radiation damage caused by electron beams but also clearly reveals the intrinsic structure of electron beam-sensitive materials at the atomic scale. Furthermore, sufficiently clear images can be obtained at lower electron beam doses, thereby reducing the damage to samples caused by prolonged high-dose electron beam irradiation. This is particularly important for the study of electron beam-sensitive materials, helping to protect the integrity and stability of the samples.

[0070] To describe in detail the technical solutions provided by the embodiments of this disclosure, the image processing method of this disclosure is further described in the following embodiments 1-3.

[0071] Example 1: In this Example 1, the electron beam sensitive material (i.e., the aforementioned target sample) is a single-layer tungsten diselenide (WS2) material. The operating equipment is a JEM ARM300F dual-correction transmission electron microscope with a working voltage of 300 kV. The specific operating procedure is as follows: 1. To avoid electron beam irradiation damage, the electron beam irradiation dose and beam current were reduced to below 5 pA. The pixel size of the acquired multi-frame images was set to 1024 pixels * 1024 pixels, and the electron beam single pixel dwell time was set to 2 μs. At a magnification of 10 million, 6 high-angle annular dark-field images were continuously acquired (i.e., the aforementioned multi-frame continuous high-angle annular dark-field images of the target sample without irradiation damage).

[0072] 2. To reduce image noise, Gaussian filtering was used to filter the six high-angle annular dark field images, with the standard deviation sigma set to 2.

[0073] 3. Using the rgb2gray function in MATLAB, the six filtered high-angle annular dark field images were converted into two-dimensional matrix forms. The two-dimensional matrices were then normalized, with the maximum pixel value set to 1, forming a two-dimensional matrix dataset.

[0074] Preferably, the rgb2gray function in MATLAB is used to convert the six filtered high-angle annular dark field images into a two-dimensional matrix. The two-dimensional matrix is ​​normalized, with the maximum pixel value set to 1. An empty 3D array is created, with dimensions equal to the number of rows × columns × layers (frames) of the image. By iterating through all image layers, the image data (i.e., the two-dimensional matrix) of each layer is imported into the corresponding position in the 3D array. In this way, all the images are stacked together in chronological order, forming a 3D stack file.

[0075] 4. Mark the pixels in the two-dimensional matrix corresponding to each frame image whose values ​​are greater than the set threshold of 0.3 as the atomic pillar center (i.e., the first pixel); filter out all atomic pillar center positions in each two-dimensional matrix and save them as a separate two-dimensional matrix.

[0076] Preferably, pixels with values ​​greater than a set threshold of 0.3 in the two-dimensional matrix corresponding to each frame of the 3D stack file are marked as atomic pillar centers; all atomic pillar center positions in each two-dimensional matrix are filtered out and saved as a separate two-dimensional matrix.

[0077] 5. Based on the position of the center of the atomic column, the atoms in the image are fitted using an ellipse fitting method with multiple contour lines. The radius of the atomic column is set to 5 pixels. Values ​​greater than 0.5 in the two-dimensional matrix belong to the fitting range. An image (i.e., the aforementioned first preprocessed image) is determined based on the fitting range in each two-dimensional matrix. The above steps are repeated to obtain multiple frames of images (i.e., the aforementioned multiple frames of the first preprocessed image).

[0078] 6. Read two consecutive images (i.e., the first preprocessed image mentioned above); use the crosscorrelation function to calculate the cross-correlation between the consecutive images; find the position of the maximum value in the cross-correlation matrix and determine the translation vector; create a translation matrix to center the images; repeat the above operations to align all images (i.e., the first preprocessed image mentioned above).

[0079] Among them, see Figure 3 The amount of drift (i.e., translation vector) in the x and y directions between consecutive frames of images.

[0080] 7. Overlay all the high-angle annular dark field images after alignment (i.e., the aforementioned multi-frame second preprocessed images) to form a new image, and crop out the atomic-level high-angle annular dark field image with the best signal-to-noise ratio (i.e., the aforementioned target high-angle annular dark field image).

[0081] Example 2: In this Example 2, the electron beam sensitive material (i.e., the aforementioned target sample) is perovskite CsPbBr3 nanocrystalline quantum dots. The operating equipment is a JEM ARM300F dual-correction transmission electron microscope with a working voltage of 300 kV. The specific operating procedure is as follows: 1. To avoid electron beam irradiation damage, the electron beam irradiation dose and beam current were reduced to below 5 pA. The pixel size of the acquired multi-frame images was set to 1024 pixels * 1024 pixels, and the residence time of a single electron beam pixel was set to 1 us. At a magnification of 10 million, eight high-angle annular dark-field images (i.e., the aforementioned multi-frame continuous high-angle annular dark-field images of the target sample without irradiation damage) were acquired continuously.

[0082] 2. To reduce image noise, Gaussian filtering was used to filter the eight high-angle annular dark field images, with the standard deviation sigma set to 2.

[0083] 3. Using the rgb2gray function in MATLAB, the six filtered high-angle annular dark field images were converted into two-dimensional matrix forms. The two-dimensional matrices were then normalized, with the maximum pixel value set to 1, forming a two-dimensional matrix dataset.

[0084] Preferably, the rgb2gray function in MATLAB is used to convert the six filtered high-angle annular dark field images into a two-dimensional matrix. The two-dimensional matrix is ​​normalized, with the maximum pixel value set to 1. An empty 3D array is created, with dimensions equal to the number of rows × columns × layers (frames) of the image. By iterating through all image layers, the image data (i.e., the two-dimensional matrix) of each layer is imported into the corresponding position in the 3D array. In this way, all the images are stacked together in chronological order, forming a 3D stack file.

[0085] 4. Mark the pixels in the two-dimensional matrix corresponding to each frame image whose values ​​are greater than the set threshold of 0.2 as the center of the atomic pillar; filter out all the center positions of the atomic pillars in each two-dimensional matrix and save them as a separate two-dimensional matrix.

[0086] 5. Based on the center position of the atomic column, the atoms in the image are fitted using an ellipse fitting method with multiple contour lines. The atomic radius is set to 5 pixels. Values ​​greater than 0.6 in the two-dimensional matrix are included in the fitting range. An image (i.e., the aforementioned first preprocessed image) is determined based on the fitting range in each two-dimensional matrix. The above steps are repeated to obtain multiple frames of images (i.e., the aforementioned multiple frames of the first preprocessed image).

[0087] 6. Read two consecutive images (i.e., the second preprocessed image mentioned above); use the crosscorrelation function to calculate the cross-correlation between the consecutive images, find the position of the maximum value in the cross-correlation matrix, determine the translation vector, and create a translation matrix for image centering; repeat the above operations to align all images (i.e., the first preprocessed image mentioned above).

[0088] 7. Overlay all the high-angle annular dark field images after alignment (i.e., the aforementioned multi-frame second preprocessed images) to form a new image, and crop out the atomic-level high-angle annular dark field image with the best signal-to-noise ratio (i.e., the aforementioned target high-angle annular dark field image).

[0089] Example 3: In this embodiment 3, the electron beam sensitive material (i.e., the aforementioned target sample) is a three-layer IrO2 material epitaxially formed on rutile titanium dioxide. The operating equipment is a JEM ARM300F dual-correction transmission electron microscope with a working voltage of 300 kV. The specific operating procedure is as follows: 1. To avoid electron beam irradiation damage, the electron beam irradiation dose and beam current were reduced to below 5 pA. The pixel size of the acquired multi-frame images was set to 800 pixels * 800 pixels, and the residence time of a single electron beam pixel was set to 2 μs. At a magnification of 15 million, eight high-angle annular dark-field images (i.e., the aforementioned multi-frame continuous high-angle annular dark-field images of the target sample without irradiation damage) were acquired continuously.

[0090] 2. To reduce image noise, the Wiener filtering method was used to filter the eight high-angle annular dark field images.

[0091] 3. Using the rgb2gray function in MATLAB, the six filtered high-angle annular dark field images were converted into two-dimensional matrix forms. The two-dimensional matrices were then normalized, with the maximum pixel value set to 1, forming a two-dimensional matrix dataset.

[0092] Preferably, the rgb2gray function in MATLAB is used to convert the six filtered high-angle annular dark field images into a two-dimensional matrix. The two-dimensional matrix is ​​normalized, with the maximum pixel value set to 1. An empty 3D array is created, with dimensions equal to the number of rows × columns × layers (frames) of the image. By iterating through all image layers, the image data (i.e., the two-dimensional matrix) of each layer is imported into the corresponding position in the 3D array. In this way, all the images are stacked together in chronological order, forming a 3D stack file.

[0093] 4. Mark the pixels in the two-dimensional matrix corresponding to each frame image whose values ​​are greater than the set threshold of 0.3 as the atomic pillar center (i.e., the first pixel); filter out all atomic pillar center positions in each two-dimensional matrix and save them as a separate two-dimensional matrix.

[0094] Preferably, pixels with values ​​greater than a set threshold of 0.3 in the two-dimensional matrix corresponding to each frame of the 3D stack file are marked as atomic pillar centers; all atomic pillar center positions in each two-dimensional matrix are filtered out and saved as a separate two-dimensional matrix.

[0095] 5. Based on the position of the center of the atomic column, the atoms in the image are fitted using an ellipse fitting method with multiple contour lines. The atomic radius is set to 5 pixels. Values ​​greater than 0.5 in the two-dimensional matrix belong to the fitting range. An image (i.e., the aforementioned first preprocessed image) is determined based on the fitting range in each two-dimensional matrix. The above steps are repeated to obtain multiple frames of images (i.e., the aforementioned multiple frames of the first preprocessed image).

[0096] 6. Read two consecutive images (i.e. the first preprocessed image mentioned above), use the crosscorrelation function to calculate the cross-correlation between the consecutive images, find the position of the maximum value in the cross-correlation matrix, and determine the translation vector; create a translation matrix to center the images; repeat the above operations to align all images (i.e. the first preprocessed image mentioned above).

[0097] 7. Overlay all the high-angle annular dark field images after alignment (i.e., the aforementioned multi-frame second preprocessed images) to form a new image, and crop out the atomic-level high-angle annular dark field image with the best signal-to-noise ratio (i.e., the aforementioned target high-angle annular dark field image).

[0098] The image imaging apparatus provided in this disclosure is described below, and the image imaging apparatus described below can be referred to in correspondence with the image imaging method described above.

[0099] Figure 4 This is a functional block diagram illustrating an image imaging apparatus according to an embodiment of the present disclosure.

[0100] like Figure 4 As shown, the image imaging apparatus 400 according to an embodiment of the present disclosure includes a memory 401 and a processor 402.

[0101] Specifically, memory 401 is used to store computer-readable instructions; and

[0102] Specifically, the processor 402 is configured to execute the computer-readable instructions, causing the image imaging device to perform the image imaging method as described above.

[0103] Figure 5This is a schematic diagram illustrating a non-transitory computer-readable storage medium according to an embodiment of the present disclosure. Figure 5 As shown, a non-transitory computer-readable storage medium 500 according to an embodiment of the present disclosure stores computer-readable instructions 501 thereon. When the computer-readable instructions 501 are executed by a processor, the image imaging method according to an embodiment of the present disclosure described with reference to the above figures is performed. The computer-readable storage medium includes, but is not limited to, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, optical disk, magnetic disk, etc.

[0104] Figure 6 This is a schematic diagram illustrating a computer program product according to an embodiment of the present disclosure. Figure 6 As shown, a computer program product 600 according to an embodiment of this disclosure stores a computer program 601 thereon. When the computer program is executed by a processor, it implements the image processing method described above. The computer program product includes, but is not limited to, system software, application software, and games. System software is the basic software of a computer, responsible for managing the computer's hardware and applications, including operating systems, device drivers, etc. Application software is software designed to meet specific needs, such as office software, image processing software, etc. Games are software used for entertainment, providing various gaming experiences. In addition, the computer program product may also include embedded software, firmware, etc., for controlling and operating various hardware devices.

[0105] The image imaging method, apparatus, storage medium, and computer program product according to embodiments of the present disclosure have been described above with reference to the accompanying drawings. The present disclosure improves the resolution and clarity of the imaging by acquiring multiple consecutive high-angle annular dark-field images of a target sample without radiation damage, and then performing filtering, 3D stacking reconstruction, and alignment processing on these high-angle annular dark-field images, making the target high-angle annular dark-field images closer to the true state of the target sample. To achieve higher spatial resolution, current mainstream aberration-corrected transmission electron microscopes typically operate at a working voltage of 300 kV. However, high-energy electron beams easily cause radiation damage, thereby destroying the intrinsic structure of the material. To address this problem, the present disclosure provides an image imaging method specifically for electron beam-sensitive materials. By reducing the electron beam exposure time and dose, continuously acquiring multiple frames of images, and then performing image denoising, filtering, centering correction, and stacking processing steps, an atomic-scale high-angle annular dark-field image with a high signal-to-noise ratio is finally obtained. This method not only effectively avoids radiation damage caused by electron beams but also clearly reveals the intrinsic structure of electron beam-sensitive materials at the atomic scale. Furthermore, sufficiently clear images can be obtained at lower electron beam doses, thereby reducing the damage to samples caused by prolonged high-dose electron beam irradiation. This is particularly important for the study of electron beam-sensitive materials, helping to protect the integrity and stability of the samples.

[0106] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0107] The block diagrams of devices, apparatuses, devices, and systems disclosed herein are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0108] Additionally, as used herein, the "or" used in a list of items beginning with "at least one" indicates a separate list, such that a list of, for example, "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not imply that the described example is preferred or better than other examples.

[0109] It should also be noted that in the systems and methods of this disclosure, the components or steps can be decomposed and / or recombined. These decompositions and / or recombinations should be considered as equivalent solutions to this disclosure.

[0110] Various changes, substitutions, and modifications can be made to the technology described herein without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of this disclosure is not limited to the specific aspects of the processes, machines, manufactures, events, means, methods, and actions described above. Currently existing or later-developed processes, machines, manufactures, events, means, methods, or actions that perform substantially the same function or achieve substantially the same result as the corresponding aspects described herein can be utilized. Therefore, the appended claims include such processes, machines, manufactures, events, means, methods, or actions within their scope.

[0111] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0112] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations therein.

Claims

1. An image imaging method, characterized in that, include: Acquire multiple consecutive high-angle annular dark-field images of the target sample without radiation damage; Each frame of the high-angle annular dark field image is filtered and transformed into a two-dimensional matrix to obtain a two-dimensional matrix dataset. Based on each two-dimensional matrix in the two-dimensional matrix dataset, determine the first pixel point in each two-dimensional matrix that satisfies the preset conditions; Based on the first pixel point corresponding to each of the two-dimensional matrices, obtain the first preprocessed image corresponding to each of the two-dimensional matrices; Based on the first preprocessed image corresponding to each of the two-dimensional matrices, obtain aligned multi-frame second preprocessed images; Based on the multiple frames of the second preprocessed image, a target high-angle annular dark field image is obtained.

2. The image imaging method as described in claim 1, characterized in that, The acquisition of multiple consecutive high-angle annular dark-field images of the target sample without radiation damage includes: By reducing the electron beam exposure time and exposure dose, multiple consecutive high-angle annular dark field images of the target sample without radiation damage were obtained.

3. The image imaging method as described in claim 1, characterized in that, The step of filtering and performing two-dimensional matrix transformation on each frame of the high-angle annular dark field image to obtain a two-dimensional matrix dataset includes: Each frame of the high-angle annular dark field image is filtered using either Gaussian filtering or Wiener filtering. Each frame of the high-angle annular dark field image after filtering is converted into a two-dimensional matrix; The two-dimensional matrix corresponding to each frame of the high-angle annular dark field image is combined to form the two-dimensional matrix dataset.

4. The image imaging method as described in claim 1, characterized in that, The preset conditions include: The value of the first pixel is greater than a preset threshold.

5. The image imaging method as described in claim 1, characterized in that, The step of obtaining a first preprocessed image corresponding to each two-dimensional matrix based on the first pixel point corresponding to each two-dimensional matrix includes: Based on the first pixel point corresponding to each of the two-dimensional matrices, determine the second pixel point within a preset range in each of the two-dimensional matrices; The first preprocessed image is obtained based on the second pixel point of each of the second matrices.

6. The image imaging method as described in claim 1, characterized in that, The step of obtaining aligned multi-frame second preprocessed images based on the first preprocessed image corresponding to each of the two-dimensional matrices includes: Based on each frame of the first preprocessed image, determine the correlation between each two adjacent frames of the first preprocessed image. Based on the correlation, image alignment is performed on the first preprocessed image of each frame to obtain the aligned second preprocessed image of the multiple frames.

7. The image imaging method as described in claim 1, characterized in that, The step of obtaining the target high-angle annular dark field image based on the multiple frames of the second preprocessed image includes: The multiple frames of the second preprocessed images are superimposed to determine the region with the highest signal-to-noise ratio as the target high-angle annular dark field image.

8. An image imaging device, characterized in that, include: Memory, used to store computer-readable instructions; as well as A processor for executing the computer-readable instructions, causing the image imaging apparatus to perform the image imaging method as described in any one of claims 1-7.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the image imaging method as described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the image imaging method as described in any one of claims 1 to 7.

Images