Data acquisition and processing techniques for three-dimensional reconstruction

Abstract

Disclosed herein are apparatuses and processes for generating data for three-dimensional reconstruction. An example method includes at least: exposing a subsequent surface of a sample; acquiring an image of the subsequent surface; comparing the image of the subsequent surface to an image of a reference surface; based on the comparison exceeding a threshold, acquiring a composition or crystal map of the subsequent surface; and based on the comparison not exceeding the threshold, exposing a next surface.

Description

Technical Field

[0001] This invention generally relates to charged particle microscopy, and more specifically to sample preparation for three-dimensional reconstruction in charged particle microscopy. Background Technology

[0002] Charged particle microscopy can be used for a range of imaging and analytical techniques. One simple technique, known as slicing and observation, involves data collection on sequences of the sample's surface, where layers of the sample are removed after data collection on the exposed surface. This technique is widely used, for example, in materials research and biological sample analysis, and can provide a reconstructed volume of the sample after some image processing. While this technique may not be very time-consuming if only images are acquired, acquiring other types of data, such as compositional or crystal data, increases the time required to obtain the desired data. For example, if processing a sample volume comprising hundreds of layers, and the desired data includes both images and elemental information, the processing time can increase dramatically to days or weeks.

[0003] While existing attempts to improve throughput exist, they all have their problems. One existing solution involves acquiring elemental information, such as image each surface while simultaneously acquiring elemental information at every n surfaces. This obviously improves processing time, but the elemental data is finite and cannot capture variations in the sample. Clearly, the choice of n can affect the coverage of the elemental data, but at the cost of processing time, i.e., throughput. Although there may actually be other possible solutions, it is still desirable to provide a complete dataset of images and compositional and / or crystal information with fast processing time. Summary of the Invention

[0004] This document discloses apparatus and processes for generating data for 3D reconstruction. An exemplary method includes at least: exposing a subsequent surface of a sample; acquiring an image of the subsequent surface; comparing the image of the subsequent surface with an image of a reference surface; acquiring a compositional or crystalline map of the subsequent surface based on a comparison exceeding a threshold; and exposing the next surface based on a comparison not exceeding the threshold. An exemplary device includes at least: an electron column coupled to provide an electron beam, a focused ion beam column coupled to provide a focused ion beam, a plurality of detectors arranged to detect at least secondary electrons, X-rays, and backscattered electrons, and a controller coupled to at least the electron column, the focused ion beam column, and each of the plurality of detectors. The controller includes or is coupled to code that, when executed by the controller, causes the device to: expose a subsequent surface of the sample by the focused ion beam; acquire an image of the subsequent surface; compare the image of the subsequent surface with a reference image, the reference image being an image of a reference surface; acquire a compositional or crystallographic map of the subsequent surface based on a comparison exceeding a threshold; and expose the next surface based on a comparison not exceeding a threshold. Attached Figure Description

[0005] Figure 1 This is an exemplary dual-beam system for generating 3D reconstructions of multiple datasets according to embodiments of this disclosure; Figure 2 This is an exemplary sample processing workflow according to embodiments of the present disclosure; Figure 3 This is an exemplary method for acquiring multimodal data while processing a sample, according to embodiments of the present disclosure; Figure 4 These are illustrative image sequences based on embodiments disclosed herein; Figure 5 This is an exemplary workflow for propagating crystal and / or compositional maps to an unmapped surface according to embodiments of this disclosure; Figure 6 This is an illustrative image sequence according to embodiments of the present disclosure; and Figure 7 This is an example functional block diagram according to an embodiment of the present disclosure.

[0006] Throughout the various views in the accompanying drawings, similar reference numerals refer to the corresponding parts. Detailed Implementation

[0007] Embodiments of the present invention are described below in the context of using multiple data acquisition modalities in charged particle microscopy to generate three-dimensional (3D) reconstructions of samples, so that the 3D reconstructions include multiple data types. For example, multiple surfaces of a sample can be imaged using slicing and observation techniques, wherein each image is acquired after material layers are removed, and further includes compositional and / or crystal maps of periodically acquired subsets of surfaces. While not every surface can be mapped to compositional or crystal properties, a neural network is used to determine which surfaces to map based on changes in the image to a reference threshold. Once the images and maps are acquired, or concurrently with the acquisition of images and maps, a second neural network is used to propagate the maps to surfaces that do not have associated maps using the pixel values ​​of the images to map the corresponding number of compositional or crystal maps. Once all surfaces have associated compositional and / or crystal maps, 3D reconstructions of all data types are available. However, it should be understood that the methods described herein are generally applicable to a wide variety of different tomographic imaging methods and devices, including cone-beam and parallel-beam systems, and are not limited to any particular device type, beam type, object type, length scale, or scan trajectory.

[0008] As used in this application and claims, unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “the” include the plural forms. Furthermore, the term “comprising” means “including.” Further, the term “joint” does not exclude the presence of intermediate elements between joined items.

[0009] The systems, apparatuses, and methods described herein should not be construed as limiting in any way. In fact, this disclosure is directed to all novel and non-obvious features and aspects of the various disclosed embodiments, whether individually or in various combinations and sub-combinations formed with each other. The disclosed systems, methods, and apparatuses are not limited to any particular aspect or feature or combination thereof, nor are they required to provide any one or more specific advantages or solve any one or more specific problems. Any operational theory is provided for ease of interpretation, but the disclosed systems, methods, and apparatuses are not limited to such operational theories.

[0010] Although some operations of the disclosed methods are described in a specific order for ease of presentation, it should be understood that this descriptive approach encompasses rearrangement unless the specific language used below requires a particular order. For example, in some cases, the operations described in sequence may be rearranged or performed simultaneously. Furthermore, for simplicity, the accompanying drawings may not show the various ways in which the disclosed systems, methods, and apparatus can be used in conjunction with other systems, methods, and apparatuses. Additionally, this specification sometimes uses terms such as “produce” and “provide” to describe the disclosed methods. These terms are high-level abstractions of the actual operations performed. The actual operations corresponding to these terms will vary depending on the specific embodiment and are readily discernible to those skilled in the art.

[0011] In some instances, values, programs, or devices are referred to as “lowest,” “best,” “smallest,” etc. It will be recognized that such descriptions are intended to indicate that there are many functional alternatives available for use, and that such choices are not necessarily better, smaller, or preferred over others.

[0012] Conventional techniques for generating 3D reconstructed data for samples can be time-consuming, especially if multiple data types are expected for a full-volume sample. For example, if slicing and observation methods are employed to acquire the desired data, the acquisition time can be quite long, ranging from days to weeks, depending on the sample volume. Of course, if only images of each surface are desired, then processing time may not be an issue. However, if multiple types of data are expected for each exposed surface, the acquisition time increases dramatically beyond acceptable limits. Besides images, other types of data can include compositional data and / or crystallographic data. For example, compositional maps can be formed using energy-dispersive X-ray spectroscopy (EDS), wavelength-dispersive spectroscopy (WDS), or cathodoluminescence (CL), to name a few, while crystallographic maps can be formed using electron backscatter diffraction (EBSD). While acquiring multiple data types for each surface is possible, the processing time becomes excessive. Alternatively, periodic acquisitions of slower data types, such as composition or crystallography, can be performed at predetermined intervals while all surfaces are imaged. While this provides some slow data, such execution is agnostic to the sample or its variations, and information may be missed. Therefore, it is still desirable to obtain multiple data points on each surface in order to produce multi-data 3D reconstructions.

[0013] One solution could be to monitor images of each exposed surface and determine if there is a significant change in the amount of variation in the sample relative to a reference surface. If the change is considered significant, a mapping (composition and / or crystals) of the newly exposed surface is acquired along with the image. However, if the change is not significant, a new surface is exposed and an image is acquired. This determination is then repeated. In this way, reference surfaces with associated reference datasets are acquired based on the variations in the sample. Once the sample has been fully processed, the reference surface dataset is used to generate mappings for other surfaces for which only images have been acquired. This interpolation can be performed using one or more neural networks that use the reference dataset and images of other surfaces to generate interpolated mappings. Once all surfaces have associated images and mappings, both acquired and interpolated, a multi-dataset 3D reconstruction can be formed.

[0014] To further illustrate, a slicing and observation technique is used to process the sample. This technique involves imaging and mapping a reference surface of the sample, removing layers of the sample to expose new surfaces, and then imaging the new surfaces. After imaging the new surfaces, a neural network compares the image of the new surfaces with the image of the reference surfaces to determine if there are significant changes in the sample, such as the appearance of new features, or the expansion or contraction of features, to name a few. If the comparison indicates a significant change, a map of the new surfaces is then generated, and this new surface becomes the current reference surface. However, if the comparison does not indicate a significant change, the new layers are removed, and the newly exposed surfaces are imaged. The current reference surface remains the reference surface (rather than being replaced), and a comparison is performed between the image of the newly exposed layers and the reference image. Based on this comparison, a map of the newly exposed surfaces is then generated, or a subsequent surface is exposed. This process is repeated for samples of desired volumes, resulting in several surfaces having both an associated image and an associated map, but the remaining surfaces having only an associated image. Again, whether a surface has both an image and a map, or only an image, depends on a comparison of the surface image with the most recent reference image, e.g., whether a significant change has occurred.

[0015] Once the volume has been processed, one or more additional neural networks can be used to generate maps of surfaces that do not have associated maps, where the generated maps are based on interpolation from surrounding reference maps and their corresponding images. For example, the neural network can use two neighboring reference surface images and maps to associate the pixel values ​​of the images with the pixel values ​​of the maps. This association of pixel values ​​can then be used to generate maps based on surfaces whose pixel values ​​only have associated images. Once all surfaces have both images and maps, a 3D reconstruction of a sample of images and maps can be formed.

[0016] By using neural networks to determine when to acquire maps of newly exposed surfaces and to generate maps of surfaces that do not have associated maps, the throughput of such slicing and observation processes is improved, while providing a complete and robust dataset.

[0017] Alternatively, in addition to or replacing decision-based map acquisition methods, the process can also acquire sparse crystal and / or composition maps at each layer. A sparse map can be a map with all pixels having a fast acquisition time of less than that of data and / or data that results in noise. Then, once the sample volume has been fully processed, the layer with the sparse map can be processed by a neural network to produce the complete map. The sparse map process can not increase processing time because the sparse map can be acquired simultaneously with the image acquisition.

[0018] Figure 1 This is an exemplary dual-beam system 100 for generating 3D reconstructions of multiple datasets according to embodiments of the present disclosure. System 100 can be used to implement the data collection techniques discussed herein. In some embodiments, system 100 will perform data acquisition and composition and / or crystal map interpolation to provide 3D reconstructions. However, in other embodiments, system 100 will acquire data, and crystal map interpolation and 3D reconstruction will be performed by a separate computing system, such as on a user's desktop or at a cloud computing-based system. Although examples of suitable hardware are provided below, the invention is not limited to implementation in any particular type of hardware.

[0019] SEM 141, along with power supply and control unit 145, is provided to dual-beam system 100. An electron beam 143 is emitted from cathode 152 by applying a voltage between cathode 152 and anode 154. The electron beam 143 is focused into a fine point by converging lens 156 and objective lens 158. The electron beam 143 is then used to perform a two-dimensional scan on the sample by deflector 160. The operation of converging lens 156, objective lens 158, and deflector 160 is controlled by power supply and control unit 145.

[0020] An electron beam 143 can be focused onto a substrate 122 located on a platform 125 within a lower chamber 126. The substrate 122 can be positioned on a surface of the platform 125 or on a TEM sample holder 124 extending from the surface of the platform 125. When electrons in the electron beam strike the substrate 122, secondary electrons are emitted. These secondary electrons are detected by a detector 140. In some embodiments, a STEM detector 162 positioned below the TEM sample holder 124 and the platform 125 collects electrons transmitted through the sample mounted on the TEM sample holder.

[0021] System 100 also includes a FIB system 111 comprising a vacuum chamber with an ion column 112, an ion source 114 and a focusing assembly 116 positioned within the ion column, the focusing assembly including an extractor electrode and an electrostatic optics system. The axis of the focusing column 116 can be tilted, for example, by 52 degrees from the axis of the electron column 141. The ion column 112 includes an ion source 114, an extraction electrode 115, a focusing element 117, and a deflection element 120, which operate uniformly to form a focused ion beam 118. The focused ion beam 118 originates from the ion source 114, passes through the focusing assembly 116, and faces a substrate 122 between electrostatic deflection devices schematically indicated at 120. The substrate may contain, for example, a semiconductor wafer positioned, on a movable platform 125 within a lower chamber 126. In some embodiments, a sample may be positioned on a TEM grid support 124, wherein the sample may be a bulk extracted from the substrate 122. The bulk is then subjected to further processing using the FIB according to the techniques disclosed herein.

[0022] Platform 125 is movable in the horizontal plane (X and Y axes) and the vertical plane (Z axis). Platform 125 is also tiltable and rotatable about the Z axis. In some embodiments, a separate TEM stage 124 may be used. This TEM stage is also preferably movable in the X, Y, and Z axes, and tiltable and rotatable. In some embodiments, the tilting of platform 125 / TEM support 124 can be in-plane and out-of-plane of the ion beam 118, and the rotation of the platform about the ion beam 118.

[0023] Door 161 is opened to insert substrate 122 onto platform 125. Depending on the tilt of platforms 124 / 125, the Z-axis will be in the direction of the optical axis of the relevant column. For example, during the data collection phase of the disclosed technology, the Z-axis will be in a direction parallel to, for example, the FIB optical axis indicated by ion beam 118. In such a coordinate system, the X-axis and Y-axis will be referenced to the Z-axis. For example, the X-axis can be displayed... Figure 1 The Y-axis will be inside the page, while the Y-axis will be inside the page, and all three axes will remain perpendicular to each other.

[0024] An ion pump 168 is used to evacuate the neck 112. Under the control of a vacuum controller 132, the chamber 126 is evacuated using a turbine molecular and mechanical suction system 130. The vacuum system provides approximately 1 × 10⁻⁶ vacuum within the chamber 126. －7 To and 5×10 －4 The vacuum between the chambers. If etching aids, etching restraint gases, or deposition precursor gases are used, the chamber floor pressure may increase, typically to about 1 × 10⁻⁶. －5 Entrust.

[0025] A high-voltage power supply provides an appropriate accelerating voltage to the electrodes in the focusing column 116 to excite and focus the ion beam 118. When it strikes the substrate 122, material is sputtered, i.e., physically ejected from the sample. Alternatively, the ion beam 118 can decompose a precursor gas to deposit material.

[0026] A high-voltage power supply 134 is connected to the appropriate electrodes in the ion source 114 and the ion beam focusing assembly 116 to form and guide an ion beam 118 of approximately 1 keV to 60 keV to the sample. A deflection controller and amplifier 136, operating according to a predetermined pattern provided by the pattern generator 138, are coupled to a deflection plate 120, thereby allowing manual or automatic control of the ion beam 118 to trace a corresponding pattern on the upper surface of the substrate 122. In some systems, the deflection plate is positioned before the final lens, as is known in the art. When a blanking controller (not shown) applies a blanking voltage to the blanking electrode, the beam blanking electrode (not shown) within the ion beam focusing column 116 causes the ion beam 118 to strike the blanking aperture (not shown) instead of the substrate 122.

[0027] Ion source 114 typically provides an ion beam based on its type. In some embodiments, ion source 114 is, for example, a liquid metal ion source that can provide a gallium ion beam. In other embodiments, ion source 114 may be a plasma-type ion source that can deliver a variety of different ion species, such as oxygen, xenon, and nitrogen, to name just a few. Ion source 114 is typically capable of focusing a beam into a sub-1 / 10-micron-wide beam at substrate 122 or TEM grid support 124 for modifying substrate 122 by ion milling, ion-induced etching, material deposition, or for imaging substrate 122.

[0028] Detectors 140 may be included for detecting various types of emissions from the sample during processing and / or imaging. Such data may include secondary electrons, backscattered electrons, X-rays, and / or photons. For secondary electron detection, a detector such as an Everhart Thornley or multichannel plate is connected to video circuitry 142, which supplies drive signals to a video monitor 144 and receives deflection signals from a system controller 119. For backscattered electron detection, EBSD detectors known in the art can be used. X-rays can be detected by EDX or WDS detectors, and photons can be detected using any known optical detector tuned for a variety of wavelength ranges. In different embodiments, the positions of the various detectors 140 within the lower chamber 126 may vary.

[0029] The micromanipulator 147 can precisely move an object within a vacuum chamber. The micromanipulator 147 may include a precision electric motor 148 positioned outside the vacuum chamber to provide X, Y, Z, and θ control of a portion 149 positioned within the vacuum chamber. The micromanipulator 147 may be equipped with different end effectors for manipulating small objects. In the embodiments described herein, the end effector is a fine probe 150.

[0030] A gas delivery system 146 extends into the lower chamber 126 for introducing and guiding gaseous vapors to the substrate 122. For example, iodine may be delivered to enhance etching, or a metal-organic compound may be delivered to deposit metal.

[0031] System controller 119 controls the operation of various parts of system 100. Through system controller 119, a user can scan the ion beam 118 or electron beam 143 in a desired manner by inputting commands into a conventional user interface (not shown). Alternatively, system controller 119 can control system 100 according to programming instructions stored in memory 121. Memory 121 may also include code for one or more neural networks to implement the techniques disclosed herein. In some embodiments, system 100 includes one or more neural networks in memory 121 or is coupled to one or more neural networks via network 170 to assist in decision making and map interpolation / accumulation, both of which will be discussed in more detail below. Controller 119 may also be coupled via network 170 to neural network 172, which may be stored on one or more servers. For example, network 170 may be an intranet, the Internet, a local area network, or a wide area network.

[0032] In operation according to the techniques disclosed herein, system 100 performs a slicing and observation process on sample 122. The basic slicing and observation process includes imaging the surface of the sample using SEM 141 or FIB 111, where secondary electrons emitted from the sample and / or backscattered electrons form an image, followed by removing layers from the sample using FIB 111 to expose new surfaces. The new surfaces are then imaged, and this process is repeated until the desired sample volume is imaged. In addition to the basic slicing and observation process, one or more surfaces may also have compositional and / or crystallographic maps formed on them before or after surface image capture. For example, one or more surfaces may have EDS and / or EBSD maps obtained on them, based on a neural network's determination of whether the sample is changing in terms of structure, composition, or morphology.

[0033] To further illustrate, the surface of the sample is imaged using SEM 141, and EDS and / or EBSD maps of the same sample are obtained. Since the surface has both an associated image and an associated map, it can be referred to as a reference surface with associated reference data. After imaging and mapping, the sample layers are removed using FIB 111 to grind away layers or slices of the sample. The removal of material results in the exposure of a new surface. This new surface is then imaged, for example, using SEM 141. A neural network then compares the image of the new surface with the image of the reference surface to determine if there are significant changes in the visible features in the image of the new surface. In some embodiments, significant changes are based on a threshold criterion of the difference between the two images, where the threshold can be 0.5 in the range of 0 to 1.0. For example, based on the magnitude of the new feature / morphology, the appearance of new structures or different morphologies in the new image can be assigned a number greater than 0.5, and the absence of such changes can be assigned a number less than 0.5. Based on this comparison, the neural network determines what steps system 100 should perform next.

[0034] If the comparison shows no significant change (e.g., the difference between the two images is less than 0.5), then system 100 again uses FIB 111 to remove another layer from the sample and images the newly exposed surface. It should be noted that other forms of layer removal can also be used, such as slicing the layer with a slicing machine, laser ablation of the layer, or wide ion beam slicing. All such layer removal techniques are considered in this paper.

[0035] However, if comparisons show a significant change (e.g., a difference greater than or equal to 0.5 between two images), then an EDS and / or EBSD map of the newly exposed surface is obtained. This newly exposed surface then becomes the current reference surface. Once the newly exposed surface has been imaged and mapped, the new layer is removed, and the process is repeated. This sequence of imaging, decision-making, layer removal, mapping, etc., is repeated for samples of the desired volume, thus obtaining various data for the sample volume. It should be noted that the various data will include a subset of surfaces with both associated images and maps, while the remaining surfaces will only have associated images. In this case, 3D reconstruction using only the images is possible, but significant discrepancies may exist in the corresponding 3D reconstructed map data.

[0036] The lack of maps for each surface is addressed by using one or more additional neural networks to generate maps of surfaces that do not already have maps. For example, EDS / EBSD maps can be generated for surfaces without maps using neighboring reference surfaces and their associated reference datasets. For illustration, one or more neural networks can associate map pixel values ​​with image pixel values ​​based on the reference dataset, and then use this association to generate maps for surfaces without maps. When processing the entire volume of data, maps for each surface are obtained, some of which have associated acquired maps, and others have associated interpolated maps. Once each surface has a map, the corresponding 3D reconstruction can be obtained.

[0037] Alternatively or additionally, sparse EDS and / or EBSD maps of each non-reference surface can be acquired simultaneously with image acquisition. This sparse data can then be filled in when the map data is backpropagated to the non-reference surfaces using the reference map.

[0038] Figure 2 This is an example sample processing workflow 201 according to embodiments of the present disclosure. Workflow 201 illustrates sample grinding, imaging, and data collection techniques, which may also be referred to as slicing and observation, as discussed above. In the disclosed techniques, slicing and observation are used to obtain data on the sample to construct a 3D reconstruction including multiple data types. Workflow 201 can be performed by a dual-beam charged particle microscope (such as, for example, system 100) and is more specifically geared towards the data collection aspects of the disclosed techniques. The 3D reconstruction and data propagation aspects will be discussed in more detail below.

[0039] Workflow 201 is performed on sample 222, an instance of sample 122, which may be mounted on a platform such as platform 125. Workflow 201 uses an ion beam 218 to remove material from sample 222, and then images the newly exposed surface 272 with an electron beam 243. A new exposed surface 272 is formed after each layer or slice of sample 222 is removed. For example, the ion beam 218 removes slice 274A that exposes the new surface 272. Removal of slice 274A and subsequent slices can be performed by milling the slices using the ion beam 218, or it can be removed using ion beam-induced etching by venting a gas precursor to surface 272 and then etching the slices away through interaction with the ion beam 218. The thickness of each slice can be on the order of 2 to 5 nanometers, but the size of the structure within sample 222 can determine the required slice thickness. For example, smaller structures may require thinner slices, while larger structures can tolerate thicker slices. As the workflow progresses, slice 274B is removed, and then an image of the newly exposed surface 272 is acquired. This two-step process can then be repeated to remove slices 247C and 274D.

[0040] In addition to electron beam imaging, workflow 201 may further include obtaining compositional and / or crystal maps of some surfaces based on the determinations discussed above. Maps can be acquired in response to electron beam illumination of the surface, and for EBSD maps, backscattered electrons can be measured, or for EDS maps, the energy of emitted X-rays can be measured. While not every surface can have an associated map, workflow 201 outlines a general process for obtaining images and maps of surfaces 272 of sample 222. Alternative or additional methods may involve acquiring sparse compositional and / or crystal maps of each non-reference surface, where such sparse data can be supplemented and combined with backpropagation from the reference map.

[0041] Figure 3 This is an exemplary method 301 for acquiring multimodal data while processing a sample, according to embodiments of the present disclosure. Method 301 can be implemented in any type of microscopy environment capable of acquiring multiple types of data from the sample. An exemplary microscope includes a dual-beam system 200. Regardless of the microscope being implemented, method 301 acquires at least two types of data from a subset of surfaces and only one type of data from the remaining surfaces. Method 301 outlines embodiments of how to determine when to acquire a second type of data when processing a sample using a sectioning and observation-type workflow. Implementation of such techniques results in higher quality and more efficient throughput of 3D data from the sample than currently available. Of course, all types of data can be acquired for each surface, but such brute-force techniques fail to benefit from the disclosed technique.

[0042] Method 301 may begin at process block 303, which includes acquiring a compositional and / or crystallographic map of a reference surface of the sample. The acquisition of the compositional and / or crystallographic map can be performed using various data acquisition techniques as discussed above, and includes at least, for example, EDS, WDS, EBSD, and / or CL, but other compositional or crystallographic techniques are also considered herein and are within the scope of this disclosure. Generally, the technique used to acquire the map may be referred to as a second mode, which differs from the first mode used in the image acquisition process. The acquired map provides a pixelated map of the sample relating to the composition of the sample at each pixel and / or the crystal orientation of the sample at each pixel. Once the additional surfaces are exposed and imaged, such data can be used to construct the associated 3D volume of the sample.

[0043] Process block 303 can be followed by process block 305, which includes acquiring an image of the reference surface. The image can be acquired using a first mode, which is typically faster than the second mode. The first mode can be an SEM image or a FIB image of the surface, providing more information about the surface morphology. This data is also used to generate the 3D volume of the sample. It should be noted that the order of process blocks 303 and 305 can be interchanged, and is neither required nor restricted. Figure 4 The order shown. It should also be noted that once the initial reference surface is obtained, process block 305 may not be executed after decision block 313, because the image has already been obtained in process block 309.

[0044] Process block 305 can be followed by process block 307, which includes exposing the subsequent surface of the sample. Exposing the subsequent surface may include removing a material layer from the sample using any available technique. For example, FIB can be used to grind or etch away the layer to expose the subsequent surface. Alternatively, a laser can be used to ablate the material layer, or a slicer can be used to cut the material layer. Regardless of the material removal technique used, the subsequent surface is exposed so that at least an image of the subsequent surface can be obtained.

[0045] Process block 307 may be followed by process block 309, which includes acquiring an image of the subsequent surface. As discussed above, the image is acquired using a first mode. The first mode can be used to image secondary electrons and / or backscattered electrons emitted from the surface in response to illumination by an electron beam (e.g., SEM-based imaging) or a FIB beam (e.g., FIB-based imaging). Regardless of the first mode, a grayscale image of the surface is acquired, where the grayscale differences between pixels indicate morphological and / or material differences.

[0046] It should be noted that in some instances of method 301, a sparse map of the composition and / or crystal data can be acquired simultaneously with image acquisition steps 305 and 309. The sparse map can have data about a small number of localized regions on the sample surface. For example, when an image of the surface is acquired using a pixel-by-pixel strategy (e.g., by scanning an electron beam across the surface in a desired manner so that the beam stops at pixels on the surface, i.e., identifiable locations), EDS and / or EBSD data can be acquired at a subset of pixels to obtain the sparse map. This sparse data can then be augmented, for example, in data propagation method 501.

[0047] Process block 309 can be followed by process block 311, which includes comparing an image of the subsequent surface with an image of the reference surface. This comparison can be performed by one or more neural networks, such as Siamese networks, and can be performed on a pixel-by-pixel basis. Generally, the comparison aims to determine whether there are significant changes in the visible structure on the surface that has not been covered by layers, which can be indicated in the image. Accordingly, significant differences in the images are determined by comparison, where significant differences are based on a threshold. For example, the comparison can determine differences between images where differences above a threshold, such as in the range of 0 to 1 (with a threshold of 0.5), indicate significant changes. On the other hand, differences below the threshold indicate no significant changes. Differences equal to the threshold also indicate significant changes.

[0048] Process block 311 may be followed by process block 313, which includes determining what process to perform next based on a comparison. If the comparison exceeds a threshold, then the composition and / or crystal map of the subsequent surface is acquired. Thus, the subsequent surface now has an associated image and an associated map. Additionally, the subsequent surface becomes the next reference surface. However, if the comparison does not exceed the threshold, the method continues to process block 307 to expose the additional surface.

[0049] The reference surface may alternatively be referred to as the current reference surface because it has both an image and one or more associated map maps. Generally, method 301 involves data acquisition of map maps and images of a subset of sample surfaces, where each surface in the subset will have an associated image and a crystal / composition map. The remaining surfaces may only have associated images.

[0050] It should be noted that method 301 can be used to expose and image / map a large number of surfaces, where a subset of the surfaces will be reference surfaces, as those surfaces will have associated maps and images. The remaining surfaces may only have associated images. Each time a comparison exceeds a threshold, a new reference image is identified due to changes in surface morphology. It should be noted that there is no predetermined number or periodicity of reference surfaces, and the resulting reference surfaces are determined by changes in the sample during processing. Additionally, when a new reference surface is identified, subsequently exposed surfaces are compared to the new reference surface, rather than any previous reference surface.

[0051] Figure 4 This is an illustrative image sequence 401 according to embodiments disclosed herein. Image sequence 401 includes data from four consecutive surfaces, where two surfaces have associated images and maps, while the intermediate surface has only associated images. The images and maps of image sequence 401 are acquired using method 301 implemented in a dual-beam charged particle microscope such as DB system 100.

[0052] Image sequence 401 includes data on surfaces 403, 405, 407, and 409. Surfaces 403 and 409 are reference surfaces and have associated images and maps, such as crystal and composition maps. For example, surface 403 has an associated image 411 and two associated composition maps 413 and 415. For example, the two composition maps 413 and 415 are single-element maps. Although the composition map of surface 403 is divided into two, one for each of the two elements, such branching of the maps is not necessary.

[0053] Surfaces 405 and 407 have only associated images 417 and 419, respectively. Finally, similar to surface 403, surface 409 has associated image 421 and associated composition maps 423 and 425.

[0054] The data associated with surfaces 403 to 409 can be part of a larger dataset of associated samples and can be used to provide multi-dataset 3D reconstructions of the samples.

[0055] Figure 5 This is an exemplary workflow 501 according to embodiments of the present disclosure for propagating crystal and / or composition maps to an unmapped surface. Workflow 501 can be performed in real time during the slicing and observation process as data is acquired. Alternatively, workflow 501 can be performed after the sample has been processed and all desired data has been obtained. In such embodiments, map propagation can be performed by a microscope or at a user's desktop with access to one or more neural networks. Generally, method 501 results in the generation of a composition map for a surface that does not have an associated acquired map, in order to obtain a 3D reconstruction having both an image and a composition / crystal map.

[0056] Workflow 501 includes reference images and maps 503, neural networks 505 and 507, image 509, and interpolation map 511. Reference images and maps 503 are images and crystal / composition maps of a subset of the sample's surfaces, which have been identified as reference surfaces. Accordingly, each reference surface has an associated image and crystal and / or composition map. Additionally, the reference surfaces and their associated images / maps can be in order from a first reference surface to an nth reference surface, such that there are one or more gap surfaces between the first and second reference surfaces. Generally, in addition to the first and nth reference surfaces, each reference surface can have preceding and following reference surfaces. For ease of discussion, each reference surface will have a set of data, images, and one or more maps, and such data units will be referred to as RS. i Here, RS represents the reference set, and i indicates the identity of the reference surface. Generally, a reference set can be identified as RS. i-1 RS i and RS i+1 .

[0057] On the other hand, image 509 is an image of interstitial surfaces, which may not have associated maps, whether crystalline or compositional. Accordingly, each of the interstitial surfaces has two corresponding reference surfaces, one preceding and one following it. Image 509 can also be referenced using naming conventions such as I1, I2, I3, ..., I... m Or more generally as I j , where j is from 1 to m. As discussed in some previous examples, the interstitial surface may also have a sparse map of composition and / or crystal information, which can be extended using the interpolation process disclosed in workflow 501.

[0058] Generally, there are multiple surfaces that are imaged, such as using method 301, where a subset of those surfaces are also mapped. The remaining surfaces either have only associated images or only associated maps. The subset of surfaces with both associated images and associated maps will be the reference images used to form maps for surfaces that only have images, using neural networks 505 and 507. Neural networks 505 and 507 can both be U-Nets known in the art and have been trained using similar data.

[0059] In operation, for example, neural network 505 receives a reference image and a map 503 as input, and determines the image from the reference image 505. i To RS i+1 Bidirectional optical flow. Bidirectional optical flow essentially uses two sets of data to associate image pixel values ​​with associated image pixel values. This mapping is then used as input to neural network 507, which also receives image 509 as input. Subsequently, neural network 505 in RS... i and Image I j Between and in I j and RS i+1 A refined bidirectional optical flow is produced between them, I j It is related to RS i and RS i+1 The surfaces between the associated surfaces. The neural network 507 further produces RS. i and RS i+1 A visual map of each pixel in the image. Therefore, and by applying a refined optical flow and visual map to this map, an interpolated map of each gap surface (with only image 509) can be obtained from workflow 501.

[0060] From different lens perspectives, workflow 501 correlates the grayscale pixel values ​​of the image with the composition / crystal map pixel values, and then uses this correlation to generate a map for each image in image 509. As a final result, each surface of the sample will have both an associated map and an image, in which some maps are acquired and some are interpolated.

[0061] Figure 6 This is an illustrative image sequence 601 according to embodiments of the present disclosure. Image sequence 601 may be a continuation of image sequence 401, for example, including interpolated maps. For example, surfaces 605 and 607 may have interpolated maps generated using workflow 501. For example, images and maps of surfaces 603 and 609 may be provided to workflow 501 together with images of surfaces 605 and 607 so that interpolated maps 633 and 635 can be generated for surface 605. Similarly, surface 607 may also be generated with an interpolated map. Generally, images and maps of RS 603 and 609 can be used to generate interpolated maps of surfaces 605 and 607.

[0062] Figure 7 This is an example functional block diagram 700 according to an embodiment of the present disclosure. Figure 7 A block diagram illustrating a possible computer system 700 included in embodiments of the present invention is shown. The computing system 700 may be an instance of computing hardware included in system 100, such as controller 119. The computer system 700 includes at least a bus 740 or other communication mechanism for transmitting information, and a hardware processor 742 coupled to the bus 740 for processing information. The hardware processor 742 may be, for example, a general-purpose microprocessor. The computing system 700 may be used to implement the methods and techniques disclosed herein, such as methods 301 and 401, and may also be used to acquire images and segment said images into one or more classes.

[0063] Computer system 700 also includes main memory 744, such as random access memory (RAM) or other dynamic storage device, coupled to bus 740, for storing information and instructions to be executed by processor 742. Main memory 744 can also be used to store temporary variables or other intermediate information during the execution of instructions to be executed by processor 742. When stored in a non-transitory storage medium accessible to processor 742, such instructions make computer system 700 a special-purpose machine customized to perform the operations specified in the instructions.

[0064] The computer system 700 further includes a read-only memory (ROM) 746 or other static storage device coupled to a bus 740 for storing static information and instructions for the processor 742. A storage device 748, such as a magnetic disk or optical disk, is provided and coupled to the bus 740 for storing information and instructions.

[0065] Computer system 700 can be coupled to display 750, such as a cathode ray tube (CRT), via bus 740 for displaying information to the computer user. Input device 752, including alphanumeric keys and other keys, is coupled to bus 740 for transmitting information and command selections to processor 742. Another type of user input device is cursor control 754, such as a mouse, trackball, or cursor arrow keys, for transmitting directional information and command selections to processor 742, and for controlling cursor movement on display 750. This input device typically has two degrees of freedom on two axes (a first axis (e.g., x) and a second axis (e.g., y)), allowing the device to specify a position in a plane.

[0066] Computer system 700 may implement the techniques described herein using custom hardwired logic, one or more ASICs or FPGAs, firmware, and / or program logic combined with the computer system to make or program the computer system 700 into a dedicated machine. According to one embodiment, the techniques herein are executed by computer system 700 in response to processor 742 executing one or more sequences of one or more instructions contained in main memory 744. Such instructions may be read into main memory 744 from another storage medium, such as storage device 748. Execution of the sequence of instructions contained in main memory 744 causes processor 742 to perform the process steps described herein. In alternative embodiments, hardwired circuitry may be used instead of or in combination with software instructions.

[0067] As used herein, the term "storage medium" refers to any non-transitory medium that stores data and / or instructions that enable a machine to operate in a specific manner. Such storage media can include non-volatile media and / or volatile media. Non-volatile media include, for example, optical discs or magnetic disks, such as storage device 748. Volatile media include dynamic memory, such as main memory 744. Common forms of storage media include, for example, floppy disks, floppy hard disks, hard disks, solid-state drives, magnetic tape or any other magnetic data storage media, CD-ROMs, any other optical data storage media, any physical media with a perforated pattern, RAM, PROMs and EPROMs, flash memory-EPROMs, NVRAMs, any other memory chips or cartridges, content-addressable memory (CAM), and ternary content-addressable memory (TCAM).

[0068] Storage media differ from transmission media, but can be used in conjunction with them. Transmission media participate in the information transmission between storage media. For example, transmission media include coaxial cables, copper wires, and optical fibers, including conductors containing bus 740. Transmission media can also take the form of sound waves or light waves, such as those generated during wireless and infrared data communication.

[0069] Various forms of media can involve carrying one or more sequences of one or more instructions to processor 742 for execution. For example, the instructions may initially be carried on a disk or solid-state drive of a remote computer. The remote computer may load the instructions into its dynamic memory and transmit them over a telephone line using a modem. A modem local to computer system 700 may receive data over the telephone line and convert the data into an infrared signal using an infrared transmitter. An infrared detector may receive the data carried in the infrared signal, and appropriate circuitry may place the data on bus 740. Bus 740 carries the data to main memory 744, from which processor 742 retrieves and executes the instructions. Instructions received by main memory 744 may optionally be stored on storage device 748 before or after execution by processor 742.

[0070] Computer system 700 also includes a communication interface 756 coupled to bus 740. Communication interface 756 provides bidirectional data communication coupled to network link 758 connected to local network 760. For example, communication interface 756 may be an Integrated Services Digital Network (ISDN) card, cable modem, satellite modem, or modem to provide data communication connectivity to a corresponding type of telephone line. As another example, communication interface 756 may be a Local Area Network (LAN) card to provide data communication connectivity to a compatible LAN. Wireless links may also be implemented. In any such implementation, communication interface 756 transmits and receives electrical, electromagnetic, or optical signals carrying digital data streams representing various types of information.

[0071] Network link 758 typically provides data communication to other data devices via one or more networks. For example, network link 758 may provide a connection to host computer 762 or data devices operated by Internet service provider (ISP) 764 via local network 760. ISP 764, in turn, provides data communication services via a global packet data communication network now commonly referred to as the "Internet" 766. Both local network 760 and Internet 766 use electrical, electromagnetic, or optical signals that carry digital data streams. Signals through various networks, signals on network link 758, and signals through communication interface 756 are exemplary forms of transmission media carrying digital data to and from computer system 700.

[0072] Computer system 700 can send messages and receive data, including program code, via a network, network link 758, and communication interface 756. In an Internet instance, server 768 can transmit application request codes via the Internet 766, ISP 764, local network 760, and communication interface 756.

[0073] The received code may be executed by processor 742 when it is received, and / or stored in storage device 748 or other non-volatile memory for later execution.

[0074] The embodiments discussed herein to illustrate the disclosed techniques should not be considered limiting, but rather merely examples of implementations. For instance, instead of obtaining a complete map of a subset of surfaces, or in addition to obtaining a complete map of a subset of surfaces, sparse maps can be obtained from all surfaces, and then neural networks can be used to refine the sparse maps to complete them. Those skilled in the art will understand that numerous other ways in which the disclosed techniques can be implemented are considered herein and within the scope of this disclosure.

[0075] An exemplary method for generating data for three-dimensional reconstruction includes at least: exposing a subsequent surface of a sample; acquiring an image of the subsequent surface; comparing the image of the subsequent surface with an image of a reference surface; acquiring a composition or crystal map of the subsequent surface based on the comparison exceeding a threshold; and exposing the next surface based on the comparison not exceeding the threshold.

[0076] The exemplary method described above further includes acquiring an image of a reference surface and acquiring a crystal or compositional map of the reference surface.

[0077] The exemplary method described above further includes: acquiring an image of the next surface; comparing the image of the next surface with a reference image; acquiring the composition or crystallographic map of the next surface based on the comparison exceeding a threshold; and exposing an additional surface based on the comparison not exceeding a threshold.

[0078] The exemplary method described above, wherein comparing an image of a subsequent surface with an image of a reference image includes determining whether the difference between the images is greater than a threshold.

[0079] The exemplary method described above, wherein exposing the subsequent surface includes grinding a layer from the sample with a focused ion beam, thereby exposing the subsequent surface during the grinding process.

[0080] The exemplary method described above further includes the steps of repeatedly exposing, acquiring, and comparing multiple surfaces, wherein a subset of the multiple surfaces will have associated images and associated crystal or compositional maps, and wherein the remaining surfaces of the multiple surfaces will have associated images.

[0081] The exemplary method described above further includes using one or more neural networks to propagate the crystal or compositional map to the remaining surfaces of the plurality of surfaces.

[0082] The exemplary method described above, wherein propagating a crystal or composition map to the remaining surfaces of a plurality of surfaces using one or more neural networks includes interpolating crystal or composition maps from a subset of surfaces of the plurality of surfaces using one or more neural networks to form crystal or composition maps of the remaining surfaces of the plurality of surfaces.

[0083] The exemplary method described above, wherein propagating a crystal or composition map to the remaining surfaces of a plurality of surfaces using one or more neural networks includes associating pixel values ​​of a crystal or composition map of an individual surface of a subset of surfaces with pixel values ​​of an image of an individual surface of the subset of surfaces using one or more neural networks, and assigning pixel values ​​of the crystal or composition map to pixel values ​​of an image of one of the remaining surfaces based on the associated pixel values ​​using one or more neural networks.

[0084] In the above exemplary method, the process of comparing at least the image of the subsequent surface with the image of the reference surface is performed by a neural network.

[0085] The exemplary method described above, wherein acquiring an image of a subsequent surface includes acquiring an image of the subsequent surface using an electron beam or a focused ion beam.

[0086] The exemplary method described above, wherein obtaining the composition or crystal map of a subsequent surface includes obtaining the composition map of the subsequent surface using energy-dispersive X-ray spectroscopy or cathodic luminescence, and obtaining the crystal map of the subsequent surface using electron backscatter diffraction.

[0087] The exemplary method described above further includes obtaining a sparse composition or crystal map of a subsequent surface.

[0088] An exemplary device for generating data for 3D reconstruction includes at least an electron column coupled to provide an electron beam, a focused ion beam column coupled to provide a focused ion beam, a plurality of detectors arranged to detect at least secondary electrons, X-rays, and backscattered electrons, and a controller coupled to at least the electron column, the focused ion beam column, and each of the plurality of detectors. The controller includes or is coupled to code that, when executed by the controller, causes the device to: expose a subsequent surface of the sample by the focused ion beam; acquire an image of the subsequent surface; compare the image of the subsequent surface with a reference image, the reference image being an image of a reference surface; acquire a compositional or crystallographic map of the subsequent surface based on a comparison exceeding a threshold; and expose the next surface based on a comparison not exceeding a threshold.

[0089] The exemplary device described above, wherein the controller further includes or is coupled to code that, when executed by the controller, causes the device to acquire an image of a reference surface and to acquire a crystal or compositional map of the reference surface.

[0090] The exemplary device described above, wherein the controller further includes or is coupled to code that, when executed by the controller, causes the device to: acquire an image of a next surface; compare the image of the next surface with a reference image; acquire a composition or crystallographic map of the next surface based on a comparison exceeding a threshold; and expose a subsequent surface based on a comparison not exceeding a threshold.

[0091] The exemplary device described above compares an image of a subsequent surface with a reference image, including determining whether the difference between the images is greater than a threshold.

[0092] The exemplary device described above, wherein the controller further includes or is coupled to code that, when executed by the controller, causes the device to repeatedly expose, acquire, and compare multiple surfaces, wherein a subset of the multiple surfaces will have associated images and crystal or compositional maps, and wherein the remaining surfaces of the multiple surfaces will have associated images.

[0093] The exemplary device described above, wherein the controller further includes or is coupled to code that, when executed by the controller, causes the device to use one or more neural networks to propagate a crystal or compositional map to the remaining surfaces of a plurality of surfaces.

[0094] The exemplary device described above, wherein the controller further includes or is coupled to code that, when executed by the controller, causes the device to use one or more neural networks to interpolate crystal or compositional maps of a subset of surfaces from a plurality of surfaces to form crystal or compositional maps of the remaining surfaces of the plurality of surfaces.

[0095] The exemplary device described above, wherein the controller further includes or is coupled to code that, when executed by the controller, causes the device to use one or more neural networks to associate pixel values ​​of a crystal or composition map of an individual surface of a subset of surfaces with pixel values ​​of an image of an individual surface of the subset of surfaces, and to use one or more neural networks to assign pixel values ​​of the crystal or composition map to pixel values ​​of an image of one of the remaining surfaces based on the associated pixel values.

[0096] The exemplary device described above includes a controller comprising or coupled to one or more neural networks trained to perform a comparison step of an image of a subsequent surface with a reference image, the reference image being an image of a reference surface, and based on the comparison, providing control signals to the device.

[0097] The exemplary device described above, wherein the controller further includes or is coupled to code that, when executed by the controller, causes the device to acquire a compositional map of a subsequent surface using energy-dispersive X-ray spectroscopy or cathodoluminescence, and to acquire a crystal map of a subsequent surface using electron backscatter diffraction.

[0098] The device according to the preceding claims, wherein the controller further includes or is coupled to code that, when executed by the controller, causes the device to acquire a sparse composition or crystal map of a subsequent surface.

Claims

1. A method for generating three-dimensional reconstruction data, comprising: Acquire an image of the reference surface of the sample; Remove the material layer from the sample to expose the subsequent surfaces of the sample; Acquire an image of the subsequent surface; as well as The following steps selectively determine whether to obtain the composition or crystallographic map of the subsequent surface: The image of the subsequent surface is compared with the image of the reference surface; If the comparison exceeds a threshold, the subsequent surface becomes a reference surface and the composition or crystal map of the subsequent surface is obtained; as well as Based on the fact that the comparison does not exceed the threshold, the material layer is removed from the sample to expose the next surface of the sample without obtaining the composition or crystal map of the subsequent surface.

2. The method according to claim 1, further comprising: Obtain an image of the next surface; The image of the next surface is compared with the image of the reference surface; Based on the comparison exceeding a threshold, the composition or crystal map of the next surface is obtained; and Based on the fact that the comparison does not exceed the threshold, the additional surface is exposed.

3. The method of claim 1, further comprising: The steps of repeatedly exposing, acquiring, and comparing multiple surfaces, wherein a subset of the multiple surfaces will have associated images and associated crystal or compositional maps, and the remaining surfaces of the multiple surfaces will have associated images.

4. The method of claim 3, further comprising: One or more neural networks are used to propagate the crystal or compositional map to the remaining surfaces of the plurality of surfaces.

5. The method of claim 4, wherein using one or more neural networks to propagate the crystal or compositional map to the remaining surfaces of the plurality of surfaces comprises: The crystal or compositional map of the subset of the plurality of surfaces is interpolated using the one or more neural networks to form the crystal or compositional map of the remaining surfaces of the plurality of surfaces.

6. The method of claim 5, wherein using one or more neural networks to propagate the crystal or compositional map to the remaining surfaces of the plurality of surfaces comprises: The pixel values ​​of the crystal or composition map of an individual surface of the surface subset are associated with the pixel values ​​of the image of that individual surface using the one or more neural networks; and Using the one or more neural networks, pixel values ​​of the crystal or constituent map are assigned to the pixel values ​​of an image of one of the remaining surfaces based on associated pixel values.

7. The method of claim 1, wherein at least the process of comparing the image of the subsequent surface with the image of the reference surface is performed by a neural network.

8. The method of claim 1, further comprising: Obtain the sparse composition or crystal map of the subsequent surface.

9. An apparatus for generating three-dimensional reconstruction data, comprising: Electron columns coupled to provide an electron beam; Coupled to provide a focused ion beam column; Multiple detectors are arranged to detect at least secondary electrons, X-rays, and backscattered electrons; as well as A controller coupled at least to the electron column, the focused ion beam column, and each of the plurality of detectors, the controller comprising code stored in a non-transitory storage medium, the code causing the device to: Acquire an image of the reference surface of the sample; The focused ion beam removes the material layer from the sample to expose the subsequent surfaces of the sample; Acquire an image of the subsequent surface; as well as The following steps selectively determine whether to obtain the composition or crystallographic map of the subsequent surface: The image of the subsequent surface is compared with a reference image, which is an image of the reference surface; If the comparison exceeds a threshold, the subsequent surface becomes a reference surface and the composition or crystal map of the subsequent surface is obtained; as well as Based on the fact that the comparison does not exceed the threshold, the material layer is removed from the sample to expose the next surface of the sample without obtaining the composition or crystal map of the subsequent surface.

10. The device of claim 9, wherein the code, when executed by the controller, further causes the device to: Obtain an image of the next surface; The image of the next surface is compared with the reference image; Based on the comparison exceeding a threshold, the composition or crystal map of the next surface is obtained; as well as If the comparison does not exceed the threshold, the subsequent surface is exposed.

11. The device of claim 9, wherein the code, when executed by the controller, further causes the device to: The steps of repeatedly exposing, acquiring, and selectively determining multiple surfaces, wherein a subset of the multiple surfaces will have associated images and associated crystal or composition maps, and the remaining surfaces of the multiple surfaces will have associated images but no associated crystal or composition maps.

12. The device of claim 11, wherein the code, when executed by the controller, further causes the device to: One or more neural networks are used to propagate the crystal or compositional map to the remaining surfaces of the plurality of surfaces.

13. The device of claim 12, wherein using one or more neural networks to propagate the crystal or compositional map to the remaining surfaces of the plurality of surfaces comprises: The crystal or compositional map of the subset of the plurality of surfaces is interpolated using the one or more neural networks to form the crystal or compositional map of the remaining surfaces of the plurality of surfaces.

14. The device of claim 12, wherein using one or more neural networks to propagate the crystal or compositional map to the remaining surfaces of the plurality of surfaces comprises: The one or more neural networks are used to associate the pixel values ​​of the crystal or composition map of an individual surface of the surface subset with the pixel values ​​of the image of the individual surface of the surface subset. as well as Using the one or more neural networks, pixel values ​​of the crystal or constituent map are assigned to the pixel values ​​of an image of one of the remaining surfaces based on associated pixel values.

15. The device of claim 9, wherein the code, when executed by the controller, further causes the device to: Obtain the sparse composition or crystal map of the subsequent surface.

Images