How to Examine a Sample Using a Charged Particle Microscope
The HSV color space and algorithms like artificial neural networks are used to assign hues to phases in charged particle microscopy, addressing the challenge of interpreting phase information and enhancing image readability and consistency.
Patent Information
- Application Number
- JP2020081284
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2019-05-06
- Filing Date
- 2020-05-01
- Publication Date
- 2025-01-07
- Estimated Expiration
- 2040-05-01
AI Technical Summary
Interpreting phase information in charged particle microscopy images is challenging due to the difficulty in assigning consistent and meaningful hues to different phases, making it time-consuming and difficult for users to understand the sample composition.
A method using the HSV color space to assign different hues to different phases based on spectral information, ensuring equal hue spacing angles and utilizing algorithms like artificial neural networks to enhance readability and consistency in image representation.
The method provides informative, consistent, and easily interpretable color images that enhance the user's understanding of sample composition by mapping phases with visually pleasing and readable hues.
Smart Images

Figure 0007608695000001 
Figure 0007608695000002 
Figure 0007608695000003
Abstract
Description
[Technical field]
[0001] The present invention relates to a method for examining a sample using a charged particle microscope.
[0002] The invention also relates to a charged particle microscope for examining a sample using such a method. [Background technology]
[0003] Charged particle microscopy is a well-known and increasingly important technique for imaging microscopic objects, especially in the form of electron microscopes.Historically, the basic type of electron microscope has evolved into many well-known device types, such as transmission electron microscopes (TEM), scanning electron microscopes (SEM), and scanning transmission electron microscopes (STEM), as well as into various subtypes, such as the so-called "dual beam" device (e.g., FIB-SEM), which further uses a "machining" focused ion beam (FIB) to enable supporting activities such as ion beam milling or ion beam induced deposition (IBID).Those skilled in the art will be familiar with the different types of charged particle microscopes.
[0004] Irradiation of a sample with a scanning electron beam stimulates emission of "auxiliary" radiation from the sample in the form of secondary electrons, backscattered electrons, X-rays, and cathodoluminescence (infrared, visible, and / or ultraviolet photons). One or more components of this emitted radiation can be detected and used for sample analysis.
[0005] Typically in an SEM, backscattered electrons are detected by a solid-state detector, and each backscattered electron is amplified to create many electron-hole pairs in the semiconductor detector. The backscattered electron detector signal is used to form an image as the beam is scanned, and the brightness of each image point is determined by the number of backscattered electrons detected at the corresponding point on the sample as the primary beam moves across the sample. The image only provides information about the topology of the sample being inspected.
[0006] In a process called "energy dispersive X-ray analysis" or "EDS", the energy of incoming X-rays from a sample in response to an electron beam is measured and plotted in a histogram to form a material-specific spectrum. The measured spectrum can be compared to known spectra of various elements to determine which elements and minerals are present in the sample.
[0007] It is known to provide a set of color images containing EDS information, where each color image represents a different phase using a different hue. In this so-called element mapping technique, the user selects one or more phases associated with the sample and assigns a desired hue to each of those phases. This results in a set of images, each containing information about the associated phases of a different hue. The set of color images can be compared to an SEM image to identify areas of interest. However, it is relatively difficult to interpret these images in a meaningful and consistent way by the user. To that end, attempts have been made to combine the different phases and the corresponding colors into a single color image. However, selecting phases and assigning hues in a meaningful and consistent way is difficult and rather time-consuming. Summary of the Invention
[0008] It is therefore an object to provide an improved method for inspecting a sample, in particular the method providing an image of the sample with improved phase information.
[0009] For this purpose, a method according to claim 1 is provided. The method defined herein comprises the steps of providing a charged particle beam as well as a sample and scanning the charged particle beam across the sample. A first type of radiation emitted from the sample in response to the charged particle beam scanned over the sample is detected using a first detector. A plurality of distinct phases are then assigned to the sample, using the spectral information of the detected first type of radiation. By way of example, the EDS process described above can be used in this step to form a material specific spectrum and to determine which elements and minerals (i.e. phases) are present in the sample. Other methods of obtaining and using spectral information can also be used.
[0010] As defined herein, the method further comprises associating a plurality of different hues corresponding to the plurality of mutually different phases with reference to the HSV color space. This means that different hues (e.g., green, red, orange, blue) are assigned to different phases (e.g., elements, chemicals, minerals, etc.). The HSV color space is a color space that uses hue (often referred to as "color" e.g., red, green, blue, etc.), saturation (often referred to as the intensity or purity of the color), and value (often referred to as the lightness or darkness of the color) to describe the attributes of a particular color. The HSV color space can be represented as a cylinder shape with hue and its angle dimension starting from the red primary at 0°, passing through the green primary at 120° and the blue primary at 240°, before wrapping back to red at 360°. The central vertical axis comprises neutral, achromatic, or gray colors ranging from black with a value of 0 at the bottom to white with a value of 1 at the top. The larger the radius, the more saturated the color.
[0011] In principle, any color can be described in the HSV color space, regardless of the actual color space used. Thus, the methods described herein are not limited to the use of any particular color space, but can also be applied to the RGB color space (as all RGB colors can also be described with reference to the HSV color space), or any other color space as the situation requires.
[0012] According to the method, a control unit is used to provide an image representation of the sample, the image representation comprising the associated different hues. The control unit is thus used to provide phase information to a user, the information being coded in the different hues. The image representation can be a single image, but the use of multiple images, or a data table comprising image representations, etc. is also conceivable. The image representation also preferably comprises different values and / or different saturations.
[0013] As defined herein, the method comprises the additional step of providing a preselected range of continuous hues. The preselected range of continuous hues has increasing or decreasing hue values, i.e. increasing or decreasing angle dimensions, with respect to the HSV color space, but need not be provided or stored in an ordered manner during preselection. It is noted that it is not necessary to provide a complete angle range, i.e. not all hues (red, purple, blue, green, yellow, orange, [red], and / or intermediate) need to be used in the preselected color range. For example, it is conceivable that a preselected range of hues with limited angle values is provided, e.g. red, orange, yellow (i.e. 0° to 60°), etc. In other embodiments, different variations of a single hue are provided, i.e. bluish green, green, and yellowish green (i.e. 90° to 150°).
[0014] Thus, the preselected continuous color range includes an angular hue range relative to HSV color space. In an embodiment, the preselected continuous color range includes a full angular hue range relative to HSV space. In another embodiment, the preselected range includes a limited angular hue range relative to HSV space. According to the method, the step of associating a corresponding plurality of different hues with said plurality of mutually different phases with reference to an HSV color space comprises the steps of: - selecting the plurality of different hues from the preselected range of contiguous hues such that the selected hues comprise corresponding intervals within the preselected range of contiguous hues; and - associating said selected hue with said plurality of distinct hues;
[0015] In this way, the sample image representation includes the selected hues having similar hue interval angles relative to the HSV color space. The hues are selected such that the hue interval angles between the hues ultimately associated and used are substantially equal from the start hue to the end hue, and the differences between the different phases are directly visible, thus improving the readability of the final image and ensuring that interpretation of the results can be performed quickly and reliably by the user. This is particularly true when a single image containing multi-phase information is provided, especially when the number of phases shown in the final image is relatively large.
[0016] Note that if the full angle range (0° to 360°) is used, the hue interval angle is approximately equal to 360° / n, where n is the number of hues used. If only a limited angle hue range is used, the interval angle is determined accordingly, such that substantially equal interval angles are used from the start color to the end color.
[0017] In an embodiment, a fixed hue spacing angle is used for the additional phases, the fixed hue spacing angle being equal to the golden angle. By using the golden angle (approximately 137.5°) to select the additional hues for each additional phase, it is ensured that all hues are selected such that the next color is selected from the largest unused gap and no colors are repeated for a given model, regardless of the number of phases that need to be associated.
[0018] Thus, the above described method can utilize the mapping of the topology of the HSV color space into a high-dimensional data space of phases in a defined manner. The method allows for the creation of informative, consistent and more readable color images containing relevant information about the sample under examination. By using the method, the images are also visually pleasing to the user. This achieves the object of the present invention.
[0019] Advantageous embodiments are described below.
[0020] In an embodiment, the step of selecting the plurality of different hues comprises using information about the plurality of different phases as input variables. The information may include the number of phases detected in the sample, which helps to determine the hue interval angle. The information may also include the number of different phases, which means that similar or somewhat similar phases are grouped together and assigned a single hue or similar hues. Similar phases may be phases that have a similar composition, for example because they contain one or more chemical elements that are the same, such as these or compatible hues.
[0021] In an embodiment, the step of providing a range of preselected continuous hues may comprise a step of assigning one or more of the preselected hues to a corresponding one or more phases. This step may be performed by a control unit. The step may be initiated by said control unit or by a user. In this way, it is possible to assign, for example, blue hues to silicon, purple hues to oxygen, and orange hues to platinum. It is therefore possible to determine the specific boundary conditions of the hues that will ultimately be used. This provides consistency between different samples and different experiments, and allows the user to interpret the results of the samples more easily. The boundary conditions of preassigning specific hues to different phases obviously set some limitations on the hue interval angles that can be obtained by this embodiment, which means that not all hue interval angles need to be equal or similar. However, given these boundary conditions, the method preferably selects additional hues for additional phases such that the selected hues include mutually corresponding intervals within said range of preselected continuous hues.
[0022] In an embodiment, the method comprises repeating the steps of selecting the plurality of different hues and associating the reselected hues with the plurality of mutually different phases. The repeating step can be initiated by a control unit or by a user. The reselection can be performed, for example, during data collection of a sample. As more data is entered, additional phases can be detected and hues can be reselected based thereon. Additionally or alternatively, the reselection can be performed, for example, after setting or changing boundary conditions by a user. For example, given an initial result, it is conceivable that a user would like to assign specific hues to specific phases (or compositions) and request a change of other selected hues. This repeating step can also comprise providing further preselected contiguous hue ranges, the preselected contiguous hue ranges being different from the initially provided preselected contiguous hue ranges. For example, it is conceivable that a wider or smaller angle hue range is used during the repeated steps of selection and association.
[0023] In an embodiment, said first type of radiation is used to encode different saturation degrees within said image representation.
[0024] In an embodiment, a second detector is used to detect a second type of radiation different from the first type. The radiation of the second type can be, for example, secondary electrons and / or backscattered electrons. The radiation of the second type is used to code a different value in the image representation. In other words, an SEM is used to obtain a greyscale image of the sample under inspection, and color is used, preferably on top of the image, in the image representation to code the first type of radiation, preferably associated with EDS.
[0025] In an embodiment, the method comprises using an algorithm for selecting and associating said plurality of different hues.
[0026] The algorithm may, in an embodiment, comprise one or more of the following steps: -Provide the first color scheme. -Provide one or more adjustment variables. - generating a final color scheme and using this final color scheme to provide an image representation of said sample;
[0027] The adjustment variables can include one or more preferences, such as global preferences and / or user preferences. The adjustment variables can also include negative and positive adjustment values that are used to modify one or more preferences. For example, the composition of a sample can be used to generate adjustment values that modify a global preference to a final preference, thereby generating a final color scheme.
[0028] It should be noted that the above-mentioned algorithm utilizes a preselected continuous range of hues (i.e., an initial color scheme) and selects associated colors (i.e., a final color scheme) based on one or more adjustment variables as boundary conditions for selecting the final color scheme. Although the selected colors may provide corresponding hue interval angles to each other, it is conceivable that the hue interval angles do not substantially correspond. Thus, according to an embodiment, a method of inspecting a sample using a charged particle microscope is provided, comprising: - providing a charged particle beam with a sample; - scanning the charged particle beam over the sample; - detecting a first type of radiation from the sample using a first detector in response to the beam scanned across the sample; - assigning a plurality of distinct phases to the sample using spectral information of the detected first type of radiation; - associating a corresponding plurality of different hues with said plurality of mutually different phases, with reference to an HSV color space; - providing, by a control unit, an image representation of said sample, said image representation including said associated different hues; It is characterized by the following steps: -Provides a continuous range of preselected hues. - Using at least spectral information of the detected first type of radiation to provide one or more adjustment variables. - selecting said plurality of different hues from said preselected continuous range of hues based on said adjustment variable; and - associating said selected hue with said plurality of distinct hues;
[0029] Use of such an algorithm ensures that appropriate color schemes are used to create relevant, informative, consistent, and more readable color image representations. Many variations of such algorithms are contemplated, and some embodiments are illustrated by way of examples in the accompanying drawings. First, however, some other embodiments of the methods disclosed herein are described.
[0030] In an embodiment, the method comprises using an artificial neural network (ANN) for selecting and associating said plurality of different hues. ANNs can be advantageously used to select and associate relevant hues based on available data and are relatively efficient in doing so.
[0031] In an embodiment, the method comprises using a Self-Organizing Map (SOM) as said artificial neural network.
[0032] In an embodiment, different nodes of the ANN represent different phases. An outer edge of the ANN may correspond to the preselected continuous range of hues. For HSV color space, the ANN includes information on at least hue and saturation. Furthermore, the method may comprise the step of initializing the ANN to include a plurality of preselected phases as corresponding preselected nodes.
[0033] In an embodiment, the method comprises the step of training said ANN, said training step in particular comprising the step of matching said ANN with said assigned multiple distinct phases.
[0034] In an embodiment, the method comprises the step of pre-assigning particular hues to particular phases.
[0035] In an embodiment, the method comprises normalizing the color characteristic based on information regarding a plurality of distinct phases.
[0036] According to an aspect, there is provided a charged particle microscope for inspecting a sample, comprising: an optical column for focusing a beam of charged particles emitted from said charged particle source onto a sample, said optical column including a charged particle source, a final probe forming lens, and a scanner; a sample stage positioned downstream of the final probe forming lens and configured to hold the sample; a first detector for detecting a first type of radiation arising from the sample in response to the incidence of charged particles emitted by the charged particle source; a control unit and a processing device connected to the first detector.
[0037] As defined in this specification, in particular the control unit and / or the processing unit are configured to perform at least a part of the method, and therefore the charged particle microscope is configured to perform a method as described above.
[0038] In an embodiment, the charged particle microscope further comprises a second detector for detecting a second type of radiation arising from the sample in response to incidence of charged particles emitted from the charged particle source.
[0039] In an embodiment, the charged particle microscope further comprises an output device for outputting a combination of the processed first detection information and the processed second detection information, in particular, the output device is configured to output a color image, the color space of the color image comprising hue, value and saturation, a first type of radiation (e.g., EDS data) is used to define at least one of the hue and saturation of a color image; The second type of radiation (eg SEM data) is used to define values for the color image.
[0040] The invention will now be explained in more detail on the basis of exemplary embodiments and the accompanying schematic drawings. [Brief description of the drawings]
[0041] [Figure 1] FIG. 1 shows a vertical cross-sectional view of a charged particle microscope according to a first embodiment of the present invention. [Diagram 2] FIG. 2 shows a vertical cross-sectional view of a charged particle microscope according to a second embodiment of the present invention. [Diagram 3] FIG. 3 shows a representation of the HSV color space. [Figure 4] FIG. 4 shows a first embodiment for selecting and assigning hues to phases. [Diagram 5] FIG. 5 illustrates the use of an artificial neural network to select and assign hues to phases. [Figure 6] FIG. 6 shows a second embodiment for selecting and assigning hues to phases. [Figure 7] FIG. 7 illustrates an embodiment in which normalized color characteristics are used. [Figure 8]FIG. 8 shows an embodiment of an algorithm for selecting and assigning hues to phases. [Figure 9] FIG. 9 shows a sample image representation using the methods disclosed herein. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0042] FIG. 1 (not to scale) is a highly schematic depiction of an embodiment of a charged particle microscope M according to an embodiment of the present invention. More specifically, an embodiment of a transmission microscope M is shown, which in this case is a TEM / STEM (although in the context of the present invention it can usefully be an SEM (see FIG. 2) or e.g. an ion-based microscope). In FIG. 1, within a vacuum enclosure 2, an electron source 4 generates an electron beam B propagating along an electron-optical axis B′ and traversing an electron optical illuminator 6 which serves to direct / focus the electrons onto a selected portion of a sample S, which can e.g. be (locally) thinned / flattened. Also shown is a deflector 8, which can be used (among other things) to perform a scanning movement of the beam B.
[0043] The sample S is held in a sample holder H that can be positioned in multiple degrees of freedom by a positioning device / stage A, which moves a cradle A' to which the holder H is (removably) fixed; for example, the sample holder H can be equipped with fingers that can move (among other things) in the XY plane (see the Cartesian coordinate system shown; typically, movements parallel to Z and tilted about X / Y are also possible). Such movements allow different parts of the sample S to be illuminated / imaged / inspected by the electron beam B traveling along an axis B' (in Z direction) (and / or allow a scanning movement to be performed, as an alternative to beam scanning). If desired, an optional cooling device (not depicted) can be in intimate thermal contact with the sample holder H to maintain the sample holder H (and the sample S thereon) at, for example, a cryogenic temperature.
[0044] The electron beam B will interact with the sample S in such a way as to cause it to emit various types of "stimulated" radiation, including (for example) secondary electrons, backscattered electrons, X-rays, and light radiation (cathodoluminescence). If desired, one or more of these types of radiation can be detected with the aid of an analyzer 22, for example a combination scintillator / photomultiplier tube or an EDX or EDS (energy dispersive X-ray spectroscopy) module, in which case an image can be constructed using essentially the same principles as in an SEM. However, alternatively or supplementarily, the electrons that traverse (pass) the sample S, emerge / emit from the sample, and continue to propagate (substantially, but generally, deflected / scattered to some extent) along the axis B' can be investigated. Such a transmitted electron flux is incident on an imaging system (projection lens) 24, which typically comprises a wide variety of electrostatic / magnetic lenses, deflectors, correctors (such as stigmators), etc. In normal (non-scanning) TEM mode, this imaging system 24 can focus the transmitted electron beam onto a phosphor screen 26, which can be retracted / withdrawn (schematically indicated by arrow 26') out of the way of axis B', if desired. An image (or diffractogram) of (a part of) the sample S is formed on the screen 26 by the imaging system 24, which image can be viewed through a viewing port 28 positioned in a suitable part of the wall of the housing 2. The retraction mechanism of the screen 26 can be, for example, mechanical and / or electrical in nature, but is not depicted in the drawings.
[0045] As an alternative to viewing an image on screen 26, one can instead take advantage of the fact that the focal depth of the electron beam leaving imaging system 24 is typically quite deep (e.g., about one meter). As a result, various other types of analytical equipment can be used downstream of screen 26, such as: -TEM camera 30. At the position of camera 30 the electron beam can form a still image (or diffractogram) which can be processed by controller / processor 20 and displayed on a display device (not shown), such as a flat panel display. When not required, camera 30 can be retracted / retracted (as shown diagrammatically by arrow 30') to move the camera off axis B'. - STEM camera 32. The output from camera 32 can be recorded as a function of the (X,Y) scanning position of beam B on sample S, and an image can be constructed that is a "map" of the output from camera 32 as a function of X,Y. Camera 32 differs from the pixel matrix typically present in camera 30 in that it can include a single pixel, e.g., 20 mm in diameter. Furthermore, camera 32 can typically be a 10 mm pixel, e.g., 10 mm pixel. 2 A much higher acquisition speed (e.g., 10 images / sec) 6 points / sec). Again, when not needed, camera 32 can be retracted / withdrawn (as shown diagrammatically by arrow 32') to move the camera off axis B' (such retraction would not be required, for example, in the case of a doughnut-shaped annular dark field camera 32, where a central hole allows the electron flux to pass when the camera is not in use). As an alternative to imaging using the cameras 30 or 32, a spectroscopic device 34 can also be invoked, which can be, for example, an EELS module.
[0046] It should be noted that the order / position of components 30, 32, and 34 is not strict and many possible variations are possible. For example, spectroscopic device 34 could be integrated with imaging system 24.
[0047] In the embodiment shown, the microscope M further comprises a retractable X-ray computed tomography (CT) module, generally indicated at 40. In computed tomography (also called tomographic imaging), a source and a detector (at the opposite pole) are used to interrogate a sample along different lines of sight to obtain transmitted views of the sample from various viewpoints.
[0048] It should be noted that a controller (computer processor) 20 is connected to the various components shown via control lines (bus) 20'. This controller 20 may provide various functions such as synchronizing operations, providing set points, processing signals, performing calculations, and displaying messages / information on a display device (not shown). Of course, the controller 20 (schematically depicted) may be located (partially) inside or outside the housing 2 and may have a unitary or composite structure, as desired.
[0049] Those skilled in the art will understand that the interior of the housing 2 does not have to be kept in an airtight vacuum. For example, in so-called "environmental TEM / STEM", a background atmosphere of a given gas is intentionally introduced / maintained in the housing 2. Those skilled in the art will also understand that in practice it may be advantageous to confine the volume of the housing 2, possibly so that the housing 2 essentially envelops the axis B', so that the electron beam employed passes through a small diameter tube, but expands to accommodate structures such as the source 4, the sample holder H, the screen 26, the camera 30, the camera 32, the spectrometer 34, etc., in the form of a small diameter tube (e.g., about 1 cm in diameter).
[0050] The charged particle microscope M according to the invention, an embodiment of which is shown in Fig. 1, therefore comprises a charged particle source 4, a final probe-forming lens 6 and a scanner 8, and comprises an optical column O for focusing a beam B of charged particles emitted from the charged particle source 4 onto a sample. The device further comprises a sample stage A, H, positioned downstream of the final probe-forming lens 6 and configured to hold the sample S. The device further comprises a first detector 22 for detecting a first type of radiation arising from the sample in response to the incidence of the charged particles B emitted from the charged particle source 4. In the embodiment shown, the first detector 22 is an analysis device 22 combining a scintillator / photomultiplier or an EDS (Energy Dispersive X-ray Spectroscopy) module, as described above. In a preferred embodiment, the first detector is an EDS. Furthermore, the device according to the invention comprises a control device 20 (by a line 20') connected to the first detector 22 (shown diagrammatically). According to the invention, the charged particle microscope M is adapted to carry out the method according to the invention, which will be described below with the aid of FIGS.
[0051] Reference is now first made to Figure 2, which shows another embodiment of the apparatus according to the invention. Figure 2 (not to scale) is a very schematic representation of a charged particle microscope M according to the invention, more specifically showing an embodiment of a non-transmission microscope M, in this case an SEM (although in the context of the invention, for example, an ion-based microscope may be equally useful). In the figure, parts corresponding to those in Figure 1 are indicated using the same reference numerals and are not described separately here. The following parts (among others) are added to Figure 1: -2a: a vacuum port which can be opened to introduce / remove parts (components, samples) into / from the interior of the vacuum chamber 2 or onto which, for example, auxiliary devices / modules can be mounted. The microscope M can be equipped with multiple such ports 2a, if necessary. - 10a, 10b: lenses / optical elements within the illuminator 6 shown diagrammatically. -12: Optionally, a voltage source that allows the sample holder H or at least the sample S to be biased (floated) to a certain potential with respect to ground. -14: Display such as FPD or CRT. 22a, 22b: a segmented electron detector 22a, comprising a number of independent detection segments (e.g. quadrants) arranged around a central aperture 22b (allowing the passage of the beam B). Such a detector can be used, for example, to investigate the (angular dependence of) the output (secondary or backscattered) electron flux emanating from the sample S.
[0052] Thus, the charged particle microscope M shown in FIG. 2 comprises an optical column O including a charged particle source 4, a final probe-forming lens 6, 10a, 10b and a scanner 8, which focuses a beam B of charged particles emitted from said charged particle source 4 onto a sample S. The device further comprises a sample stage A, H positioned downstream of said final probe-forming lens 6 and configured to hold said sample S. The device further comprises a first detector 22 for detecting a first type of radiation arising from said sample in response to the incidence of the beam B of charged particles emitted from said charged particle source 4. In the embodiment shown, the first detector 22 is said analysis device 22 combined with a scintillator / photomultiplier or EDS (Energy Dispersive X-ray Spectroscopy) module, as described above. In an alternative embodiment, the first detector 22 can be a segmented detector 22a, 22b. In a preferred embodiment, said first detector is an EDS. Furthermore, the device according to the invention comprises the control device 20 which is connected to the first detector 22 (by a line 20 ′).
[0053] The apparatus shown in Figures 1 and 2 can be used to test a sample using the method of the present invention. In general, the embodiments of the method all comprise the following general steps: - scanning the charged particle beam B over the sample S; In response to the beam B being scanned across the sample S, a first detector 22 is used to detect a first type of radiation from the sample S. - assigning a plurality of distinct phases to the sample using spectral information of the detected first type of radiation. - associating a corresponding plurality of different hues with said plurality of different phases with reference to the HSV color space; providing, by a control unit 20, an image representation of said samples, said image representation comprising said associated different hues;
[0054] The method disclosed herein further comprises the steps of: -Provides a continuous range of preselected hues. - selecting the plurality of different hues from the preselected range of contiguous hues such that the selected hues comprise corresponding intervals within the preselected range of contiguous hues; and - associating said selected hue with said plurality of distinct hues;
[0055] The above mentioned method steps are explained in more detail below.
[0056] FIG. 3 shows an example of an HSV color space 101 used in the method defined herein. The HSV color space 101 is a color space that uses hue (often called "color", e.g. red, green, blue), saturation (often called the intensity or purity of the color) and value (often called the lightness or darkness of the color) to describe the attributes of a particular color. The HSV color space can be represented as a cylinder shape with hue and its angle dimension starting from the red primary at 0°, passing through the green primary at 120° and the blue primary at 240°, before wrapping back to red at 360°, as shown in FIG. 3. The central vertical axis comprises neutral, achromatic, or gray colors that range from black with a value of 0 (bottom of FIG. 3) to white with a value of 1 (top of FIG. 3). The larger the radius, i.e. from the center outwards, the more saturated the color. The details of the HSV color space are known per se to those skilled in the art of color spaces.
[0057] 4 shows an embodiment of a method in which different hues can be assigned to different phases. In this embodiment, a charged particle beam and a sample are provided, and the charged particle beam is scanned over the sample. A first type of radiation of the sample, such as X-rays, is detected. Based on the detected radiation, different phases of oxygen (O), aluminum (Al) and silicon (Si) are assigned to the sample.
[0058] 4 shows a schematic image representation of a portion of a color space 201 that can be used to provide an image representation of an inspected sample. Here, it is seen that a preselected range 201 of a total of eight consecutive hues 211-218 is provided, the following hues being used: 211 - red; 212-orange; 213-yellow; 214 - green; 215 - blue-green; 216-blue; 217-Violet; and 218 - Purple.
[0059] Based on the preselected range of continuous colors and the available different phases, a number of different hues can be selected. The selection is made such that the selected hues include intervals that correspond to each other within said preselected range of continuous hues. In the embodiment shown, the colors red 211, yellow 213 and blue 216 are selected. These colors have approximately equal hue interval angles within said preselected range of continuous hues, with reference to the HSV color space. In the embodiment shown, red 211 is then assigned to oxygen, yellow 213 to silicon and blue 216 to aluminum. Alternatively, it is conceivable that red 211 is assigned to aluminum, yellow 213 to oxygen and blue 216 to silicon.
[0060] The assignment of the selected hues to the different phases can be done randomly or in a more structured way, for example using an algorithm (see also the description of FIG. 8). For example, it is conceivable that boundary conditions are provided either by the control unit or by a user. These boundary conditions can include hues required for certain phases, such as the hue red 211 having to be assigned to oxygen (O). In this way, certain hues can be pre-assigned to certain phases. Based on these boundary conditions, additional colors can be selected from a list of pre-selected hues.
[0061] Alternatively, any hue, such as the first color, may be selected from said preselected continuous range of hues. It is not necessary to select the first hue within the preselected continuous range of hues. For example, for O, Si, and Al, the hues orange 212, green 214, and purple 217 may be selected, since these hues also exhibit approximately equal hue interval angles.
[0062] Alternatively, subsequent hues within the preselected range of colors could be used, for example, for Si, O and Al, the hues orange 212, yellow 213 and green 214 could be used, since these hues also exhibit approximately equal hue spacing angles.
[0063] FIG. 5 illustrates how an artificial neural network (ANN) may be useful for selecting and associating multiple different hues. FIG. 5 illustrates a color space 301 generated by such an ANN, which in an embodiment may be a self-organizing map (SOM). In this way, the chemical compositions can be mapped onto a network of nodes (311-316, 321, 322) that respect the interdependencies given by the color model. The network can be adapted according to the incoming data, i.e. when additional data are detected by the detector. Such an ANN, and in particular the SOM, may be useful in general for generating and selecting hues to be used in image representations of samples. The main advantage is that the ANN can be used and programmed to reflect the (dis)similarity of the phase compositions. - Phases with similar composition should have similar colors. -Overlapping phases must have adjacent colors on the color wheel to allow mixing. - Two different phases should not acquire similar colors; and - A distinct phase (different from all other phases) must have a distinct color.
[0064] Additionally, boundary conditions can be added to ensure that the ANN produces consistent results across experiments. For example, it is desirable for a particular phase, such as copper, to acquire the same color regardless of its surroundings. Thus, the ANN can be initialized to include a number of preselected phases as corresponding preselected nodes.
[0065] Furthermore, the ANN can be programmed to specifically adapt to the available data. In this way, for example, it is possible to prevent colors from being assigned if a particular phase is not present in the scene, or to assign them to a different phase instead. Ultimately, it is desirable to utilize hues throughout the color space in the final image representation of the sample, as this provides a way to communicate technical information to the user in the most readable and information dense manner. Thus, the ANN can be programmed to select the plurality of different hues from the preselected range of continuous hues such that the selected hues substantially include intervals that correspond to one another within the range of the preselected continuous hues. Of course, it is conceivable that not all hues have a corresponding hue interval angle, since certain boundary conditions exist.
[0066] Furthermore, similar phases can be given similar compatible colors. Thus, in an embodiment, it is considered that there are generally two distinct phases, namely phase A and phase B. For these two distinct phases, two opposite hues can be selected, such as orange and blue. Thus, phase A is represented by a blue hue and phase B by an orange hue. However, there are several variations of phase B in the sample, which can be visually represented by selecting different orange hues for these distinct phase B variations. Thus, for example, one can use orange-red to yellow (so-called compatible) hues for these phase B variations, and a single blue hue (so-called complementary) for phase A. In this way, hues are still selected from said preselected range of continuous hues, such that the selected hues include intervals that correspond to each other within said preselected range of continuous hues.
[0067] 5, it can be seen that the ANN contains information on at least hue and saturation, with reference to the HSV color space: the most saturated colors (e.g., the purest phases) are at the border of the network, and the less saturated colors (e.g., the less pure phases) are at the center of the network.
[0068] Figure 6 shows an embodiment of the method disclosed herein, in which an alternative approach is used, compared to Figure 4. In Figure 6, a total of eight hues (red 411, orange 412, yellow 413, green 414, cyan 415, blue 416, purple 417 and purple 418) are provided, which are associated with specific elements. For example, silicon (Si) can be pre-associated with red hue 411, aluminum (Al) can be pre-associated with purple hue 418, and oxygen (O) can be pre-associated with green hue 414. When data is input and it is found that Si, Al and O are present as elements of a sample, the corresponding red, purple and green hues can be provided as a pre-selected continuous range of hues. In other words, in this case, only three possible hues are provided: red, purple and green. Here, as shown in FIG. 6 with x marks in a triangle 401, different phases are present and different hues can be selected from a range of preselected consecutive hues (red, purple, green) such that the selected hues comprise corresponding intervals to each other within said preselected range of consecutive hues. In this regard, mixing or shifting of hues is contemplated in order to establish corresponding hue interval angles to each other. For example, in FIG. 6, it can be seen that clusters of x marks (each cluster representing a similar phase) are arranged within a triangle such that there are substantially corresponding hue interval angles between them. Thus, the final image has corresponding hue interval angles for each cluster of phases present in the sample.
[0069] Furthermore, based on the information about the different phases, the method may comprise a step of normalizing the color characteristics. FIG. 6 shows that the color space is slightly condensed with respect to the full color space (triangle). FIG. 7 (left side) shows an example where the different phases are highly condensed between the B and C edges of the color space. By normalizing the color characteristics, which may include normalization of hue, value and / or saturation, the use of colors in the final image representation can be expanded. In this regard, it is noted that the colors (dots) used in FIG. 7 (right side) after normalization provide an expansion of the color space. Thus, the image representation of the sample becomes more information dense and easier to read for the user.
[0070] FIG. 8 shows a diagram of an embodiment of the method aspect disclosed herein. FIG. 8 shows in schematic form an embodiment of an algorithm that can be used to assign hues to detected phases. Blocks U, V, W generally represent a user interface block U, a detector block V, and a processor block W. These blocks U, V, W provide inputs for an algorithm generally represented by algorithm blocks i-vi. Here, the algorithm comprises a step of generalizing an intermediate color scheme in algorithm block i, rendering a negative adjustment ii and a positive adjustment iv to generate a preference adjustment in algorithm block iii, and generating a final preference v using both the preference adjustment and the global preference coming from the processor block W. The final preference v allows to assign a color to a particular phase vi, so that the intermediate color scheme i can be updated again. The intermediate color scheme generated in this way is output to the user interface block U to provide the final color scheme.
[0071] The user interface block U can provide user preferences for the intermediate color scheme i. The detector block V can provide information on integral composition, phase overlap statistics, and / or other variables related to the detection properties of the sample to provide negative and / or positive adjustments ii, iv. The processor block W can provide global preferences to the algorithm.
[0072] In general, the algorithm may use the following steps: 1. Place "locked" elements in an intermediate color scheme i. The locked elements are provided by the user via a user interface block U. 2. Extract the next unplaced element that has the largest proportion in the sample. 3. Reduce the preference for colors that are already occupied. 4. Increasing the preference for colors that are compatible with already placed co-occurring elements, e.g., with maximum or corresponding hue interval angles. 5. These preference adjustments iii are combined with the global preferences from the processor unit block W to generate the final preferences v. 6. Extract colors according to final preferences vi ("Arrange elements"). 7. Repeat steps 2-6 until all detected elements or phases have been located. Negative and / or positive adjustments may be made, for example by equalizing weights according to integral composition.
[0073] Thus, in the embodiment shown in Figure 8, a preselected continuous range of hues is provided and one or more adjustment variables are provided using spectral information of at least the detected first type of radiation. Based on these adjustment variables, the plurality of different hues are selected from the preselected continuous range of hues. Finally, the selected hues are associated with the plurality of mutually different hues.
[0074] Note that the increases and decreases in steps 3 and 4 are analogous to the positive adjustment iv and negative adjustment ii, respectively, described above with respect to FIG.
[0075] FIG. 9 shows a black and white version of an image representation of a sample, in this case platinum rhodium particles on a ceramic (Si, Al, O). The image representation includes different hues selected by the methods disclosed herein. The hues include blue-green for aluminum, blue for silicon, orange for platinum, purple for oxygen, and yellow for rhodium. Platinum and rhodium are similar phases and are assigned compatible yellow and orange hues. Si, Al, and O are different phases and are assigned bluish hues to maximize the hue interval angle used in the image representation. As a result, the Pt-Rh particles are clearly visible and more information about the sample is obtained.
[0076] Several embodiments of this method have been disclosed herein. The desired protection is determined by the appended claims.
[0077] References: Figure 7 - Adapted from: Pawlowsky-Glahn, Egozcue, Tolosana-Delgado: Lecture Notes on Compositional Data Analysis (http: / / www.sediment.uni-goettingen.de / staff / tolosana / extra / CoDa.pdf, p. 36, see MARTIN-FERNANDEZ, Josep A. et al., Measuring Difference in Compositional Data Based on Divergence Measurements, in: Proceedings of IAMG. 1999, p. 211-216). [Explanation of symbols]
[0078] 2 Vacuum enclosure 2a Vacuum port 4 Electron source 4. Charged Particle Source 6 Electro-optical illuminator 6 Final probe forming lens 8 Deflector 8. Scanner 10a, 10b Lens / Optical element 12 Voltage Source 14 Display 20 Controllers and / or Processors 20' control line 22 Analyzer 22a Electron detector 22b center opening 24 Imaging System 26 Fluorescent Screen 26', 30', 32' Arrows 28 Sighting Port 30 TEM Camera 32 STEM Camera 34 Spectroscopic equipment 40 Microscope 101 HSV color space 201 Range 211~218, 411~418 Hue 301 color space 311~316, 321, 322 nodes 401 Triangle A Positioning device / stage A' Cradle B Electron beam B' axis H Holder O Optical column S sample M microscope U, V, W Blocks
Claims
1. 1. A method of inspecting a sample using a charged particle microscope, comprising: providing a charged particle beam with a sample; scanning the charged particle beam over the sample; detecting a first type of radiation from the sample using a first detector in response to the charged particle beam being scanned across the sample; assigning a plurality of distinct phases to the sample using spectral information of the detected first type of radiation; referencing an HSV color space to associate a corresponding plurality of different hues with said plurality of different phases; providing, by a control unit, an image representation of said sample, said image representation including said associated different hues; The associating step includes: providing a preselected continuous range of hues; selecting said plurality of different hues from said preselected range of contiguous hues, said selected hues comprising equal hue interval angles within said preselected range of contiguous hues; associating the selected hue with the plurality of distinct hues; repeating the step of selecting the plurality of different hues and associating the reselected hues with the plurality of different hues.
2. The method of claim 1 , wherein the step of selecting the plurality of different hues comprises using information about the plurality of different hues as an input variable.
3. The method of claim 1 or 2, comprising using an algorithm for selecting and associating the plurality of different hues.
4. The method of claim 3 , wherein the algorithm is non-deterministic.
5. 1. A method of inspecting a sample using a charged particle microscope, comprising: providing a charged particle beam with a sample; scanning the charged particle beam over the sample; detecting a first type of radiation from the sample using a first detector in response to the charged particle beam being scanned across the sample; assigning a plurality of distinct phases to the sample using spectral information of the detected first type of radiation; referencing an HSV color space to associate a corresponding plurality of different hues with said plurality of different phases; providing, by a control unit, an image representation of said sample, said image representation including said associated different hues; The associating step includes: providing a preselected continuous range of hues; selecting said plurality of different hues from said preselected range of contiguous hues, said selected hues comprising equal hue interval angles within said preselected range of contiguous hues; associating the selected hue with the plurality of distinct hues; and using an artificial neural network (ANN) for selecting and associating the plurality of different hues.
6. The method of claim 5 comprising using a self-organizing map (SOM) as the ANN.
7. The method of claim 5 or 6, wherein different nodes of the ANN represent different phases.
8. A method according to any one of claims 5 to 7, wherein the outer edge of the ANN corresponds to the preselected range of continuous hues.
9. The method according to any one of claims 5 to 8, wherein the ANN contains information on at least hue and saturation, with reference to the HSV colour space.
10. A method according to any one of claims 6 to 9, comprising the step of initialising the ANN to include a number of preselected phases as corresponding preselected nodes.
11. A method according to any one of claims 6 to 10, comprising the step of training the ANN, said training step in particular comprising the step of matching the ANN with the assigned multiple distinct phases.
12. A method according to any preceding claim, comprising the step of pre-assigning particular hues to particular phases.
13. A method according to any preceding claim, comprising normalising colour characteristics based on information about the plurality of distinct phases.
14. 1. A method of inspecting a sample using a charged particle microscope, comprising: providing a charged particle beam with a sample; scanning the charged particle beam over the sample; detecting a first type of radiation from the sample using a first detector in response to the charged particle beam being scanned across the sample; assigning a plurality of distinct phases to the sample using spectral information of the detected first type of radiation; referencing an HSV color space to associate a corresponding plurality of different hues with said plurality of different phases; providing, by a control unit, an image representation of said sample, said image representation including said associated different hues; The associating step includes: providing a preselected continuous range of hues; providing one or more adjustment variables using at least spectral information of the detected first type of radiation; selecting the plurality of different hues from the preselected continuous range of hues based on the adjustment variable; associating a selected hue with the plurality of distinct hues; repeating the step of selecting the plurality of different hues and associating the reselected hues with the plurality of different hues; A method comprising:
15. A charged particle microscope for examining a sample using the method according to any one of claims 1 to 14, comprising: an optical column for focusing a beam of charged particles emitted from the charged particle source onto a sample, the optical column including a charged particle source, a final probe forming lens, and a scanner; a sample stage positioned downstream of the final probe forming lens and configured to hold the sample; and a first detector for detecting a first type of radiation arising from the sample in response to the incidence of charged particles emitted from the charged particle source; a control unit and a processor connected to the first detector; Charged particle microscope, wherein the charged particle microscope is configured to carry out the method according to any one of claims 1 to 14.
Citation Information
Patent Citations
Steel pipe internal corrosion analyzer and steel pipe internal corrosion analysis method
JP2014178328A
Material identification using multiple images
JP2014506996A
Sub-pixel Analysis and Display of Fine Grained Mineral Samples
US20150122992A1