Electron microscope and calibration method

Abstract

Provided is an electron microscope capable of correcting the sensitivity of a plurality of sensor pixels having mutually different sensitivities. An electron microscope according to the present invention comprises: an electron optical system that irradiates a sample with an electron beam and forms an image by means of electrons that have passed through the sample; a camera that includes an image sensor having a plurality of sensor pixels, and that captures a frame image based on output values output from each of the plurality of sensor pixels by incidence of electrons into the image sensor; and a correction coefficient calculation unit that calculates a plurality of correction coefficients for correcting the sensitivity of the plurality of sensor pixels, the correction coefficient calculation unit obtaining the mode of the output value for each sensor pixel from a plurality of frame images captured under the condition that electrons incident on the image sensor follow the Poisson process, and calculating the mode of the output value for each sensor pixel from a plurality of frame images captured under the condition that electrons incident on the image sensor follow the Poisson process. A plurality of correction coefficients are calculated on the basis of the mode obtained for each sensor pixel.

Description

Technical Field

[0001] The present invention relates to an electron microscope and a calibration method. Background Art

[0002] As a camera for a transmission electron microscope, a direct detection camera that directly incident electrons on a sensitive area of ​​an image sensor is known. For example, in Patent Document 1, as a camera for a transmission electron microscope, a direct detection camera having a direct detector (image sensor) is disclosed. In the direct detector disclosed in Patent Document 1, the interaction between the electron beam and the semiconductor layer is suppressed by thinning the semiconductor layer for detecting the electron beam. As a result, the noise caused by the excess signal exceeding the incident electron amount generated by the interaction between the electron beam and the semiconductor layer can be reduced.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Publication No. 2022-002317 Summary of the invention

[0006] Problem that the invention aims to solve

[0007] The direct detector disclosed in Patent Document 1 has a plurality of pixels for forming an image based on an electron beam. In such an image sensor having a plurality of pixels, the sensitivity may sometimes differ for each pixel due to manufacturing errors, etc. Therefore, an electron microscope capable of correcting the sensitivity that differs for each pixel is desired.

[0008] Solutions for solving problems

[0009] One embodiment of the electron microscope of the present invention comprises:

[0010] An electron optical system that irradiates an electron beam to a sample and uses electrons transmitted through the sample to perform imaging;

[0011] A camera including an image sensor having a plurality of sensor pixels, capturing a frame image based on output values ​​outputted from each of the plurality of sensor pixels by incident electrons on the image sensor; and

[0012] a correction coefficient calculation unit that calculates a plurality of correction coefficients for correcting the sensitivity of the plurality of sensor pixels,

[0013] The correction coefficient calculation unit

[0014] The mode of the output values ​​for each of the sensor pixels is obtained from a plurality of the frame images captured under the condition that the electrons incident on the image sensor follow a Poisson process.

[0015] The plurality of correction coefficients are calculated based on the mode found for each of the sensor pixels.

[0016] In such an electron microscope, it is possible to correct the sensitivity of a plurality of sensor pixels having mutually different sensitivities.

[0017] One embodiment of the calibration method of the present invention is a method for calibrating an image sensor in an electron microscope, the electron microscope comprising:

[0018] an electron optical system that irradiates an electron beam to a sample and performs imaging using electrons transmitted through the sample; and

[0019] a camera including the image sensor having a plurality of sensor pixels, and capturing a frame image based on output values ​​outputted from each of the plurality of sensor pixels by incident electrons on the image sensor,

[0020] The calibration method comprises:

[0021] A step of finding the mode of output values ​​for each of the sensor pixels from a plurality of the frame images captured under the condition that the electrons incident on the image sensor follow a Poisson process; and

[0022] A step of calculating a plurality of correction coefficients for correcting the sensitivity of the plurality of sensor pixels based on the mode calculated for each of the sensor pixels.

[0023] In such a calibration method, it is possible to correct the sensitivities of a plurality of sensor pixels having sensitivities different from each other. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 This is a diagram showing the structure of an electron microscope according to one embodiment of the present invention.

[0025] Figure 2 It is a diagram showing an example of the configuration of an image capture processing device.

[0026] Figure 3 This is a flowchart showing an example of the flow of image generation processing by the image processing unit.

[0027] Figure 4 It is a figure for demonstrating the black removal process.

[0028] Figure 5 It is a diagram for explaining the sensitivity correction process.

[0029] Figure 6 is a cross-sectional view schematically showing an image sensor.

[0030] Figure 7 This is a graph showing the amount of signal generated and its frequency when one incident electron enters an arbitrary sensor pixel of the image sensor.

[0031] Figure 8 It is a diagram for explaining counting processing.

[0032] Fig. 9 This is a flowchart showing an example of processing by the correction coefficient calculation unit.

[0033] Fig.10 is a histogram of the output values ​​of any sensor pixel.

[0034] Fig.11 This is a diagram showing an example of a mode table.

[0035] Fig.12 This is a flowchart showing an example of dark-time image update processing performed by the camera control unit.

[0036] Fig.13 : is a flowchart showing a modified example of the dark-time image update processing of the camera control unit.

[0037] Fig.14 : is a flowchart showing a modified example of the dark-time image update processing of the camera control unit.

[0038] Description of Reference Numerals

[0039] 2…Frame image, 2A…Frame image, 2B…Frame image, 2C…Frame image, 2a…Image pixel, 3…Sensor pixel, 4…Dark image, 4A…New dark image, 4B…Existing dark image, 4a…Image pixel, 6…Sensitivity correction table, 6a…Cell, 8…Mode table, 8a…Cell, 10…Electron optical system, 12…Electron source, 14…Illumination system, 16…Imaging system, 18…Phosphor plate, 20…Specimen stage, 22…Specimen holder, 24…Goniometer, 30…Phase machine, 32…image sensor, 40…electronic optical system control unit, 50…image processing device, 100…electron microscope, 140…converging lens, 160…objective lens, 162…intermediate lens, 164…projection lens, 320…support layer, 322…wiring layer, 324…sensitive layer, 325…photodiode, 500…processing unit, 502…image processing unit, 504…correction coefficient calculation unit, 506…camera control unit, 510…operation unit, 520…display unit, 530…storage unit. DETAILED DESCRIPTION

[0040] Hereinafter, preferred embodiments of the present invention will be described in detail using the accompanying drawings. Moreover, the embodiments described below are not intended to unduly limit the contents of the present invention described in the claims. In addition, all the structures described below are not necessarily essential components of the present invention.

[0041] 1. Electron Microscopy

[0042] First, an electron microscope according to an embodiment of the present invention will be described with reference to the drawings. Figure 1 FIG. 1 is a diagram showing a configuration of an electron microscope 100 according to an embodiment of the present invention.

[0043] The electron microscope 100 includes an electron optical system 10 , a sample stage 20 , a camera 30 , an electron optical system control unit 40 , and an image processing device 50 .

[0044] The electron optical system 10 irradiates the sample S with an electron beam and forms an image using the electrons transmitted through the sample S. The electron optical system 10 includes an electron source 12, an irradiation system 14, an imaging system 16, and a fluorescent plate 18 that also functions as a shutter of the electron beam.

[0045] The electron source 12 emits an electron beam. The electron source 12 is, for example, an electron gun that accelerates electrons emitted from a cathode at an anode to emit an electron beam.

[0046] The irradiation system 14 irradiates the electron beam emitted from the electron source 12 to the sample S. For example, the irradiation system 14 irradiates the sample S with a parallel beam. The irradiation system 14 includes a plurality of condensing lenses 140. The condensing lenses 140 focus the electron beam emitted from the electron source 12. Although not shown in the figure, the irradiation system 14 may also include lenses other than the condensing lenses 140, apertures, and the like.

[0047] The imaging system 16 is an optical system for performing imaging using the electron beam transmitted through the sample S. The imaging system 16 includes an objective lens 160 , an intermediate lens 162 , and a projection lens 164 .

[0048] The objective lens 160 is a primary lens for forming a TEM image (specimen image) by using an electron beam transmitted through the sample S. An electron diffraction pattern is formed on the rear focal plane of the objective lens 160. The intermediate lens 162 focuses on the TEM image or electron diffraction pattern formed by the objective lens 160 by changing the focal length. Thus, the TEM image or electron diffraction pattern can be magnified and formed on the object plane of the projection lens 164. The projection lens 164 magnifies the image (TEM image or electron diffraction pattern) formed by the intermediate lens 162 and forms an image on the fluorescent plate 18 and the camera 30.

[0049] Although not shown in the figure, the imaging system 16 may include lenses other than the objective lens 160 , the intermediate lens 162 , and the projection lens 164 , an aperture, and the like.

[0050] The fluorescent plate 18 visualizes the TEM image and the electron diffraction pattern. In the fluorescent plate 18, the fluorescent material is excited by the collision of electrons, and the released visible light forms light and dark corresponding to the intensity of the electrons. The fluorescent plate 18 is arranged in the front stage of the camera 30. The fluorescent plate 18 also serves as a gate for the electron beam. For example, when the camera room where the camera 30 is arranged is closed by the fluorescent plate 18, the electron beam is not incident on the camera 30. When the fluorescent plate 18 is made to pop up and the camera room is opened, the electron beam is incident on the camera 30, and the camera 30 can be used to take TEM images and electron diffraction patterns.

[0051] The sample stage 20 holds the sample S. The sample stage 20 can tilt the sample S. The sample stage 20 includes a sample holder 22 and a goniometer 24. The sample holder 22 holds the sample S. The sample holder 22 is inserted into the goniometer 24. The goniometer 24 can tilt or move the sample S.

[0052] The camera 30 includes an image sensor 32 having a plurality of sensor pixels. The camera 30 is a direct detection camera that directly detects electrons using the image sensor 32. Examples of direct detection cameras include K2 manufactured by Gatan, K3 manufactured by Gatan, Falcon 4 manufactured by Thermo Fisher Scientific, and DE-64 manufactured by Direct Electron.

[0053] Here, as cameras for electron microscopes, indirect detection cameras and direct detection cameras are known. The indirect detection camera uses a scintillator to convert electrons into light, and uses an image sensor to detect the light. In contrast, the direct detection camera does not use a scintillator, but uses an image sensor 32 to directly detect electrons. In the direct detection camera, since the blur caused by the expansion of light in the process of converting electrons into light using a scintillator can be reduced, a high-resolution image can be obtained. Moreover, in the direct detection camera, a high contrast can be obtained with a low irradiation amount. Therefore, the electron microscope 100 equipped with a direct detection camera is suitable for a cryo-electron microscope for observing biological samples in a frozen state, etc.

[0054] The camera 30 captures a frame image. The frame image has a plurality of image pixels corresponding to a plurality of sensor pixels of the image sensor 32 in a one-to-one manner, and the pixel value of each of the plurality of image pixels has a value corresponding to the output value of the corresponding sensor pixel. The output value of the sensor pixel corresponds to the amount of signal generated (amount of charge) by electrons incident on the sensor pixel.

[0055] Furthermore, a plurality of sensor pixels of the image sensor 32 may be collectively regarded as one sensor pixel. That is, the pixel value of an image pixel may be a value corresponding to the output values ​​of a plurality of sensor pixels regarded as one sensor pixel.

[0056] The camera 30 captures the TEM image or electron diffraction pattern formed on the image sensor 32 and outputs it as a frame image. A frame is the minimum capturing unit, and a frame image is an image obtained by capturing one frame. The camera 30 can capture 400 frames per second, for example. That is, the camera 30 can capture 400 frame images per second. In the electron microscope 100, hundreds to thousands of frame images are accumulated to generate one image (TEM image, electron diffraction pattern).

[0057] The image sensor 32 is, for example, a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor, a CCD (Charge Coupled Device) image sensor, etc. By using a CMOS image sensor as the image sensor 32, frame images can be captured at a high frame rate compared to when a CCD image sensor is used. That is, by using a CMOS image sensor as the image sensor 32, the number of frame images that can be captured per unit time can be increased.

[0058] The electron optical system control unit 40 controls the electron optical system 10. By controlling the electron optical system 10 using the electron optical system control unit 40, a TEM image or an electron diffraction pattern can be imaged. The electron optical system control unit 40 includes, for example, a CPU (Central Processing Unit), a processing circuit such as an FPGA (Field Programmable Gate Array), and a storage circuit such as a semiconductor memory. In the electron optical system control unit 40, various control processes are performed by executing a program stored in the storage circuit by a processing circuit such as a CPU.

[0059] The image processing device 50 performs processes such as a process of controlling the camera 30 , an image process for a frame image, a calibration process of the image sensor 32 , and a process of acquiring a dark-time image.

[0060] Figure 2 2 is a diagram showing an example of the configuration of the image capture processing device 50 .

[0061] like Figure 2As shown, the image processing device 50 includes a processing unit 500 , an operation unit 510 , a display unit 520 , and a storage unit 530 .

[0062] The operation unit 510 is used to allow the user to input operation information, and output the input operation information to the processing unit 500. The function of the operation unit 510 can be realized by an input device such as a keyboard, a mouse, a button, a touch panel, or a touch pad.

[0063] The display unit 520 displays an image generated by the processing unit 500. The function of the display unit 520 can be realized by an LCD (Liquid Crystal Display), a touch panel type display, or the like.

[0064] The storage unit 530 stores programs, data, etc. used by the processing unit 500 to perform various calculation processes and control processes. In addition, the storage unit 530 is also used as a work area of ​​the processing unit 500 to temporarily store calculation results, etc. executed by the processing unit 500 according to various programs. The function of the storage unit 530 can be realized by RAM (Random Access Memory), ROM (Read Only Memory), and a hard disk.

[0065] The processing unit 500 performs processing such as controlling the camera 30, image processing for frame images, calibration processing of the image sensor 32, and processing for obtaining dark-time images. The functions of the processing unit 500 can be realized by executing programs stored in the storage unit 530 by various processors such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and a DSP (Digital Signal Processor). The processing unit 500 includes an image processing unit 502, a correction coefficient calculation unit 504, and a camera control unit 506.

[0066] The image processing unit 502 generates one image from a plurality of frame images captured by the camera 30. Here, a TEM image or an electron diffraction pattern image is generated as the image. The image processing unit 502 generates an image by performing black removal processing, sensitivity correction processing for correcting the sensitivity of each sensor pixel of the image sensor 32, counting processing, and accumulation processing on the plurality of frame images. The details of each processing will be described later.

[0067] The correction coefficient calculation unit 504 calculates a plurality of correction coefficients for correcting the sensitivity of a plurality of sensor pixels constituting the image sensor 32. The image processing unit 502 corrects the sensitivity of a plurality of sensor pixels of the image sensor 32 using the plurality of correction coefficients calculated by the correction coefficient calculation unit 504 in the sensitivity correction process.

[0068] The camera control unit 506 controls the camera 30. For example, the camera control unit 506 performs a process of acquiring a dark image used in a black removal process. The dark image is an image captured in a state where no electron beam is incident on the camera 30. The camera control unit 506 acquires control information of the electron optical system 10 from the electron optical system control unit 40, and controls the camera 30 based on the acquired control information to acquire the dark image. For example, the camera control unit 506 determines whether an electron beam is incident on the camera 30 based on the control information of the electron optical system 10, and when it is determined that no electron beam is incident on the camera 30, controls the camera 30 to acquire the dark image.

[0069] 2. Processing of the image processing unit

[0070] Figure 3 : is a flowchart showing an example of the flow of the image generation process by the image processing unit 502 .

[0071] The image processing unit 502 generates one image from a plurality of frame images captured by the camera 30. Figure 3 As shown, the image processing unit 502 performs frame image acquisition processing S10, black removal processing S20, sensitivity correction processing S30, counting processing S40 and accumulation processing S50. Thus, one image can be generated from a plurality of frame images. The following describes the case where an image is generated by taking a TEM image using the camera 30.

[0072] (1) Frame Image Acquisition Process S10

[0073] In the electron optical system 10, the irradiation system 14 irradiates the sample S with an electron beam, and the imaging system 16 forms a TEM image on the image sensor 32 using the electrons transmitted through the sample S. In this state, the camera control unit 506 causes the camera 30 to capture a preset number of frame images. Thus, the camera 30 outputs the set number of frame images. The outputted plurality of frame images are transmitted to the processing unit 500. Here, it is assumed that the processing unit 500 obtains M (M>2) frame images for explanation.

[0074] (2) Black removal process S20

[0075] Next, the image processing unit 502 performs black removal processing S20 on each of the acquired M frame images. The black removal processing S20 is a process of subtracting a dark image from a frame image. By performing the black removal processing S20, regular noise of the image sensor 32 can be removed.

[0076] Figure 4 It is a diagram for explaining the black removal process S20.

[0077] like Figure 4 As shown, the image processing unit 502 subtracts the dark image 4 from the frame image 2. Specifically, the image processing unit 502 subtracts the pixel value of the corresponding image pixel 4a of the dark image 4 from the pixel value of the image pixel 2a of the frame image 2. In this way, the regular noise of the image sensor 32 can be removed from the frame image 2. Through the black removal process S20, M frame images 2A can be obtained from the M frame images 2 after the regular noise of the image sensor 32 is removed. In addition, Figure 4 In the example shown, the frame image 2 has 5×5 image pixels 2a, but the number of pixels of the frame image 2 is not particularly limited. For example, the frame image 2 may have 4000×4000 image pixels 2a.

[0078] (3) Sensitivity calibration process S30

[0079] The image processing unit 502 performs a sensitivity correction process S30 for correcting the sensitivity of the sensor pixels for each of the M frame images 2A after the black removal process.

[0080] Figure 5 It is a diagram for explaining the sensitivity correction process S30.

[0081] The image processing unit 502 uses the correction coefficient to correct the pixel value of each image pixel 2a of the frame image 2A. The storage unit 530 stores a sensitivity correction table 6. Figure 5 As shown in FIG. 1 , the sensitivity correction table 6 is a table storing the correction coefficient calculated for each sensor pixel. In the sensitivity correction table 6 , the coordinates of the sensor pixel and the correction coefficient for correcting the sensitivity of the sensor pixel are registered in association. Figure 5 In the illustrated example, the sensitivity correction table 6 has a plurality of cells 6 a corresponding one-to-one to a plurality of sensor pixels constituting the image sensor 32 , and a correction coefficient is stored in each of the plurality of cells 6 a .

[0082] The image processing unit 502 corrects the pixel value of each image pixel 2a of the frame image 2A using the correction coefficient stored in the corresponding cell 6a. For example, the pixel value of each image pixel 2a of the frame image 2A is divided by the correction coefficient stored in the corresponding cell 6a. Through the sensitivity correction process S30, M frame images 2B after the sensitivity of the sensor pixels is corrected can be obtained from the M frame images 2A.

[0083] (4) Counting Process S40

[0084] Next, the image processing unit 502 performs the counting process S40 for each of the M frame images 2B after the sensitivity of the sensor pixels has been corrected. The reason for performing the counting process S40 will be described below.

[0085] As described above, camera 30 is a direct detection camera that utilizes image sensor 32 to directly detect electrons. Figure 6 is a cross-sectional view schematically showing the image sensor 32 .

[0086] like Figure 6 As shown, the image sensor 32 includes a support layer 320, a wiring layer 322, and a sensitive layer 324. The support layer 320 supports the wiring layer 322 and the sensitive layer 324. The wiring layer 322 is arranged on the sensitive layer 324. The wiring layer 322 includes stacked metal layers constituting a plurality of wirings, a plurality of vias, etc. In addition, the wiring layer 322 includes a stacked insulating layer for insulating the wirings.

[0087] A plurality of photodiodes 325 are formed in the sensitive layer 324 . The photodiodes 325 are separated from each other by a device isolation portion having an STI (shallow trench isolation) structure or a LOCOS (Local Oxidation of Silicon) structure. One photodiode 325 constitutes one sensor pixel 3 .

[0088] The sensitive layer 324 is, for example, a semiconductor layer having a thickness of about 50 μm. When electrons are incident on the photodiode 325, electron-hole pairs are generated due to the photoelectric effect. In the image sensor 32, the electrons (or holes) are converted into voltage and read out as electrical signals. Although not shown in the figure, an amplifier for amplifying the tiny electrical signal generated by the photodiode 325 is formed in each sensor pixel 3. The electrical signal generated by the photodiode 325 is taken out through a plurality of wirings formed in the wiring layer 322. In the image sensor 32, by thinning the sensitive layer 324, the interaction between the electron beam and the semiconductor layer is suppressed, thereby reducing noise.

[0089] Figure 7It is a graph showing the amount of signal generated (charge amount) and its frequency when one incident electron enters any sensor pixel 3 of the image sensor 32 .

[0090] In 1 sensor pixel 3, as Figure 7 As shown in FIG. 3 , the amount of signal generated by the photoelectric effect becomes a Landau distribution. This is because, by thinning the sensitive layer 324, the energy of the incident electrons does not completely disappear in the sensitive layer 324, but passes through the sensitive layer 324. For example, when the material of the sensitive layer 324 is silicon, the thickness of the sensitive layer 324 is 50 μm or less, and the thickness of the wiring layer 322 is 10 μm or less, when electrons with an acceleration voltage of 100 kV or more are incident on the image sensor 32, the amount of signal generated by the photoelectric effect follows the Landau distribution. That is, when the thickness of the sensitive layer 324 is 50 μm or less, the thickness of the wiring layer 322 is 10 μm or less, and the acceleration voltage is 100 kV or more, the output value of the sensor pixel 3 follows the Landau distribution.

[0091] Thus, in the image sensor 32 in which the output signals of the plurality of sensor pixels 3 follow the Landau distribution, even if the frame image 2 is simply accumulated to generate an image, the linearity between the pixel value of each image pixel 2a of the frame image 2 and the number of incident electrons is insufficient.

[0092] Therefore, in order to improve the linearity between the pixel value of each image pixel 2a of the frame image 2 and the number of incident electrons, the image processing unit 502 performs counting processing S40 for each of the M frame images 2B. In the counting processing S40, the frame image 2B is binarized and the incident electrons are counted.

[0093] Figure 8 : is a diagram for explaining the counting process S40. For example, when the threshold is set to 140, when the pixel value of each image pixel 2a of the frame image 2B is greater than 140, the pixel value is set to "1", and when the pixel value is less than 140, the pixel value is set to "0". In this way, a binarized frame image 2C can be obtained. Here, the threshold is a value greater than the noise, and is set to a small value within a range that does not include noise. In this way, the possibility of counting noise as incident electrons or missing incident electrons can be reduced.

[0094] In the frame image 2C, the image pixel 2a corresponding to the sensor pixel 3 into which the electrons are incident is represented by "1", and the image pixel 2a corresponding to the sensor pixel 3 into which the electrons are not incident is represented by "0". In this way, by performing the counting process S40, the number of electrons incident on the image sensor 32 and the position of the sensor pixel 3 into which the electrons are incident can be known in each frame.

[0095] (5) Accumulation Processing S50

[0096] The image processing unit 502 accumulates the binarized M frame images 2C to generate a TEM image. The image processing unit 502 accumulates the pixel values ​​of the corresponding image pixels 2a of the M frame images 2C.

[0097] Through the above processing, a TEM image with high linearity between pixel values ​​and the number of incident electrons can be generated from a plurality of frame images 2. The image processing unit 502 displays the generated TEM image on the display unit 520 and ends the image generation processing.

[0098] 3. Image sensor sensitivity correction

[0099] 3.1. Sensitivity of sensor pixels

[0100] In the image sensor 32, sensitivity varies for each sensor pixel 3. The image sensor 32 includes an amplifier for each sensor pixel 3, and one of the causes of the difference in sensitivity is variation in amplifier characteristics due to manufacturing errors or the like.

[0101] When the sensitivity differs among the sensor pixels 3 in this way, the Landau distribution represented by the output signal of each sensor pixel 3 will have a relative difference between the sensor pixels 3. Such a relative difference in sensitivity will affect the counting process S40.

[0102] For example, when the sensitivity of the target sensor pixel 3 is lower than the average sensitivity of all the sensor pixels 3 constituting the image sensor 32, the amount of signal obtained from the incident electron in the target sensor pixel 3 will be relatively reduced. Therefore, even if the electron is incident on the target sensor pixel 3, it will be determined as being below the threshold in the counting process S40 and will not be counted as an incident electron, that is, there is a possibility of missed counts. In addition, when the sensitivity of the target sensor pixel 3 is higher than the average sensitivity of all the sensor pixels 3 constituting the image sensor 32, the amount of signal due to noise increases, and even if no electron is incident, it will be counted as an incident electron, that is, there is a possibility of miscounting.

[0103] Therefore, by correcting the sensitivities of the plurality of sensor pixels 3 having different sensitivities from each other, the accuracy of the counting process S40 can be improved.

[0104] As a calibration method for correcting sensitivity, for example, in the case of a CMOS image sensor for the purpose of detecting visible light, there is a method in which, after obtaining the output value of each sensor pixel using uniform illumination, a normalization coefficient is calculated for each sensor pixel so that the output value of each sensor pixel becomes uniform between the sensor pixels to obtain a correction coefficient. However, in the image sensor 32, since the signal amount obtained from the incident electron follows the Landau distribution, the correction coefficient cannot be calculated by this method.

[0105] 3.2. Calibration method of sensor pixel sensitivity

[0106] Fig. 9 This is a flowchart showing an example of the processing of the correction coefficient calculation unit 504.

[0107] First, the correction coefficient calculation unit 504 obtains a plurality of frame images 2 captured under the condition that the electrons incident on the image sensor 32 follow a Poisson process ( S100 ).

[0108] In capturing the frame image 2, first, the electron optical system 10 is adjusted in such a manner that the electrons incident on the image sensor 32 follow a Poisson process. Specifically, the electron beam is irradiated onto the vacuum portion where the sample S does not exist, and the electron optical system 10 is adjusted in such a manner as to achieve a parallel irradiation condition in which the electron beam uniformly irradiates the image sensor 32. In this state, the electron optical system 10 is adjusted in such a manner that the average and variance of the number of electrons incident on each sensor pixel 3 per frame are consistent. In addition, the average number of electrons incident on each sensor pixel 3 per frame is made less than 1. The average number of electrons incident on each sensor pixel 3 per frame is preferably less than 0.1, that is, 0.1e - / pixel / frame or less. The number of electrons incident on each sensor pixel 3 per frame can be adjusted by adjusting the electron optical system 10. By irradiating the image sensor 32 with electrons under such conditions, the Poisson process is established.

[0109] It is assumed here that electrons incident on one sensor pixel 3 of the image sensor 32 do not invade adjacent sensor pixels 3 .

[0110] Next, the image sensor 32 is irradiated with electrons under the condition that the electrons incident on the image sensor 32 follow the Poisson process, and a preset number of frame images 2 are captured by the camera 30. For example, several hundred to several thousand frame images 2 are captured. The greater the number of frame images 2, the more statistical fluctuations can be reduced. Here, it is assumed that N (N>2) frame images 2 are obtained for explanation.

[0111] Through the above steps, N frame images 2 can be captured. The N frame images 2 captured by the camera 30 are transmitted to the image capture processing device 50. Thus, the correction coefficient calculation unit 504 can obtain the N frame images 2.

[0112] The above-mentioned process of capturing N frame images 2 is automatically performed by the camera control unit 506 controlling the electron optical system control unit 40 and the camera 30. Alternatively, the user may capture N frame images 2 by operating the electron microscope 100.

[0113] In the above-mentioned process S100 of acquiring N frame images 2, the case where the value of each image pixel 2a of the frame image 2 is a pixel value is described, but the value of each image pixel 2a of the frame image 2 may also be a voltage value. That is, the output value of the sensor pixel 3 may be a pixel value or a voltage value.

[0114] Next, the correction coefficient calculation unit 504 obtains information on the output value for each sensor pixel 3 from the obtained N frame images 2 and creates a histogram of the output value for each sensor pixel 3 ( S102 ).

[0115] The correction coefficient calculation unit 504 extracts the pixel value of the image pixel 2a at the same coordinate from each of the N frame images 2. Thus, N pixel values ​​can be obtained for each image pixel 2a at the same coordinate. The pixel value of the image pixel 2a has a value corresponding to the output value of the corresponding sensor pixel 3. Therefore, N output values ​​can be obtained for each sensor pixel 3 from the N frame images 2. That is, N output values ​​are sampled for each sensor pixel 3.

[0116] The correction coefficient calculation unit 504 creates a histogram for each sensor pixel 3 based on information on the output value acquired for each sensor pixel 3 .

[0117] Fig.10 is a histogram of the output values ​​of any sensor pixel 3. Fig.10 In the histogram of the output values ​​of the sensor pixels 3 shown in FIG. 1 , the horizontal axis represents the output values ​​of the sensor pixels 3 and the vertical axis represents the frequency.

[0118] Next, the correction coefficient calculation unit 504 calculates the mode of the output values ​​from the histogram (S104). The mode of the output values ​​is the position of the peak of the histogram. Fig.10 The mode calculated from the histogram of the output value of an arbitrary sensor pixel 3 shown is 876. The correction coefficient calculation unit 504 obtains the mode for each sensor pixel 3 from the histogram created for each sensor pixel 3 .

[0119] The correction coefficient calculation unit 504 creates a mode number table 8 by associating the obtained mode number with the position of the sensor pixel 3 .

[0120] Fig.11 : is a diagram showing an example of the majority table 8. The majority table 8 has a plurality of cells 8a corresponding to a plurality of sensor pixels 3. The majority found for each sensor pixel 3 is stored in the cell 8a of the coordinate corresponding to the sensor pixel 3 where the majority is found.

[0121] Next, the correction coefficient calculation unit 504 averages the modes obtained for each sensor pixel 3 to calculate an average mode ( S106 ).

[0122] The correction coefficient calculation unit 504 calculates the average mode number by dividing the sum of all the modes stored in the mode number table 8 by the number of sensor pixels 3 (cells 8a). Fig.11 The average mode calculated from the mode table 8 shown is 873.76.

[0123] Next, the correction coefficient calculation unit 504 divides the mode found for each sensor pixel 3 by the average mode to find a plurality of correction coefficients ( S108 ).

[0124] The majority stored in each cell 8a of the majority table 8 is divided by the average majority. Figure 5 The sensitivity correction table 6 shown in FIG. 6 has a plurality of cells 6a corresponding one to one to a plurality of sensor pixels 3 constituting the image sensor 32. A correction coefficient for correcting an output value outputted from the corresponding sensor pixel 3 is stored in each cell 6a.

[0125] After the correction coefficient calculation unit 504 creates the sensitivity correction table 6, the processing ends. Figure 3 In the sensitivity correction process S30 shown, the sensitivity of each sensor pixel 3 is corrected using the sensitivity correction table 6. Thus, the sensitivity of each sensor pixel 3 can be calibrated.

[0126] 4. Obtaining dark images

[0127] 4.1. Dark images

[0128] In the black removal process S20, a dark image 4 is used. The dark image 4 is an image captured when no electron beam is incident on the camera 30. Here, the output value of the sensor pixel 3 is offset. For example, when the offset is 60, the output value of the sensor pixel 3 when no electron beam is incident on the camera 30 becomes 60. In the camera 30, since the image sensor 32 directly detects electrons, the image sensor 32 is damaged by the electron beam. When the noise increases due to the damage, the output value of the image sensor 32 also increases by the corresponding amount of the noise. For example, when the offset is 60 and the output value after the increase due to the noise is 50, the output value of the sensor pixel 3 when no electron beam is incident on the camera 30 becomes 110.

[0129] In this way, since the output value of the sensor pixel 3 changes due to the electron beam damage, it is desirable that the dark image 4 used in the black removal process S20 is updated. In the electron microscope 100, the dark image 4 can be automatically updated.

[0130] Specifically, in the electron microscope 100, the camera control unit 506 obtains the control information of the electron optical system 10 from the electron optical system control unit 40, and controls the camera 30 based on the obtained control information to obtain the dark image 4. The camera control unit 506 determines whether electrons are not incident on the camera 30 based on the control information of the electron optical system 10, and controls the camera 30 to obtain the dark image 4 when it is determined that electrons are not incident on the camera 30.

[0131] In addition, when the damage of the image sensor 32 caused by the electron beam increases and the noise increases, the accuracy of the counting process S40 decreases. Therefore, in the electron microscope 100, the camera control unit 506 determines the degree of damage of the image sensor 32 based on the dark image 4, and notifies the user to replace the image sensor 32 according to the determination result.

[0132] 4.2. Image update processing in dark time

[0133] Fig.12 1 is a flowchart showing an example of dark-time image update processing by the camera control unit 506 .

[0134] First, the camera control unit 506 obtains control information of the electron optical system 10 from the electron optical system control unit 40 (S200). The control information of the electron optical system 10 includes information on the status of each unit constituting the electron optical system 10. The electron optical system control unit 40 and the image processing device 50 are communicatively connected, for example, via a serial cable or the like. The electron optical system control unit 40 transmits the control information of the electron optical system 10 to the image processing device 50, for example, in accordance with a request of the camera control unit 506.

[0135] The camera control unit 506 determines whether electrons are incident on the camera 30 based on the control information of the electron optical system 10 (S202). The camera control unit 506 determines that electrons are not incident on the camera 30 when, for example, the electron source 12 is not releasing an electron beam, the fluorescent plate 18 is in a closed state, or a blanking device of the electron source 12 (not shown) blocks the electron beam. These states can be known from the control information of the electron optical system 10.

[0136] When the camera control unit 506 determines that electrons are incident on the camera 30 (Yes in S202), the camera control unit 506 returns to the process S200 to obtain the control information of the electron optical system 10 (S200). The camera control unit 506 repeats the process S200 of obtaining the control information of the electron optical system 10 and the determination process S202 until it is determined that electrons are not incident on the camera 30.

[0137] When it is determined that no electrons are incident on the camera 30 (No in S202), the camera control unit 506 controls the camera 30 to obtain a dark image 4 (S204). The camera control unit 506 causes the camera 30 to shoot a frame image 2 when no electrons are incident on the camera 30. Thus, the dark image 4 can be obtained. The dark image 4 may be one frame image 2 shot when no electrons are incident on the camera 30, or may be an image obtained by averaging a plurality of frame images 2 shot when no electrons are incident on the camera 30.

[0138] The camera control unit 506 stores the acquired dark image 4 in the storage unit 530 as the dark image 4 for the black removal process S20 ( S206 ).

[0139] The camera control unit 506 determines whether a specified time has passed since the dark time image 4 was acquired (S208). Here, the specified time is the time from acquiring the dark time image 4 to acquiring the next dark time image 4. The specified time is preset. The specified time can be, for example, 24 hours or 168 hours. The specified time can also be set arbitrarily by the user. The camera control unit 506 waits from acquiring the dark time image 4 in processing S204 until the specified time has passed ("No" in S208).

[0140] When the camera control unit 506 determines that the predetermined time has passed (Yes in S208), it obtains control information of the electron optical system 10 from the electron optical system control unit 40 (S210). Then, the camera control unit 506 determines whether electrons are not incident on the camera 30 based on the control information of the electron optical system 10 (S212).

[0141] When the camera control unit 506 determines that electrons are incident on the camera 30 ("Yes" in S212), it returns to the process S210 to obtain the control information of the electron optical system 10 (S210). Until it is determined that electrons are not incident on the camera 30, the camera control unit 506 repeats the process S210 of obtaining the control information of the electron optical system 10 and the determination process S212.

[0142] When determining that no electrons are incident on the camera 30 (No in S212 ), the camera control unit 506 controls the camera 30 to acquire the dark-time image 4 ( S214 ).

[0143] The process S210 of acquiring the control information of the electron optical system 10, the process S212 of determining whether electrons are incident, and the process S214 of acquiring the dark image 4 are performed in the same manner as the processes S200, S202, and S204, respectively. In addition, hereinafter, the dark image 4 newly acquired in the process S214 is referred to as a new dark image 4A, and the dark image 4 stored in the storage unit 530 as the dark image 4 for the black removal process S20 is referred to as an existing dark image 4B.

[0144] Next, the camera control unit 506 compares the new dark image 4A with the existing dark image 4B to determine whether the pixel values ​​of the plurality of image pixels 4 a of the new dark image 4A have increased due to electron beam damage ( S216 ).

[0145] For example, the camera control unit 506 compares the average value (average pixel value) of the pixel values ​​of the plurality of image pixels 4a of the new dark image 4A with the average value (average pixel value) of the pixel values ​​of the plurality of image pixels 4a of the existing dark image 4B. When the average pixel value of the new dark image 4A is greater than the average pixel value of the existing dark image 4B, the camera control unit 506 determines that the pixel value has increased due to electron beam damage.

[0146] When the camera control unit 506 determines that the pixel value has increased due to electron beam damage ("Yes" in S216), it stores the new dark image 4A as the dark image 4 for the black removal process S20 in the storage unit 530 (S222). At this time, the existing dark image 4B originally stored in the storage unit 530 is deleted. In this way, the dark image 4 for the black removal process S20 is updated.

[0147] Next, the camera control unit 506 determines whether the average pixel value of the dark image 4 stored in the storage unit 530 is greater than a threshold value (S220). The threshold value may be preset or may be set to an arbitrary value by the user.

[0148] When the camera control section 506 determines that the average pixel value of the dark-time image 4 is larger than the threshold value (Yes in S220 ), it notifies the user to replace the image sensor 32 ( S222 ).

[0149] The camera control section 506 displays a message prompting replacement of the image sensor 32 on the display section 520, for example. The notification method is not limited to notification by message, and notification may be performed using sound, vibration, light, or the like.

[0150] The camera control unit 506 determines whether the user has issued an instruction to end the dark-time image update process (S224) when it is determined that the pixel value has not increased due to electron beam damage (No in S216), when it is determined that the average pixel value of the dark-time image 4 is not greater than the threshold value (No in S220), and after the notification process S222. For example, when the user inputs an instruction to end the dark-time image update process via the operation unit 510, the camera control unit 506 determines that the user has issued an end instruction.

[0151] When the camera control unit 506 determines that the user has not issued an end instruction (No in S224), it returns to process S208 to determine whether a predetermined time has passed since the dark-time image 4 was acquired (S208). The camera control unit 506 waits until a predetermined time has passed since the dark-time image 4 was acquired in process S214 (No in S208).

[0152] In this way, the camera control unit 506 repeatedly performs processing S208, processing S210, processing S212, processing S214, processing S216, processing S218, processing S220, processing S222, and processing S224 until it is determined that the user has issued an end instruction.

[0153] When the camera control section 506 determines that the user has issued an end instruction (Yes in S224 ), it ends the dark-time image update process.

[0154] 5. Effect

[0155] The electron microscope 100 includes: an electron optical system 10 that irradiates an electron beam to a sample S and forms an image using electrons transmitted through the sample S; a camera 30 that includes an image sensor 32 having a plurality of sensor pixels 3 and captures a frame image 2 based on output values ​​outputted from each of the plurality of sensor pixels 3 by electrons incident on the image sensor 32; and a correction coefficient calculation unit 504 that calculates a plurality of correction coefficients for correcting the sensitivity of the plurality of sensor pixels 3. The correction coefficient calculation unit 504 obtains the mode of the output values ​​for each sensor pixel 3 from the plurality of frame images 2 captured under the condition that the electrons incident on the image sensor 32 follow a Poisson process, and calculates a plurality of correction coefficients based on the mode obtained for each sensor pixel 3.

[0156] Therefore, in the electron microscope 100 , the sensitivity of the plurality of sensor pixels 3 having different sensitivities can be corrected, thereby improving the accuracy of the counting process S40 .

[0157] In the electron microscope 100, the correction coefficient calculation unit 504 averages the modes obtained for each sensor pixel 3 to calculate an average mode, and divides the mode by the average mode to calculate a plurality of correction coefficients for each sensor pixel 3. Therefore, in the electron microscope 100, even in the case of an image sensor 32 in which output values ​​outputted from each of the plurality of sensor pixels 3 follow a Landau distribution when electrons are incident on the image sensor 32, a plurality of correction coefficients for correcting the sensitivity of the plurality of sensor pixels 3 having different sensitivities can be calculated.

[0158] In the electron microscope 100, the camera 30 is a direct detection camera that directly detects electrons using the image sensor 32. Therefore, in the electron microscope 100, a high-resolution TEM image can be obtained.

[0159] The electron microscope 100 includes an electron optical system control unit 40 for controlling the electron optical system 10 and a camera control unit 506 for controlling the camera 30. The camera control unit 506 obtains control information of the electron optical system 10 from the electron optical system control unit 40, and controls the camera 30 based on the obtained control information to obtain the dark image 4. Therefore, the electron microscope 100 can automatically obtain the dark image 4.

[0160] In the electron microscope 100, the camera control unit 506 determines whether electrons are incident on the camera 30 based on the control information of the electron optical system 10, and controls the camera 30 to obtain the dark image 4 when it is determined that electrons are not incident on the camera 30. Therefore, in the electron microscope 100, the dark image 4 can be automatically obtained.

[0161] The calibration method of the image sensor 32 in the electron microscope 100 includes: a step of finding the mode of output values ​​for each sensor pixel 3 from a plurality of frame images 2 captured under the condition that the electrons incident on the image sensor 32 follow a Poisson process; and a step of calculating a plurality of correction coefficients for correcting the sensitivity of the plurality of sensor pixels 3 based on the mode found for each sensor pixel 3. Therefore, the sensitivity of the plurality of sensor pixels 3 having different sensitivities can be corrected. Therefore, the accuracy of the counting process S40 can be improved.

[0162] 6. Modifications

[0163] 6.1. First variant

[0164] In the above-mentioned embodiment, the electrons incident on one sensor pixel 3 of the image sensor 32 are described as not invading the adjacent sensor pixels 3. However, the electrons incident on one sensor pixel 3 may invading other adjacent sensor pixels 3. In this case, for example, Fig. 9 In the process S100 of acquiring the frame image 2 shown, the number of sampling of the output value is increased by increasing the number of acquired frame images 2. This can reduce the influence of the electrons incident on one sensor pixel 3 invading other adjacent sensor pixels 3.

[0165] 6.2. Second Modification

[0166] In the above-described embodiment, the description is given assuming that electrons incident on one sensor pixel 3 of the image sensor 32 do not invade adjacent sensor pixels 3 . However, electrons incident on one sensor pixel 3 may invade other adjacent sensor pixels 3 .

[0167] In this case, the number of electrons incident on the image sensor 32 does not match the number of electrons determined in the frame image 2. Therefore, a process is performed to make the number of electrons incident on the image sensor 32 consistent with the number of electrons determined in the frame image 2. For example, a focused image pixel 2a and a plurality of image pixels 2a adjacent to the image pixel 2a are grouped as one group, the centroid position is calculated from the pixel values ​​of each image pixel 2a, and the image pixel 2a overlapping the centroid position is set as the image pixel 2a corresponding to the sensor pixel 3 into which the electrons are incident. The sensor pixel 3 into which the electrons are incident may be determined by a grouping method in which a plurality of image pixels 2a are grouped as described above.

[0168] 6.3. Third variant

[0169] Fig.13 : is a flowchart showing a modified example of the dark-time image update processing of the camera control section 506 .

[0170] In the above Fig.12 In the dark image update process shown in FIG. 1 , the camera control unit 506 compares the new dark image 4A with the existing dark image 4B to determine whether the pixel values ​​of the plurality of image pixels 4a of the new dark image 4A have increased due to electron beam damage (S216). Fig.12 In the dark image updating process shown, when the pixel value increases due to electron beam damage, the dark image 4 is updated.

[0171] In contrast, Fig.13 As shown, the determination process S216 may not be performed. That is, after the new dark image 4A is obtained in the process S214, the new dark image 4A is always stored in the storage unit 530 as the dark image 4 for the black removal process S20, and the existing dark image 4B is deleted from the storage unit 530. In this way, in the dark image update process, the latest dark image 4 may always be used as the dark image 4 for the black removal process S20.

[0172] 6.4. Fourth variant

[0173] Fig.14 : is a flowchart showing a modified example of the dark-time image update processing of the camera control section 506 .

[0174] In the above Fig.12 In the dark-time image update process shown in FIG. 4 , the camera control unit 506 performs a process S208 of determining whether a predetermined time has passed since the dark-time image 4 was acquired. Fig.12 In the dark-time image update process shown, when it is determined that the predetermined time has passed (Yes in S208 ) and it is determined that electrons have not entered the camera 30 (No in S212 ), the dark-time image 4 is acquired ( S214 ).

[0175] In contrast, Fig.14 In the dark-time image update process shown in FIG. 1 , the camera control unit 506 performs a process S209 of determining whether the user has issued an instruction to start acquiring the dark-time image 4. Fig.13 In the dark image update process shown, when it is determined that the user has issued an instruction to start acquisition of the dark image 4 (Yes in S209 ) and it is determined that electrons are not incident on the camera 30 (No in S212 ), the dark image 4 is acquired ( S214 ).

[0176] For example, when the user inputs an instruction to start acquiring the dark-time image 4 via the operation unit 510 , the camera control unit 506 determines that the user has issued an instruction to start acquiring the dark-time image 4 .

[0177] The above-mentioned embodiments and modifications are merely examples, and the present invention is not limited to these examples. For example, each embodiment and each modification can be combined as appropriate.

[0178] The present invention is not limited to the above-mentioned embodiments, and various modifications can be further made. For example, the present invention includes a configuration that is substantially the same as the configuration described in the embodiment. The so-called substantially the same configuration refers to, for example, a configuration with the same function, method and result, or a configuration with the same purpose and effect. In addition, the present invention includes a configuration in which a non-essential part of the configuration described in the embodiment is replaced. In addition, the present invention includes a configuration that has the same effect as the configuration described in the embodiment, or a configuration that can achieve the same purpose. In addition, the present invention includes a configuration in which a known technology is added to the configuration described in the embodiment.

Claims

1. An electron microscope, characterized in that Include: An electron optical system that irradiates an electron beam to a sample and uses electrons transmitted through the sample to perform imaging; A camera including an image sensor having a plurality of sensor pixels, capturing a frame image based on output values ​​outputted from each of the plurality of sensor pixels by incident electrons on the image sensor; and a correction coefficient calculation unit that calculates a plurality of correction coefficients for correcting the sensitivity of the plurality of sensor pixels, The correction coefficient calculation unit The mode of the output values ​​for each of the sensor pixels is obtained from a plurality of the frame images captured under the condition that the electrons incident on the image sensor follow a Poisson process. The plurality of correction coefficients are calculated based on the mode found for each of the sensor pixels.

2. The electron microscope according to claim 1, wherein: The correction coefficient calculation unit averaging the modes obtained for each of the sensor pixels to calculate an average mode, The plurality of correction coefficients are calculated by dividing the mode number by the average mode number for each of the sensor pixels.

3. The electron microscope according to claim 1, wherein: The image sensor is a sensor in which output values ​​output from each of the plurality of sensor pixels by incident electrons on the image sensor follow Landau distribution.

4. The electron microscope according to claim 3, wherein: The camera is a direct detection camera that utilizes the image sensor to directly detect electrons.

5. The electron microscope according to any one of claims 1 to 4, wherein: Include: an electron optical system control section that controls the electron optical system; and a camera control unit that controls the camera, The camera control unit obtain control information of the electro-optical system from the electro-optical system control unit, The camera is controlled based on the acquired control information to acquire a dark-time image.

6. The electron microscope according to claim 5, wherein: The camera control unit determining whether electrons are incident on the camera based on the control information, When it is determined that no electrons are incident on the camera, the camera is controlled to acquire the dark-time image.

7. A calibration method is a calibration method for an image sensor in an electron microscope, the electron microscope comprising: an electron optical system that irradiates an electron beam to a sample and performs imaging using electrons transmitted through the sample; and a camera including the image sensor having a plurality of sensor pixels, and capturing a frame image based on output values ​​outputted from each of the plurality of sensor pixels by incident electrons on the image sensor, The calibration method is characterized by comprising: A step of finding the mode of output values ​​for each of the sensor pixels from a plurality of the frame images captured under the condition that the electrons incident on the image sensor follow a Poisson process; and A step of calculating a plurality of correction coefficients for correcting the sensitivity of the plurality of sensor pixels based on the mode calculated for each of the sensor pixels.

Images