Method and system for high speed signal processing

Abstract

Methods and systems for high speed signal processing. A method and system for acquiring data from a pixelated image sensor for detecting charged particles. The method includes reading a pixel voltage of one or more of a plurality of pixels multiple times without resetting the image sensor and digitizing the pixels to a first number of bits. The camera outputs digitized compressed pixel voltages with a second, lesser number of bits. A maximum range of the digitized compressed pixel voltages is less than a maximum range of the pixel voltages.

Description

Technical Field

[0001] This description generally relates to methods and systems for data acquisition using cameras, and more specifically, to real-time signal processing of high-speed camera readout and readout data. Background Technology

[0002] A charged particle microscopy system may include a camera for detecting charged particles emitted from a sample, digitizing the detected raw signal, and outputting the digitized signal to an image processor for real-time signal processing. To increase data acquisition speed, the microscopy system requires short sensor response times, high-speed sensor data readout, and real-time data processing and storage. For cameras with pixelated image sensors, pixels may need to be reset when pixel voltage exceeds a predetermined level. One method to improve the readout speed of such pixelated image sensors is multi-frame correlated double sampling (mfCDS), disclosed in U.S. Application No. 13 / 645,725, filed October 5, 2012, entitled “Method for Acquiring Data Using an Image Sensor,” by Janssen et al. In mfCDS, multiple frames of raw data are read from the image sensor before resetting the image sensor or a specific pixel of the image sensor. The particle count can then be determined based on the differences in the sequentially acquired pixel voltages. However, the applicants recognized that the limited bandwidth within the camera and / or between the camera and the image processor can become a bottleneck for high-speed signal processing. Summary of the Invention

[0003] In one embodiment, a method for acquiring data from a camera comprising a pixelated image sensor for detecting charged particles includes: repeatedly reading pixel voltages of one or more pixels of the image sensor without resetting the image sensor; digitizing the pixel voltages into a first number of bits; and outputting the digitized compressed pixel voltages with a second lower number of bits, wherein the maximum range of the digitized compressed pixel voltages is less than the maximum range of pixel voltages, and wherein the digitized compressed pixel voltages are generated by removing at least the most significant bit (MSB) of the digitized pixel voltages. In this way, the pixel voltage at each pixel of the image sensor can be read out and transmitted from the camera to the image processor with a lower number of bits without sacrificing data quality or accuracy. High-speed signal readout and processing can be achieved using the limited bandwidth within the camera and / or between the camera and the image processor.

[0004] It should be understood that the above overview is provided to introduce, in a simplified form, some concepts further described in the detailed description. It is not intended to identify key or essential features of the claimed subject matter, the scope of which is uniquely defined by the claims following the detailed description. Furthermore, the claimed subject matter is not limited to embodiments that address any shortcomings mentioned above or in any part of this disclosure. Attached Figure Description

[0005] Figure 1 A charged particle microscope is shown.

[0006] Figure 2 It is used from Figure 1 A flowchart illustrating the method for acquiring data using a camera in a charged particle microscope.

[0007] Figure 3 Show Figure 1 The data flow of the method in the middle.

[0008] Figure 4A A method for compressing pixel voltages read from an image sensor is shown.

[0009] Figure 4B An example of implementing pixel voltage compression is shown.

[0010] Figure 4C Another example of implementing pixel voltage compression is shown.

[0011] Figure 5 This is a flowchart of a method for detecting overexposure of a sensor.

[0012] Figure 6A and Figure 6B This illustrates a method for adjusting pixel values ​​of differentially compressed frames to an effective range.

[0013] Throughout the various views in the accompanying drawings, the same reference numerals refer to the corresponding parts. Detailed Implementation

[0014] The following description relates to systems and methods for data acquisition and data processing in microscope systems, such as Figure 1 Charged particle microscopy. Charged particle microscopy can contain sources for generating charged particles. In response to irradiating the sample with charged particles, various types of charged particles emitted from the sample are detected by different cameras or detectors.

[0015] A camera or detector may include a pixelated image sensor for converting charged particles impacting pixels into pixel voltages. Pixel voltages can be read from the image sensor using the mfCDS method. Specifically, the pixel voltage of a particular pixel is read multiple times before being reset to a reset value. The number of charged particles detected by a pixel can be determined based on the difference between successive readouts of the pixel voltage. Using mfCDS, multiple frames can be read before resetting the image sensor. Reset noise is reduced by calculating the difference between successive readouts. Furthermore, the maximum frame rate of image sensor readout can be increased by reducing the frequency of resetting pixel voltages.

[0016] Pixel voltages read from the image sensor can be digitized by an analog-to-digital converter (ADC) and transmitted from the camera to the image processor for further processing. While high frame rate readout from the image sensor can be achieved using the mfCDS method, the data transfer rate (i.e., bandwidth) between the ADC and the readout electronics within the camera and / or between the camera and the image processor can limit the overall data acquisition rate and data quality of the microscope system. To increase the frame rate of the data received at the image processor, the bit depth (i.e., the number of bits) of the digitized pixel voltage per pixel must be reduced. However, reducing the bit depth may reduce data accuracy. Here, the accuracy of the digital signal is the value represented by the least significant bit (LSB) of the digitized signal.

[0017] To solve the above problems, Figure 2 This paper proposes a method for high-speed camera readout and real-time processing of data received from the camera. The data stream is in... Figure 3 As shown in the diagram. Specifically, the pixel voltages of the image sensor are read out multiple times before the sensor is reset. The pixel voltage readouts are compressed before being transmitted from the camera to the image processor. The image processor receives the compressed pixel voltages from the camera and generates differential voltages between the sequentially received compressed pixel voltages. Sample images can be formed based on the differential voltages. Between adjacent image sensor resets, the pixel voltages are approximately monotonic. That is, between sensor resets, the pixel voltage of each pixel is a monotonic signal superimposed with noise. The noise amplitude is within 1% of the maximum amplitude of the pixel voltage. The sensor is reset before the pixel voltage amplitude exceeds the maximum pixel voltage amplitude. Due to the approximately monotonic change in pixel voltage between sensor resets, and the pixel voltage change between sequential (or adjacent) image sensor readouts within a threshold level, no information is lost even as the compressed pixel voltages are transmitted between the camera and the image processor.

[0018] The maximum range of compressed pixel voltage is less than the maximum range of pixel voltage. The pixel voltage amplitude can be from zero to the maximum pixel voltage amplitude. Pixel voltage compression can be achieved by subtracting the first threshold voltage from the pixel voltage in response to a pixel voltage amplitude that is not less than the amplitude of a first threshold voltage and less than the amplitude of a second threshold voltage. The first threshold voltage can be determined based on the change in pixel voltage between adjacent pixel readouts. For example, the amplitude of the first threshold voltage is greater than the change in pixel value between adjacent pixel readouts. To further reduce the maximum range of compressed pixel voltage, pixel voltage compression can be achieved by subtracting the second threshold voltage from the pixel voltage in response to a pixel voltage amplitude that is not less than the amplitude of the second threshold voltage and less than the amplitude of the third threshold voltage, and by subtracting the third threshold voltage from the pixel voltage in response to a pixel voltage amplitude that is not less than the amplitude of the third threshold voltage and less than the amplitude of the fourth threshold voltage. The first to fourth threshold voltages can be analog voltages in volts. In one example, the pixel voltage of the image sensor increases in response to charged particles impacting the pixel, and the threshold voltage is positive. In another example, the pixel voltage of the image sensor decreases in response to charged particles impacting the pixel, and the pixel voltage is converted into an approximately monotonically increasing positive voltage before subtracting a positive threshold voltage. In yet another example, the pixel voltage of the image sensor decreases in response to charged particles impacting the pixel, and the threshold voltage is negative. The compressed pixel voltage can be digitized with fewer bits than the pixel voltage read from the image sensor. In this way, the pixel voltage is compressed or wrapped into a reduced range compared to the range of the uncompressed pixel voltage. The range of the compressed voltage is no greater than the amplitude of the first threshold voltage.

[0019] In another example, the pixel voltage read from the image sensor is digitized into a first number of bits. The digitized pixel voltage is compressed into a digitized compressed pixel voltage with a second lower number of bits. The digitized pixel voltage and the digitized compressed pixel voltage have the same precision. The digitized pixel voltage can be unsigned. In one example, if the pixel voltage decreases approximately monotonically and is negative, the digitized pixel voltage can be converted to unsigned. The compression process of subtracting a threshold voltage from the pixel voltage can be implemented by removing one or more bits from the digitized pixel voltage. In one example, the digitized pixel voltage can be compressed by removing at least the MSB of the digitized pixel voltage. In another example, the digitized pixel voltage can be compressed by saving the first bit of the digitized first pixel voltage to the second bit and removing the remaining bits, where neither the first nor the second bit is an MSB. In one example, neither the first nor the second bit is a Least Significant Bit (LSB). Figures 4A-4C Example methods for compressing pixel values ​​are shown.

[0020] The camera continuously and repeatedly reads out pixel voltages from each pixel in a region of the image sensor (i.e., frames of pixel voltages) and sends compressed pixel voltages or digitized compressed pixel voltages (i.e., compressed frames) to the image processor. Differential frames are reconstructed based on the differences in the sequentially received compressed frames. Sample images can then be generated based on the differential frames. In one example, for each pixel, the differential compressed pixel voltage is the difference between a first compressed pixel voltage and a second compressed pixel voltage. The first compressed pixel voltage corresponds to the pixel voltage read out at a first time point, and the second compressed pixel voltage corresponds to the pixel voltage read out at a second time point immediately following the first time point. There is no pixel or image sensor reset between the first and second time points. Figures 6A-6B As shown, the differential pixel voltage is reconstructed by adjusting the differentially compressed pixel voltage to an effective range. The effective range is determined based on a first threshold voltage used for compressing the pixel voltage and a predetermined noise amplitude. For example, the effective range is from the noise offset to the sum of the first threshold voltage amplitude and the noise offset. The noise offset is determined based on the noise amplitude and can be negative or zero. Adjusting the differentially compressed pixel voltage to the effective range includes increasing the first threshold voltage amplitude to the differentially compressed pixel voltage in response to the differentially compressed pixel voltage being lower than the noise offset, and subtracting the first threshold voltage amplitude from the differentially compressed pixel voltage in response to the differentially compressed pixel voltage being greater than the sum of the first threshold voltage amplitude and the noise offset. The accuracy of the differential pixel voltage is the same as that of the digitized compressed pixel voltage.

[0021] In one instance, before reconstructing the differential frames, dark frames can be subtracted from the compressed frames to perform dark correction on the compressed frames. The dark correction process can be used to remove fixed patterns present in images from sensors that have never been exposed to radiation. Furthermore, sensor overexposure can be detected based on the digitized compressed pixel voltages received by the image processor, such as... Figure 5 As shown.

[0022] In this way, pixel voltages can be transmitted between the camera and the image processor with a reduced number of bits. Due to the characteristics of pixel voltages—approximate monotonicity and finite variation over time—the differences between sequential pixel voltage readouts can be reconstructed losslessly at the image processor, even though the dynamic range of the signal transmitted between the camera and the image processor is reduced.

[0023] Go to Figure 1A transmission-type charged particle microscope 100, such as a transmission electron microscope (TEM) system or a scanning transmission electron microscope (STEM) system, is shown. The microscope includes a vacuum housing 2 and a charged particle source 4 for generating a charged particle beam 111 that propagates along a main axis 110 and passes through an electron optical illuminator 6. The electron optical illuminator 6 is used to guide / focus the charged particles onto a selected portion of a sample 60 (e.g., which may be (locally) thinned / planarized). A deflector 8 is also depicted, which can be used to achieve scanning motion of the beam 111.

[0024] The sample 60 is held in place on a sample holder 61, which can be positioned in multiple degrees of freedom by a positioning device / platform 62 that moves the holder 61 (removably) attached to a bracket 63; for example, the sample holder 61 may include fingers that can (in particular) move in the XY plane (see the depicted Cartesian coordinate system; generally, movement parallel to Z and tilting about X / Y are also possible). Such movement allows different portions of the sample 60 to be illuminated / imaged / examined by an electron beam 111 traveling (in the Z direction) along the principal axis 110 (and / or allows scanning motion to be performed as an alternative to beam scanning). Optional cooling devices (not depicted) may be introduced to make thermal contact with the sample holder 61, in order to maintain it (and the sample 60 thereon) at, for example, a low temperature.

[0025] Electron beam 111 will interact with sample 60 in a manner that causes various types of "stimulated" radiation to be emitted from sample 60, including, for example, secondary electrons, backscattered electrons, X-rays, and optical radiation (catholuminescence). If desired, one or more of these radiation types can be detected by means of detector 22, which may be, for example, a combination of scintillator / photomultiplier tubes or EDX (energy-dispersive X-ray spectroscopy) modules; in this case, the image can be constructed using essentially the same principles as in scanning electron microscopy (SEM). However, alternatively or additionally, electrons that pass through sample 60, exit from it, and continue to propagate along axis 110 (essentially, although typically with some deflection / scattering) can be studied. This transmitted electron flux enters projection lens 24, which will typically include various electrostatic / magnetic lenses, deflectors, correctors (e.g., compensators), etc. In normal (non-scanning) TEM mode, projection lens 24 can focus the transmitted electron flux onto detector 26, which can be retracted / retracted (as schematically indicated by arrow 27) away from axis 110 if needed. An image (or diffraction pattern) of sample 60 (a portion thereof) will be formed on detector (e.g., screen) 26 by projection lens 24, and this can be observed through an observation port located on an appropriate portion of the wall of housing 2. The retraction mechanism for detector 26 may be, for example, mechanical and / or electrical, and is not depicted here.

[0026] As an alternative to viewing the image on detector 26, the fact that the focusing depth of the electron flux leaving projection lens 24 is typically quite large (e.g., on the order of 1 meter) can be used instead. Therefore, various other types of analytical devices, such as TEM camera 30, STEM camera 32, and spectroscopic device 34, can be used downstream of detector 26.

[0027] At the TEM camera 30, the electron flux can form a still image (or diffraction pattern) that can be processed by the image processor 20 and the controller 50. When not needed, the camera 30 can be retracted / withdrawn (as schematically shown by arrow 31) to remove it from axis 110.

[0028] The output from STEM camera 32 can be recorded as a function of the (X, Y) scan position of beam 111 on sample 60, and an image can be constructed that is a “mapping” of the output from camera 32 as a function of X, Y. Camera 32 may include a pixel matrix. When not needed, camera 32 can be retracted / retracted (as schematically indicated by arrow 33) away from axis 110 (but in the case of camera 32, for example, a ring-shaped dark-field camera, such retraction is not necessary; in such cameras, the central aperture will allow flux through when the camera is not in use).

[0029] As an alternative to imaging using cameras 30 and / or 32, a spectroscopic device 34 may also be invoked, which may be, for example, an EELS module. The EELS module includes a spectrometer 35 for dispersing charged particles based on particle energy and a detector / camera 36 for capturing the spectrum.

[0030] It should be noted that the order / position of detectors 26, 30, 32, 34, and 36 is not strict and many possible variations can be envisioned. For example, the spectroscopic device 34 could also be integrated into the projection lens 24.

[0031] The controller 50 is connected via control lines to various components shown. The controller includes a processor 54 and a non-transitory memory 55. Instructions can be stored in the non-transitory memory 55, which, when executed, enable the controller 50 to perform various functions, such as synchronizing actions, providing setpoints, processing signals, performing calculations, receiving operator input from the user input device 53, and displaying messages / information on the display device 51. The controller 50 may be (partially) inside or outside the housing 2 and may have an integral or composite structure as needed.

[0032] One or more detectors 22 and 26, cameras 30 and 32, and a spectroscopic device 34 may be electrically connected to the image processor 20. The image processor may include a processor, memory, and one or more field-programmable gate arrays (FPGAs). Embedded software may run in the image processor to process image data received from the cameras and / or detectors at high frame rates. Processed data from the image processor may be transferred from the image processor to a controller for further processing. For example, the controller generates sample images based on data received from the image processor. The cameras and / or detectors may have separate image processors or share an image processor. In one embodiment, the image processor and controller may be integrated together as a single component. In another embodiment, the image processor may be integrated with the camera.

[0033] While a transmission electron microscope is described with the aid of examples, it should be understood that the imaging system can be other types of charged particle microscopy systems, such as SEM or focused ion beam (FIB-SEM) combined with scanning electron microscopy. The charged particles can be electrons, ions, or X-rays. One or more of the detectors or cameras, such as detectors 22 and 36, and cameras 30 and 32, can comprise one or more image sensors with multiple pixels. The pixelated image sensors can be operated according to the methods disclosed below.

[0034] Figure 2 A method 200 for reading and processing data acquired by a camera that includes at least a pixelated image sensor is shown. The camera can detect charged particles emitted from a sample in a microscope, such as... Figure 1 A charged particle microscope 100. During the execution of method 200, data flow between the microscope's components is... Figure 3 As shown in the diagram, the mfCDS method is used in conjunction with data compression to read out camera data, thereby improving the frame transmission rate within the camera and from the camera to the image processor.

[0035] In step 201, the microscopy data acquisition parameters are set. These parameters may include one or more of the following: the dose of the charged particle beam at the sample plane, the imaging / scanning area, the data readout rate at the image sensor, and the number of frames N readout between adjacent image sensor resets. The number of frames N between adjacent sensor resets can be determined based on the estimated pixel voltage change between adjacent pixel voltage readouts and the full-well capacity of the pixel. For example, the pixels of the image sensor are reset before reaching a predetermined maximum pixel voltage amplitude. The maximum pixel voltage amplitude is below the full-well capacity of the pixel. The pixel voltage change between adjacent pixel voltage readouts can be estimated based on the dose of the charged particle beam and the sample type.

[0036] At 204, a beam of charged particles is directed onto the sample. In response to the illumination of the charged particles, various types of charged particles, such as secondary electrons and X-rays, are emitted from the sample. Multiple cameras (or detectors) in the microscope sense the emitted charged particles. For example, the cameras may include one or more of a TEM camera, a STEM camera or detector, an EDX detector, and detectors in a spectroscopic device used to sense EELS spectra. The camera contains a pixelated image sensor. The pixel voltage of a particular pixel changes approximately monotonically in response to one or more charged particles impacting the pixel.

[0037] At step 206, the pixel voltage is read from the image sensor at the frequency determined in step 201 and digitized into a first number of bits. In one example, the pixel voltages of multiple pixels of the image sensor are read according to a predetermined pattern to form a frame of pixel voltages. During image sensor readout, the image sensor repeatedly reads at the frame rate determined in step 201. After acquiring N consecutive frames, the image sensor is reset by resetting the pixel voltage of each pixel to a reset voltage. The reset voltage may be different each time. For each pixel in the plurality of pixels, the pixel voltage is read once during each frame readout. Before resetting the image sensor, the pixel voltage of each pixel in the plurality of pixels is repeatedly read N times.

[0038] In step 208, the pixel voltage is compressed, and the compressed pixel voltage is output to the image processor. In one instance, the compressed pixel voltage can be digitized and transmitted to the image processor. In another instance, the pixel voltage is digitized before compression. The digitized compressed pixel voltage has a second number of bits, which is less than the first number of bits used to represent the pixel voltage. The pixel voltage is compressed to a range smaller than the maximum range of the pixel voltage. The maximum range of the compressed pixel voltage is no greater than the amplitude of a first threshold voltage. In one instance, in response to the pixel voltage amplitude being no less than the amplitude of the first threshold voltage and no less than the amplitude of the second threshold voltage, the first threshold voltage is subtracted from the pixel voltage. In another instance, the digitized pixel voltage is compressed by removing at least the MSB. The first threshold voltage can be represented by a number of bits less than the number of bits used to represent the maximum value of the pixel voltage.

[0039] like Figure 3 As shown, in one example configuration, camera 301 includes image sensor 302, ADC 303, and readout electronics 304. Pixel voltages read from image sensor 302 are digitized into a first number of bits by the ADC and then compressed into a second number of bits. Readout electronics 304 can control the timing of data readout and output the compressed digitized pixel voltages to image processor 320.

[0040] Figures 4A-4C This illustrates the process of compressing pixel voltage as the pixel voltage increases in response to charged particles impacting the pixel. Figure 4AThe y-axis represents the pixel voltage of a specific pixel in the image sensor, or the corresponding digitized pixel voltage. The x-axis represents time. Time increases as indicated by the arrows. The solid line plot 403 shows the uncompressed pixel voltage read from the image sensor. The uncompressed pixel voltage can be an analog or digital signal. The dashed line plot 404 shows the compressed pixel voltage. At T0, the image sensor is reset. Therefore, the pixel voltage is reset to the reset voltage. Here, the reset voltage is zero. In other instances, the reset voltage can be a non-zero value. The reset voltage may vary with each reset, thus introducing reset noise. Starting from T0, the pixel voltage 403 increases from T0 to T4 as more charged particles strike the pixel. At T4, the image sensor is reset again. Arrows 401 and 402 indicate the image sensor reset event. The pixel voltage is read out at a frequency of 1 / ΔT. In other words, the image sensor is read out at a frame rate of 1 / ΔT. From T0 to T1, the pixel voltage 403 lies between the reset voltage and the first threshold voltage V1, and the compressed pixel voltage 404 is equal to the pixel voltage 403. From T1 to T3, in response to pixel voltage 403 being not less than the first threshold voltage V1 and less than the second threshold voltage V2, the compressed pixel voltage 404 is equal to pixel voltage 403 minus the first threshold voltage V1. The second threshold voltage V2 is twice the first threshold voltage V1. From T2 to T3, in response to pixel voltage 403 being not less than the second threshold voltage V2 and less than the third threshold voltage V3, the compressed pixel voltage 404 is equal to pixel voltage 403 minus the second threshold voltage V2. The third threshold voltage V3 is three times the first threshold voltage V1. From T3 to T4, in response to pixel voltage 403 being not less than the third threshold voltage V3 and less than the fourth threshold voltage V4, the compressed pixel voltage 404 is equal to pixel voltage 403 minus the third threshold voltage V3. The fourth threshold voltage V4 is four times the first threshold voltage V1. In T4, since N frames have been acquired from the previous reset in T0, the pixel voltage is reset again to the reset voltage. From T4 to T5, since pixel voltage 403 is below the first threshold voltage, pixel voltage 403 is the same as compressed pixel voltage 404. After T5, as pixel voltage 403 increases to above V1 and below V2, compressed pixel voltage 404 is equal to pixel voltage 403 minus V1. Therefore, compressed pixel voltage 404 is between zero and V1. For digitized signals, Figure 4AThe compression process shown reduces the bit depth of the digitized pixel voltage by 2 bits from the digitized pixel voltage. For example, the digitized pixel voltage has 12 bits, and the digitized compressed pixel voltage has 10 bits. The first to fourth threshold voltages are 1024, 2048, 3072, and 4096, respectively. Compression introduces aliasing into the compressed pixel voltage. For example, the pixel voltage between T1 and T2 aliased with the pixel voltage between T0 and T1 (and therefore indistinguishable). By adjusting the pixel values ​​of the differentially compressed frame to an effective range, aliasing can be corrected or resolved in the image processor.

[0041] If the pixel voltage is digitized, a compressed digitized pixel voltage can be generated by saving the first bit of the digitized first pixel voltage to the second bit and removing the remaining bits. Neither the first nor the second bit is the MSB (Maximum Segment Bus). In one instance, subtracting the threshold voltage from the pixel voltage can be achieved by removing one or more bits from one side of the MSB, such as... Figure 4B As shown. For example, the digitized pixel voltage 410 has 12 bits. Figure 4A The compression shown can be achieved by removing 2 bits from the MSB side. The digitized compressed pixel voltage is 10 bits on the LSB side, as shown in 412. Therefore, in Figure 4A In the equation, D1 is 1024, D2 is 2048, D3 is 3072, and D4 is 4096.

[0042] In another example, the threshold voltage can be subtracted from the pixel voltage by removing one or more bits from both the MSB and LSB sides, such as... Figure 4C As shown. For example, the digitized pixel voltage 410 has 12 bits. The digitized compressed pixel voltage has bits 1 through 10, as shown in 421. In this example, the signal precision of the digitized compressed pixel voltage is reduced compared to the digitized pixel voltage, in order to improve the data transmission rate.

[0043] Figure 4A The diagram shows that the uncompressed pixel voltage increases monotonically between consecutive sensor resets. In another embodiment, the uncompressed pixel voltage read from the image sensor decreases monotonically between consecutive resets. In one instance, the pixel voltage can be compressed by subtracting a negative threshold voltage from the uncompressed pixel voltage. In another instance, the uncompressed pixel voltage can be converted into a monotonically increasing pixel voltage before compression, for example, by subtracting from a threshold pixel voltage, such as... Figures 4A-4C As shown.

[0044] Back Figure 2 In 210, the image processor receives compressed pixel voltages or digitized compressed pixel voltages from the camera and uses these compressed pixel voltages to form compressed frames. In one example, such as... Figure 3As shown, the image processor 320 may include one or more FPGAs 322 and a memory 323. The FPGA 322 has direct memory access to the memory 323. The image processor 320 may optionally include a processor 321 for controlling data / image processing within the FPGA 322.

[0045] In section 212, sensor overexposure is determined based on compressed frames. Sensor overexposure can be determined based on the pixel values ​​and variance of the compressed frames. Figure 3 The dose protection block 324 represents the process for determining sensor overexposure. Details of overexposure detection are presented in... Figure 5 If sensor overexposure is detected, at 214, method 200 can prevent charged particles from reaching the image sensor, for example, by closing the shutter. Method 200 can notify the operator of the sensor overexposure. Method 200 can further adjust the data acquisition parameters of the current image session or stop the current image session. If sensor overexposure is not detected, method 200 proceeds to 218.

[0046] At 218, a differential compressed frame is generated by subtracting the compressed frame from the previously acquired compressed frame. The pixel values ​​of the differential compressed frame are then adjusted to a valid range. For example, this is done by subtracting the first compressed frame acquired at the first time point t1 from the second compressed frame acquired at the second time point t2 immediately following the acquisition of the first compressed frame, i.e., E. t2 -E t1 , thus obtaining differential compressed frames.

[0047] Step 218 may optionally include dark correction of the differentially compressed frame before the subtraction. That is, the differentially compressed frame is generated by subtracting the sequentially acquired dark-corrected compressed frames. For example, as... Figure 3 As shown, at 325, the dark frame 331 stored in memory 323 can be optionally subtracted from the compressed frame to generate a dark-corrected compressed frame 326. The dark-corrected compressed frame 326 is temporarily stored in memory 323. The dark-corrected compressed frame 326 is also sent to 328 to subtract the previously stored compressed frame. After a delay of 327, upon receiving the next dark-corrected compressed frame, the dark-corrected compressed frame 326 is subtracted from the next dark-corrected compressed frame at 328 to generate a differential compressed frame.

[0048] exist Figure 3 Box 329 adjusts the pixel values ​​of the differentially compressed frame to an effective range. This is based on a predetermined noise amplitude and the values ​​used in... Figure 2The effective range is determined by a first threshold voltage at 208 for compressing the pixel voltage. Noise may include one or more types of sensor dark noise, sensor thermal noise, sensor readout noise, and sensor quantization noise. The noise amplitude can be determined a priori by examining unirradiated image frames. For example, the noise amplitude is determined based on the standard deviation of pixel values ​​in image frames acquired without irradiating the sample with a charged particle beam. A noise offset is determined based on the noise amplitude. The noise offset can be a negative value of the noise amplitude. In one instance, the effective range is the sum of the noise offset to the first threshold voltage and the noise offset, where the noise offset is non-positive. If the pixel value is less than the noise offset, the first threshold voltage is added to the pixel value. If the pixel value is greater than the sum of the first threshold voltage and the noise offset, the first threshold voltage is subtracted from the pixel value. The first threshold voltage is either the pixel voltage V1 or the digitized pixel voltage D1 used to compress the pixel voltage at 208.

[0049] Figure 6A This illustrates adjusting the pixel values ​​of a differentially compressed frame to an effective range when there is no noise or zero noise. Because the compressed pixel voltage ranges from zero to a first threshold voltage, the pixel values ​​of the differentially compressed frame (i.e., the difference between two compressed pixel voltages) range from a negative first threshold voltage -V1 to a first threshold voltage V1. The effective range 610 is from zero to V1. If a pixel value is in the invalid range indicated by the shaded area (i.e., outside the effective range 610), the pixel value is adjusted to the effective range by adding the first threshold voltage to the pixel value. For example, pixel value 611 is adjusted to pixel value 612. Thus, pixel values ​​in the invalid range are moved to the effective range, as indicated by arrow 613.

[0050] Figure 6B This illustrates how pixel values ​​in a differentially compressed frame are adjusted to an effective range when noise is present in the pixel voltage. Noise offset 601 is negative. The effective range 620 is the sum of noise offset 601 to the first threshold voltage V1 and noise offset 601, 602. Shaded areas indicate invalid ranges. If the pixel value is from -V1 to noise offset 601, the first threshold voltage V1 is added to the pixel value, causing it to move to the range from 0 to the sum 602, as shown by arrow 621. If the pixel value is from the sum 602 to V1, the first threshold voltage V1 is subtracted from the pixel value. As a result, pixel values ​​in range 623 are moved to range 624, as shown by arrow 622.

[0051] In 220, sample images are formed based on the difference images. For example... Figure 3 As shown, the differential image is transmitted from the image processor 320 to the controller 50 to generate a sample image. Step 220 may include preprocessing the differential image before forming the sample image.

[0052] In this way, data is read from the camera at a bit depth lower than that used for reading from the digital image sensor. The camera can operate at the maximum frame rate to read from the sensor data, and the overall frame rate used for data acquisition can be increased. Pixel voltage compression can be performed at high speed by removing one or more bits from the MSB of the digitized pixel voltage. Changes in pixel voltage during sequential image sensor readout can be losslessly reconstructed by adjusting the pixel values ​​of the compressed differential frames to an effective range. It should be noted that in some cases, the operations described in sequence can be rearranged or performed simultaneously.

[0053] Figure 5 A method 500 for detecting sensor overexposure based on compressed frames from camera output is illustrated. Sensor overexposure can be detected based on the amount and variance of pixel values ​​in the compressed frames. In one example, overexposure is determined based on one or more compressed frames acquired immediately after a sensor reset.

[0054] In 502, each compressed frame is divided into multiple subframes, each containing one or more pixels. Subframes can overlap each other.

[0055] At 504, the sum and variance of all pixel values ​​in the subframe are calculated and compared to the threshold sum at 506. In one instance, the variance can be the mathematical variance of the pixel values ​​in the subframe. In another instance, the variance can be calculated using other simplified approximations. The threshold sum can be determined based on the number of pixels in each subframe, the full-well capacity of each pixel, and the number of frames since the most direct sensor reset. The threshold variance can be determined by measuring the pixel value with actual intentional (non-destructive) overexposure. Overexposure is detected at 508 if the sum of all pixel values ​​in any subframe is greater than the threshold sum and the variance of the subframe is less than the threshold variance. Otherwise, overexposure is not detected at 510.

[0056] The advantage of compressing pixel voltages read from the image sensor is that it enables high frame rate data transmission even with limited bandwidth between the ADC and readout electronics and / or between the camera and the image sensor. The advantage of compressing pixel voltages by removing one or more bits from the MSB of the digitized pixel voltage is that compression can be achieved at high speed. The advantage of generating differential frames based on sequentially acquired compressed frames is that it determines the changes in pixel voltage in response to charged particle impacts on the sensor. The advantage of correcting the range of differential compressed frames to obtain differential frames is that it corrects aliasing caused by compression. The accuracy of digitized compressed pixel voltages is the same as that of pixel values ​​in differential frames.

[0057] In one presentation, a method for acquiring data from a camera including a pixelated image sensor for detecting charged particles includes: receiving first and second digitized compressed pixel voltages from the camera; determining a differential compressed pixel voltage by calculating the difference between the first and second digitized compressed pixel voltages; generating the differential pixel voltage by adjusting the differential compressed pixel voltage to an effective range determined by a predetermined noise offset and a first threshold voltage; and forming a sample image based on the differential pixel voltage.

[0058] In another presentation, a camera for detecting charged particles includes an image sensor and one or more ADCs, wherein the camera is configured to: read pixel voltages of one or more pixels of the image sensor multiple times without resetting the image sensor; digitize the pixel voltages into a first number of bits; and output the digitized compressed pixel voltages with a second lower number of bits, wherein the maximum range of the digitized compressed pixel voltages is less than the maximum range of pixel voltages, and wherein the digitized compressed pixel voltages are generated by removing at least the most significant bit (MSB) of the digitized pixel voltages.

[0059] In one embodiment, a method for acquiring data from a camera comprising a pixelated image sensor for detecting charged particles includes: repeatedly reading pixel voltages of one or more pixels of the image sensor without resetting the image sensor; digitizing the pixel voltages into a first number of bits; and outputting a digitized compressed pixel voltage with a second lower number of bits, wherein the maximum range of the digitized compressed pixel voltage is less than the maximum range of pixel voltages, and wherein the digitized compressed pixel voltage is generated by removing at least the most significant bit (MSB) of the digitized pixel voltage. In a first instance of the method, the digitized pixel voltage is unsigned. A second instance of the method optionally includes the first instance and further includes, for each of one or more pixels of the image sensor, sequentially receiving a first digitized compressed pixel voltage and a second digitized compressed pixel voltage; determining a differential compressed pixel voltage by calculating the difference between the first digitized compressed pixel voltage and the second digitized compressed pixel voltage; and generating a differential pixel voltage by adjusting the differential compressed pixel voltage to an effective range, wherein the effective range is determined based on a predetermined noise offset and the maximum range of the digitized compressed pixel voltage. A third embodiment of the method optionally includes one or more of the first to second embodiments, and further includes, wherein the effective range is the sum of the noise offset to the threshold voltage and the noise offset, and the threshold voltage is determined based on the maximum range of the digitized compressed pixel voltage. A fourth embodiment of the method optionally includes one or more of the first to third embodiments, and further includes, wherein adjusting the differential compressed pixel voltage to the effective range includes adding a threshold voltage to the differential compressed pixel voltage in response to the differential compressed pixel voltage being lower than the noise offset, and subtracting the threshold voltage from the differential compressed pixel voltage in response to the differential compressed pixel voltage being greater than the sum of the threshold voltage and the noise offset. A fifth embodiment of the method optionally includes one or more of the first to fourth embodiments, and further includes, wherein the number of second bits is determined based on the maximum range of variation of pixel values ​​between sequential readouts. A sixth embodiment of the method optionally includes one or more of the first to fifth embodiments, and further includes, wherein the digitized compressed pixel voltage has the same signal precision as the digitized pixel voltage. A seventh embodiment of the method optionally includes one or more of the first to sixth embodiments, and further includes, wherein the digitized compressed pixel voltage is generated by further removing one or more bits from the least significant bit side of the digitized pixel voltage. An eighth embodiment of the method optionally includes one or more of the first to seventh embodiments, and further includes detecting overexposure of the image sensor based on the digitally compressed pixel voltage of one or more pixels of the image sensor.

[0060] In one embodiment, a method for acquiring data from a camera comprising a pixelated image sensor for detecting charged particles includes: repeatedly reading pixel voltages of pixels of the image sensor without resetting the image sensor; compressing the pixel voltages into compressed pixel voltages, wherein the compressed pixel voltage is a difference between the pixel voltage and the first threshold voltage in response to an amplitude of the pixel voltage not less than the amplitude of a first threshold voltage and less than the amplitude of a second threshold voltage, and wherein the maximum range of the compressed pixel voltage is not greater than the amplitude of the first threshold voltage and the maximum range of the compressed pixel voltage is less than the maximum range of the pixel voltage; digitizing the compressed pixel voltage; and outputting the digitized compressed pixel voltage. In a first example of the method, the method further includes, wherein compressing the pixel voltage further includes subtracting a second threshold voltage from the pixel voltage in response to an amplitude of the pixel voltage not less than the amplitude of a second threshold voltage and less than the amplitude of a third threshold voltage. A second example of the method optionally includes the first example and further includes wherein the second threshold voltage is twice the first threshold voltage. A third example of the method optionally includes one or more of the first to second examples and further includes resetting the image sensor after reading the pixel voltages of the pixels a predetermined number of times. A fourth embodiment of the method optionally includes one or more of the first to third embodiments, and further includes resetting the image sensor in response to a pixel voltage amplitude greater than a maximum pixel voltage amplitude. A fifth embodiment of the method optionally includes one or more of the first to fourth embodiments, and further includes wherein the pixel voltage between adjacent sensor resets is a monotonic signal superimposed with noise. A sixth embodiment of the method optionally includes one or more of the first to fifth embodiments, and further includes sequentially receiving a first digitized compressed pixel voltage and a second digitized compressed pixel voltage from a camera; dark-correcting the first and second digitized compressed pixel voltages; determining a differential compressed pixel voltage by calculating the difference between the dark-corrected first and second digitized compressed pixel voltages; and generating a differential pixel voltage by adjusting the differential compressed pixel voltage to an effective range, the effective range being determined based on a predetermined noise offset and a first threshold voltage. A seventh embodiment of the method optionally includes one or more of the first to sixth embodiments, and further includes wherein the effective range is the sum of the noise offset to the first threshold voltage amplitude and the noise offset.

[0061] In one embodiment, a system for acquiring data from a sample includes: a charged particle source for irradiating the sample with charged particles; a camera for detecting charged particles emitted from the sample in response to irradiation, the camera including an image sensor having a plurality of pixels and one or more analog-to-digital converters (ADCs), wherein the camera is configured to: convert charged particles irradiating a pixel of the plurality of pixels into pixel voltages; compress the pixel voltages into compressed pixel voltages, wherein the compressed pixel voltage is the difference between the pixel voltage and the first threshold voltage if the amplitude of the pixel voltage is not less than the amplitude of the first threshold voltage and is less than the amplitude of the second threshold voltage, and wherein the maximum range of the compressed pixel voltage is not greater than the amplitude of the first threshold voltage and the maximum range of the compressed pixel voltage is less than the maximum range of the pixel voltage; digitize the compressed pixel voltage; and output the digitized compressed pixel voltage; an image processor for receiving the digitized compressed pixel voltage from the camera and generating a differential pixel voltage based on the digitized compressed pixel voltage; and a controller for forming an image of the sample based on the differential pixel voltage. In a first instance of the system, the system further includes receiving digitized compressed pixel voltages from a camera and generating differential pixel voltages based on the compressed pixel voltages, comprising: sequentially receiving a first digitized compressed pixel voltage and a second digitized compressed pixel voltage; determining the differential compressed pixel voltage by subtracting the first digitized compressed pixel voltage from the second digitized compressed pixel voltage; and generating the differential pixel voltage by adjusting the differential compressed pixel voltage to an effective range, the effective range being determined based on a predetermined noise offset and a first threshold voltage. A second instance of the system optionally includes the first instance and further includes wherein the digitized compressed pixel voltage and the differential pixel voltage have the same precision.

Claims

1. A method for acquiring data from a camera comprising a pixelated image sensor for detecting charged particles, comprising: The pixel voltage of one or more pixels of the image sensor can be read multiple times without resetting the image sensor; The pixel voltage is digitized into a first number of bits; as well as The digitally compressed pixel voltage is output with a second lower number of bits, wherein the maximum range of the digitally compressed pixel voltage is smaller than the maximum range of the pixel voltage, and wherein the digitally compressed pixel voltage is generated by removing at least the most significant bit (MSB) of the digitally compressed pixel voltage.

2. The method of claim 1, wherein the digitized pixel voltage is unsigned.

3. The method according to claim 1 or 2, further comprising: For each pixel in one or more pixels of the image sensor, a first digitized compressed pixel voltage and a second digitized compressed pixel voltage are received sequentially; The differential compressed pixel voltage is determined by calculating the difference between the first digitally compressed pixel voltage and the second digitally compressed pixel voltage. as well as Differential pixel voltages are generated by adjusting the differential compressed pixel voltages to an effective range, wherein the effective range is determined based on a predetermined noise offset and the maximum range of the digitized compressed pixel voltages.

4. The method of claim 3, wherein the effective range is the sum of the noise offset to the threshold voltage and the noise offset, and the threshold voltage is determined based on the maximum range of the digitally compressed pixel voltage.

5. The method of claim 4, wherein adjusting the differential compressed pixel voltage to the effective range comprises adding the threshold voltage to the differential compressed pixel voltage in response to the differential compressed pixel voltage being lower than the noise offset, and subtracting the threshold voltage from the differential compressed pixel voltage in response to the differential compressed pixel voltage being greater than the sum of the threshold voltage and the noise offset.

6. The method of claim 1, wherein the second lower number of bits is determined based on the maximum range of variation in pixel values ​​between sequential readouts.

7. The method of claim 1, wherein the digitized compressed pixel voltage has the same signal accuracy as the digitized pixel voltage.

8. The method of claim 1, wherein the digital compressed pixel voltage is generated by further removing one or more bits from the least significant bit side of the digital pixel voltage.

9. The method of claim 1, further comprising detecting overexposure of the image sensor based on the digitally compressed pixel voltage of the one or more pixels of the image sensor.

10. A method for acquiring data from a camera comprising a pixelated image sensor for detecting charged particles, comprising: Repeatedly read the pixel voltage of the image sensor without resetting the image sensor; The pixel voltage is compressed into a compressed pixel voltage, wherein the compressed pixel voltage is the difference between the pixel voltage and the first threshold voltage in response to the pixel voltage having an amplitude not less than the amplitude of a first threshold voltage and less than the amplitude of a second threshold voltage, and wherein the maximum range of the compressed pixel voltage is not greater than the amplitude of the first threshold voltage, and the maximum range of the compressed pixel voltage is lower than the maximum range of the pixel voltage. The compressed pixel voltage is digitized; as well as Output the digitally compressed pixel voltage.

11. The method of claim 10, wherein compressing the pixel voltage further comprises subtracting the second threshold voltage from the pixel voltage in response to the pixel voltage having an amplitude that is not less than the amplitude of the second threshold voltage and less than the amplitude of the third threshold voltage.

12. The method of claim 10, wherein the second threshold voltage is twice the first threshold voltage.

13. The method of claim 10, further comprising resetting the image sensor after reading the pixel voltage of the pixel a predetermined number of times.

14. The method of claim 10, further comprising resetting the image sensor in response to the magnitude of the pixel voltage being greater than the maximum magnitude of the pixel voltage.

15. The method of claim 10, wherein the pixel voltage between adjacent sensor resets is a monotonic signal superimposed with noise signals.

16. The method according to any one of claims 10-15, further comprising: The camera sequentially receives a first digitally compressed pixel voltage and a second digitally compressed pixel voltage; Dark correction is applied to the first digitally compressed pixel voltage and the second digitally compressed pixel voltage. The differential compressed pixel voltage is determined by calculating the difference between the dark-corrected first digitally compressed pixel voltage and the dark-corrected second digitally compressed pixel voltage. as well as Differential pixel voltages are generated by adjusting the differential compressed pixel voltages to an effective range, the effective range being determined based on a predetermined noise offset and a first threshold voltage.

17. The method of claim 16, wherein the effective range is the sum of the noise offset to the first threshold voltage amplitude and the noise offset.

18. A system for acquiring data from a sample, comprising: A charged particle source for irradiating the sample with charged particles; A camera for detecting charged particles emitted from the sample in response to the irradiation, the camera comprising an image sensor having a plurality of pixels and one or more analog-to-digital converters (ADCs), wherein the camera is configured to: The charged particles that collide with the pixels among the plurality of pixels are converted into pixel voltage; The pixel voltage is compressed into a compressed pixel voltage, wherein if the amplitude of the pixel voltage is not less than the amplitude of the first threshold voltage and is less than the amplitude of the second threshold voltage, then the compressed pixel voltage is the difference between the pixel voltage and the first threshold voltage, and wherein the maximum range of the compressed pixel voltage is not greater than the amplitude of the first threshold voltage, and the maximum range of the compressed pixel voltage is lower than the maximum range of the pixel voltage. The compressed pixel voltage is digitized; as well as Output the digitally compressed pixel voltage; An image processor is configured to receive the digitized compressed pixel voltage from the camera and generate differential pixel voltage based on the digitized compressed pixel voltage; as well as A controller for forming an image of the sample based on the differential pixel voltage.

19. The system of claim 18, wherein receiving the digitized compressed pixel voltage from the camera and generating the differential pixel voltage based on the compressed pixel voltage comprises: The first digitized compressed pixel voltage and the second digitized compressed pixel voltage are received sequentially. The differential compressed pixel voltage is determined by subtracting the first digitally compressed pixel voltage from the second digitally compressed pixel voltage; as well as The differential pixel voltage is generated by adjusting the differential compressed pixel voltage to an effective range, the effective range being determined based on a predetermined noise offset and a first threshold voltage.

Images