Sparse sampling using carrier signal modulation with a programmable randomized signal

Abstract

The system (100) includes a sparse sampling system (108) having a processor and a non-transitory computer-readable medium. The sparse sampling system (108) generates a first programmable primary carrier signal (128) and a first secondary programmable signal (130) that modulates the first programmable primary carrier signal (128) via a first randomized modulation that defines an X-coordinate of a set of scan pattern coordinates for scanning the object, a second programmable primary carrier signal (128) and a second secondary programmable signal (130) that modulates the second programmable primary carrier signal (128) via a second randomized modulation that defines a Y-coordinate of the set of scan pattern coordinates for scanning the object, and a third programmable output signal that defines a Z-coordinate of the set of scan pattern coordinates for scanning the object based on a predetermined rule. The sparse sampling system (108) also transmits the generated signals to a scanning probe instrument.

Description

Summary of the Invention

[0001] In one embodiment, a system is provided, including a sparse sampling system having a processor and a non-transitory computer-readable medium. The sparse sampling system is configured to generate a first programmable primary carrier signal and a first secondary programmable signal modulating the first programmable primary carrier signal via a first randomized modulation defining an X-coordinate of a set of scan pattern coordinates for an object scan, a second programmable primary carrier signal and a second secondary programmable signal modulating the second programmable primary carrier signal via a second randomized modulation defining a Y-coordinate of the set of scan pattern coordinates for the object scan, and a third programmable output signal defining a Z-coordinate of the set of scan pattern coordinates for the object scan based on a predetermined rule. The sparse sampling system is also configured to transmit the generated first and second programmable primary carrier signals, the first and second secondary programmable signals, and the third programmable output signal to a scanning probe instrument.

[0002] In another embodiment, a method is provided, comprising generating, by a sparse sampling system having a processor and a non-transitory computer-readable medium, a first programmable primary carrier signal and a first secondary programmable signal modulating the first programmable primary carrier signal via a first randomized modulation defining an X-coordinate of a set of scan pattern coordinates for scanning an object. The method also includes generating, by the sparse sampling system, a second programmable primary carrier signal and a second secondary programmable signal modulating the second programmable primary carrier signal via a second randomized modulation defining a Y-coordinate of the set of scan pattern coordinates for scanning the object. The method further includes generating, by the sparse sampling system, a third programmable output signal defining a Z-coordinate of the set of scan pattern coordinates for scanning the object based on a predetermined rule. The generated first and second programmable primary carrier signals, the first and second secondary programmable signals, and the third programmable output signal are transmitted by the sparse sampling system to a scanning probe instrument.

[0003] In yet another embodiment, an apparatus is provided. The apparatus includes a sparse sampling pattern generator configured to generate a first programmable primary carrier signal and a first secondary programmable signal that modulates the first primary carrier signal via a first randomized modulation that defines an X-coordinate of a set of scan pattern coordinates for scanning an object. The sparse sampling pattern generator is also configured to generate a second programmable primary carrier signal and a second secondary programmable signal that modulates the second primary carrier signal via a second randomized modulation that defines a Y-coordinate of the set of scan pattern coordinates for scanning the object. The sparse sampling pattern generator is further configured to generate a third programmable output signal that defines a Z-coordinate of the set of scan pattern coordinates for scanning the object based on a predetermined rule. The apparatus also includes a controller connected to the sparse sampling pattern generator and configured to provide the first and second programmable primary carrier signals, the first and second secondary programmable signals, and the third programmable output signal to a scan input of a scanning probe instrument configured to scan an object. The apparatus further includes at least one object signal response transducer configured to receive a scanning response signal from at least one object response detector that detects a response of the object to scan a probe signal directed at the object by the scanning probe instrument, the at least one object response transducer connected to the controller and configured to convert the analog scanning response signal into a digital scanning response signal, and a sampling reconstruction system communicatively connected to the controller, the sampling reconstruction system configured to receive the digital scanning response signal from the controller and responsively reconstruct an integrated image of the object scanned by the scanning probe instrument from the sampled object response signal.

[0004] In yet another embodiment, a system is provided that includes a sampling reconstruction system having a processor and a non-transitory computer-readable medium, the sampling reconstruction system configured to receive a set of sub-sampled addressable pixel elements and to reconstruct missing pixels from the set of sub-sampled addressable pixel elements with adaptive real-time inpainting.

[0005] Other features and advantages that characterize embodiments of the present invention will become apparent upon review of the following detailed description and associated drawings. [Brief explanation of the drawings]

[0006] [Figure 1] FIG. 1 is a schematic diagram of a sparsely sampled scanning system in which embodiments of the present invention may be employed. [Figure 2] FIG. 2 is a diagrammatic representation of one embodiment of the primary carrier signal path of the present invention. [Figure 3] FIG. 3 is a diagrammatic representation of a randomized secondary signal pattern (solid line) modulating a primary carrier signal (dashed line) to which it is referenced. [Figure 4] FIG. 4 is a diagrammatic representation of one embodiment of discrete sparsely sampled coordinates (circular markers) derived from the primary carrier signal shown in FIG. 2 and the randomized modulated secondary signal pattern shown in FIG. [Figure 5] FIG. 5 is a diagrammatic representation of a smoothed Hilbert-type space-filling curve path 502 superimposed on an unsmoothed Hilbert-type space-filling curve path 504. [Figure 6] FIG. 6 is a diagrammatic representation of a scan boundary including a primary carrier signal path and a region of interest (ROI) including the primary carrier signal path scaled relative to the primary carrier path. [Figure 7] FIG. 7 is a diagrammatic representation of an XY sparse sampling embodiment including a set of randomized sparse sample points defined by a scan boundary, a serpentine primary carrier signal path, and a set of plot marker types. [Figure 8] FIG. 8 is a diagrammatic representation of the continuous XY parametric equation for generating the primary carrier signal path. [Figure 9] FIG. 9 is a schematic diagram of a dual column scanning probe instrument that can include at least some of the embodiments of the present invention. [Figure 10A] FIG. 10A is a diagrammatic representation illustrating XYZ sparsity or partial sub-sampling of a scanned object where the Z coordinate is defined by the operation of a predetermined rule, according to one embodiment. [Figure 10B] FIG. 10B is a pictorial representation illustrating XYZ sparsity or partial sub-sampling of a scanned object where the Z coordinate is defined by the operation of a predetermined rule, according to one embodiment. [Figure 10C] FIG. 10C is a pictorial representation illustrating XYZ sparsity or partial sub-sampling of a scanned object where the Z coordinate is defined by the operation of a predetermined rule, according to one embodiment. [Figure 10D] FIG. 10D is a pictorial representation illustrating XYZ sparsity or partial sub-sampling of a scanned object where the Z coordinate is defined by the operation of a predetermined rule, according to one embodiment. [Figure 11A] FIG. 11A is a diagrammatic representation illustrating XYZ sparsity or partial sub-sampling of an object where the Z coordinate is defined by the operation of a predetermined rule, according to another embodiment. [Figure 11B] FIG. 11B is a diagrammatic representation illustrating XYZ sparsity or partial sub-sampling of an object where the Z coordinate is defined by the operation of a predetermined rule, according to another embodiment. [Figure 11C] FIG. 11C is a diagrammatic representation illustrating XYZ sparsity or partial sub-sampling of an object where the Z coordinate is defined by the operation of a predetermined rule, according to another embodiment. [Figure 11D] FIG. 11D is a pictorial representation illustrating XYZ sparsity or partial sub-sampling of an object where the Z coordinate is defined by the operation of a predetermined rule, according to another embodiment. [Figure 12A]FIG. 12A is a pictorial representation of a three-dimensional Hilbert space-filling curve, which represents one type of three-dimensional space-filling curve whose Z coordinate is defined by a predetermined rule, according to yet another embodiment. [Figure 12B] FIG. 12B is a pictorial representation of a three-dimensional Hilbert space-filling curve, which represents one type of three-dimensional space-filling curve whose Z coordinate is defined by a predetermined rule, according to yet another embodiment. [Figure 12C] FIG. 12C is a pictorial representation of a three-dimensional Hilbert space-filling curve, which represents one type of three-dimensional space-filling curve whose Z coordinate is defined by a predetermined rule, according to yet another embodiment. [Figure 12D] FIG. 12D is a pictorial representation of a three-dimensional Hilbert space-filling curve, which represents one type of three-dimensional space-filling curve whose Z coordinate is defined by a predetermined rule, according to yet another embodiment. [Figure 13A] FIG. 13A is a schematic diagram of an implementation of adaptive real-time repair (ARTI) according to an embodiment of the present disclosure. [Figure 13B] FIG. 13B is a schematic diagram of an implementation of adaptive real-time repair (ARTI) according to an embodiment of the present disclosure. [Figure 13C] FIG. 13C is a schematic diagram of an implementation of adaptive real-time repair (ARTI) according to an embodiment of the present disclosure. [Figure 13D] FIG. 13D is a schematic diagram of an implementation of adaptive real-time repair (ARTI) according to an embodiment of the present disclosure. [Figure 13E] FIG. 13E is a schematic diagram of an implementation of adaptive real-time repair (ARTI) according to an embodiment of the present disclosure. [Figure 14A] FIG. 14A is an image of 90% sparse acquisition on a 4Kx4K full pixel density image using a Hilbert-type space-filling curve, according to one embodiment of the present disclosure. [Figure 14B] FIG. 14B is a reconstructed image of FIG. 14A using ARTI. DETAILED DESCRIPTION OF THE INVENTION

[0007] Embodiments of the present disclosure generally relate to sparse sampling applied to analytical instruments that utilize one or more serial scanning systems or subsystems, and computational methods applied to reconstruct an integrated representation of objects that are sparsely sensed through their interaction with one or more analytical probes and response signals collected by one or more response signal detectors.

[0008] The acquisition time of serial scanning analytical instruments can be significantly reduced by applying sparse sampling, subsampling, or compressed sensing. Such instruments include, for example, scanning electron microscopes, electron spectrometers, imaging electron spectrometers, ion microscopes, ion spectrometers, laser confocal microscopes, and X-ray spectrometers. The object being sensed may undergo reversible changes (e.g., accumulation of electronic or ionic charge) or irreversible changes (e.g., bond changes, physical deformation, ion implantation, sputtering) upon interaction with the analytical probe. Detrimental probe-material interactions are reduced by sparse sampling. Sparse sampling and sparse sampling reconstruction benefit from approaches that mitigate artifacts and limitations associated with electromechanical scanning systems. Sources of serial scanning artifacts include, for example, dynamic hysteresis, slew, and nonlinear response. Examples of systems that experience one or more artifacts that can affect the quality of sparse sampling and sparse sampling reconstruction include, for example, magnetic scanning devices, electromagnetic scanning devices, electrostatic scanning devices, electromagnetic probe blanking systems, and electrostatic probe blanking systems.

[0009] A sparse sampling approach that mitigates serial scanning artifacts while enabling higher scanning speeds benefits the quality of sparse sampling and sparse sampling reconstructions. Constructing a sparse sampling scan pattern that is smooth and primarily continuous at the carrier signal scale while simultaneously invoking statistical randomness at the discrete modulation perturbation signal scale mitigates typical artifacts in electromechanical scanning systems and reduces the performance requirements for, or eliminates the need for, dynamic or fast probe blanking. Furthermore, a continuously variable and adaptive approach to sparse sampling allows for greater freedom in the design of scanning strategies for exploring objects and extracting information. The flexibility in the degree of sparse sampling and the structure of the carrier signal pattern enable adaptive scanning strategies based on prior knowledge of the objects being sampled or through information acquired while sensing the objects. Prior knowledge can include geometric, chemical, and structural information. Information acquired during sparse sampling is derived from the probe-object response function over the dominant interaction volume, potentially enabling forward modeling to support adaptive sparse sampling scanning strategies.

[0010] In an embodiment of the present invention, the sparse sampling approach employs a composite signal converter. In one embodiment, each element of the composite signal converter includes a primary carrier signal converter modulated by a secondary signal converter, with the output of the secondary signal converter referenced to the primary carrier signal converter output. The secondary modulation signal converter is programmably randomized. One embodiment includes a pair of such "primary-secondary" composite signal converters configured as a programmable XY scan pattern generator, with one "primary-secondary" composite signal converter generating the X coordinate and a second "primary-secondary" composite signal converter generating the Y coordinate, with all outputs integrated by a programmable logic controller. Such a composite signal converter configured as an XY pattern generator can be programmed to generate an XY pattern including multiple consecutive, ordered, randomized XY coordinates, each coordinate being the sum of a primary XY carrier signal converter and a secondary XY modulation signal converter, the latter acting as a randomizing signal added to the former. In an XY scan pattern generator configured in this manner, the XY carrier signal pattern can be considered a "guiding center" path referenced by the programmably randomized modulated XY signal pattern to define the sparse sampling coordinates. The XY scan pattern generator of the above configuration can be programmed with a variety of arbitrarily smooth and arbitrarily continuous XY phase curves, including a carrier signal XY pattern that is programmably randomized by the modulation signal XY pattern to produce an overall randomized sparsely sampled XY signal pattern.

[0011] Through this approach, the degree of sparsity generated by the aggregate XY pattern can be smoothly and continuously adjusted in fractional percent increments from 0% to over 99% sparsity. Statistical randomness is imparted through programmed randomness into the aggregate XY modulation signal pattern. Carrier signal XY patterns supported through this approach include, by way of example, continuous space-filling curves, serpentine patterns, flyback patterns, generalized polygonal patterns, and custom path coordinates. A composite signal converter configured as a pattern generator capable of carrying a variety of carrier signals that act as guided paths perturbed by the action of modulation signals to generate randomized patterns of coordinates constitutes a versatile and generalized sparse sampling approach applicable to serial scanning probe instruments. This sparse sampling approach defined herein mitigates artifacts and / or detrimental aspects inherent to serial scanning probe instruments. Before providing further details regarding different embodiments, a description of an exemplary operating environment is provided below.

[0012] Figure 1 illustrates an exemplary operating environment in which certain embodiments disclosed herein may be incorporated. The operating environment illustrated in Figure 1 is for illustrative purposes only. Embodiments of the present invention are not limited to a particular operating environment such as the operating environment illustrated in Figure 1. Embodiments of the present invention are illustratively implemented in any number of different types of operating environments.

[0013] It should be noted that like reference numerals are used in different figures to refer to identical or similar elements. It should also be understood that the terminology used herein is for the purpose of describing the embodiments and is not intended to be limiting. Unless otherwise specified, ordinal numbers (e.g., first, second, third, etc.) are used to distinguish or identify different elements or steps in a group of elements or steps and do not provide serial or numerical limitations to the elements or steps of the embodiment. For example, the "first," "second," and "third" elements or steps do not necessarily have to appear in that order, and the embodiment is not necessarily limited to three elements or steps. Furthermore, unless otherwise specified, labels such as "left," "right," "front," "rear," "upper," "lower," "front," "lower," "reverse," "clockwise," "counterclockwise," "top," "bottom," or other similar terms such as "upper," "lower," "rear," "front," "vertical," "horizontal," "proximal," "distal," and "middle" are used for convenience and do not imply, for example, a specific fixed position, orientation, or direction. Instead, such labels are used to reflect, for example, a relative position, orientation, or direction. Also, the singular forms "a," "an," and "the" should be understood to include plural references unless the context clearly dictates otherwise.

[0014] When an element is referred to as being "connected," "coupled," or "attached" to another element, it is understood that it can be directly connected, coupled, or attached to the other element, or indirectly connected, coupled, or attached to other elements, where intervening or intermediate elements may be present. In contrast, when an element is referred to as being "directly connected," "directly coupled," or "directly attached" to another element, there are no intervening elements present. Also, figures showing direct connections, couplings, or attachments between elements include embodiments in which the elements are indirectly connected, coupled, or attached.

[0015] Figure 1 is a schematic diagram of a scanning tool 100 for obtaining a representation of an object 102 that can include at least some embodiments of the present invention. As can be seen in Figure 1, the system 100 includes a scanning probe instrument 104 (e.g., a scanning electron microscope, an electron spectrometer, an imaging electron spectrometer, a scanning ion microscope, an imaging ion spectrometer, a laser confocal microscope, an X-ray spectrometer, etc.) that includes a scanning input 106 for scanning an object such as 102.

[0016] The system 100 also includes a sparse sampling system 108. Generally, the system 108 may include at least one application-specific integrated circuit (ASIC) having at least one processor, at least one non-transitory computer-readable medium, and any other suitable circuitry capable of generating signals, storing the generated signals, and providing the generated signals to a scanning device in accordance with embodiments of the present disclosure. In one embodiment, the system 108 includes at least one composite signal converter 110 capable of converting both a primary carrier signal and a secondary modulating signal. Each composite signal converter 110 may include a carrier signal converter 112 and a secondary signal converter 114. The secondary signal converter 114 is configured to modulate the primary carrier signal converter 112. The output of the secondary modulating signal converter 114 is referenced to the output of the primary carrier signal converter 112. One embodiment of the composite signal converter 110 uses a primary signal converter carrier signal having an output range corresponding to the operational (e.g., full-scale) scan field of the scanning probe device scan input 106, referenced by a secondary modulating signal converter operating at a reduced range and higher rate. One embodiment of composite signal converter 110 utilizes a single digital-to-analog converter (DAC) to convert a composite signal including programmable primary carrier signal 128 and programmable secondary modulating signal 130, performing the functions of both carrier signal converter 112 and modulating signal converter 114.

[0017] Another embodiment of the composite signal converter 110 uses a DAC as the primary carrier signal converter 112 and a separate DAC as the modulation signal converter 114, with the modulation signal DAC 114 referenced to the carrier output of the carrier signal DAC 112. The function of the DAC in all embodiments is to convert the digital signal communicated from the sparse sampling pattern generator 119 through the controller 116 to an analog signal (e.g., voltage) and then communicate the analog signal (e.g., voltage) over an appropriate transmission line (e.g., coaxial cable) to the scan input 106 of the scanning probe instrument 104. One embodiment of the sparse sampling system composite signal converter 110 configures the modulation signal converter 114 as a DAC referenced to a particular bit depth on the carrier signal converter 112 configured as a DAC. In one embodiment of the composite signal converter 110, the modulation signal DAC 114 is referenced to a bit depth corresponding to the noise floor of the primary carrier signal DAC 112. In a particular embodiment of the composite signal converter 110, the secondary modulation signal DAC 114 is referenced to the least significant bit of the primary carrier signal DAC 112. In another embodiment of the composite signal converter 110, a DAC is used as the signal converter, and the amplitude of the secondary modulation signal converter DAC 114 is limited relative to the maximum amplitude of the primary carrier signal converter DAC 112 (e.g., the DAC 114 has a smaller voltage range than the DAC 112). In one embodiment of the composite signal converter 110 using a DAC, the secondary modulation signal converter DAC 114 has a higher frequency response compared to the primary carrier signal converter DAC 112 (e.g., the DAC 114 is faster than the DAC 112). In another embodiment of the composite signal converter 110 using a DAC, the gain of the secondary modulation signal converter DAC 114 referenced to the output of the primary carrier signal converter DAC 112 is programmable. In one embodiment of the sparse sampling system 108, the composite signal converter 110 is configured as an XY pattern generator, where X includes at least one carrier signal converter 112 and at least one modulation signal converter 114, and Y includes at least one primary carrier signal converter 112 and at least one secondary modulation signal converter 114.

[0018] In one embodiment of the sparse sampling system 108 configured as an XY sparse sampling pattern generator, the outputs of the carrier signal converter 112 and the modulation signal converter 114 are transmitted to the scan input 106 of the scanning probe device via a single transmission line. In another embodiment of the sparse sampling system 108 configured as an XY sparse sampling pattern generator, the outputs of the primary carrier signal converter 112 and the secondary modulation signal converter 114 are transmitted to the scan input 106 of the scanning probe device via separate transmission lines, with their operation synchronized together via a controller 116. For example, the primary carrier signal 112 can be transmitted to a set of upper deflection coils (not shown) of a scanning transmission electron microscope, and the secondary modulation signal composite signal converter 114 can be transmitted to a set of lower deflection coils (not shown). The sparsely sampled composite signal converter 110 is scalable to “N” signal converter elements. For example, the composite signal converter 114 includes primary, secondary, and tertiary signal converter elements. The sparse sampling system 108 is scalable as a triad of composite signal converters to form an XYZ pattern generator. One embodiment of the sparse sampling system 108 configured as an XYZ pattern generator is suitable for three-dimensional scanning probe instruments, including but not limited to confocal scanning laser microscopes (CSLMs).

[0019] The system 100 further includes one or more object-response signal converters 122 that convert the object 102's "response" signals from one or more object-response detectors 126. The object-response detector(s) 126 can be of various types, depending on the type(s) of response signal collected from the object 102 (e.g., secondary electrons, backscattered electrons, Auger electrons, secondary ions, X-rays, etc.). One embodiment of the object-response signal converter 122 collects signals from the object-response detector 126 using an analog-to-digital converter (ADC) signal converter or multiple ADCs. One embodiment of the object-response signal converter 122 can include a pulse-processing converter (e.g., for converting X-ray object-response detector signals). In one embodiment of the sparse sampling system 108, the degree of oversampling from the object-response signal converter 122 relative to the dwell time used to collect the object-response detector 126 signals derived from sparsely sensed objects can be averaged to improve the signal-to-noise ratio (SNR) of the response signals. For example, assuming a sufficiently high bandwidth object response detector 126, a dwell time of 1 microsecond (1 us) and a sparse sampling system clock rate of 50 MHz (20 ns) corresponds to an oversampling ratio of 50, allowing a corresponding improvement in SNR of over 7. In one embodiment of the sparse sampling system 108, the object response detector(s) 126 operate continuously or under the control of the scanning probe instrument 104. In another embodiment of the sparse sampling system 108, the object response detector 126 is operably triggered through the action of an input / output object response detector 126 element (e.g., a general purpose input / output, or GPIO). In an alternative embodiment of the sparse sampling system 108, the object response detector 126 sends a trigger signal to the controller 116 to initiate and / or increment a scan or scan event action.

[0020] A controller 116, which may be part of the sparse sampling system 108, is operably coupled to the primary carrier signal converter 112, the secondary signal converter 114, and the object response signal converter 122. The controller 116 coordinates the operations between the signal converters 112, 114, 122 and, in some embodiments, the object response detector 126, as described above. In one embodiment, the controller 116 is a programmable logic controller (PLC). In one embodiment, the PLC is configured as an FPGA (field programmable gate array). In certain embodiments, the FPGA functions as a high-speed data transmission array. In one embodiment of the sparse sampling approach disclosed herein, the sparsely sampled XY pattern coordinates are synchronized with the response signal converter data by the PLC and accessed through an address list that pairs pattern coordinates with response signals.

[0021] In some embodiments, the patterns generated by the sparse sampling pattern generator 119 are adapted to extract information from or verify expectations of the object 102 being detected based on prior knowledge 118 of the object 102 being detected. Prior object knowledge 118 includes, but is not limited to, information about the shape of the object 102 based on design information from a computer-aided design (CAD) digital content or a graphic design system (GDS) (e.g., a GDSII digital file), etc. Prior object knowledge 118 can also be derived from lower resolution and / or larger field-of-view data that provides knowledge of hierarchical congestion and / or geometric density. Examples of such information include, but are not limited to, optical data or lower resolution data from the same or a different scanning probe instrument. Generally, prior object knowledge 118 includes structural information about the object 102, chemical information about the object 102, or any other suitable information.

[0022] In typical operation, the object 102 is positioned near the scanning probe on the support 101. In some embodiments, the support 101 is a fixed platen and the scanning probe is movable in the XY or XYZ directions. In an embodiment with a movable scanning probe, the scanning probe can move in steps or continuously while the object 102 being sensed remains fixed and stationary. In other embodiments, the support 101 is a stage that is movable in the XY or XYZ directions. In an embodiment where the scanning probe is movable and the object is placed on a movable stage, the sparse sampling approach disclosed herein can be performed while simultaneously moving the stage or substage during the sparse sampling operation, thereby allowing a continuous or primarily continuous dynamic sparse sampling pattern to sense objects over an area or volume that can extend to constitute ordered continuous, primarily continuous, or multiple continuous strips across one, two, or three dimensional space. In one embodiment of support 101, the simultaneous XY or XYZ motion stages are mechanical piezoelectric stages, laser interferometric stages, feedback encoded stages, or other precision motion stages capable of controlling the motion of the sensed object within the overall system resolution target. In one embodiment of support 101, the simultaneous motion precision stages have a step resolution of 1 nanometer or better along each axis.

[0023] In one embodiment of the support 101, the simultaneously moving precision stage is a substage permanently or temporarily fixed to an existing primary stage. In some embodiments, the target and / or scanning probe are in ambient atmosphere. In other embodiments, the target 102 and / or elements of the scanning probe instrument 104 are in a partial vacuum. The sparse sampling pattern generator 119 generates a series of pattern sets, including a primary carrier signal path and a secondary modulation signal path, which are sent to the controller 116. If applicable, the sparse sampling pattern generator 119 defines a dwell time for each sparse sampling coordinate of the pattern and communicates the dwell time data for each coordinate to the controller 116. Typically, the dwell time significantly exceeds the probe transit time of the scanning probe instrument 104 between sampling coordinates. In one embodiment of the sparse sampling system 108, the dwell time is programmable for each discrete sparse sampling pattern element. For example, each pixel element can have a dwell time proportional to the grayscale intensity of the corresponding image pattern. In one embodiment of the sparse sampling system 108, if a threshold signal-to-noise response signal value is reached, the programmable dwell time per sample coordinate may be truncated, as processed via the controller 116. For example, if a programmable pixel intensity value threshold for a back-scattered electron (BSE) object response detector is reached before the programmed dwell time for that pixel element, the dwell time is truncated for that pixel through operation of the controller 116, reducing the overall sampling time in the process.

[0024] The controller 116 adjusts the sequenced timing and distributes the adjusted output signal to the composite signal converter 110, which includes a primary carrier signal converter 112 and a secondary modulation signal converter 114. While the scanning probe instrument is functioning under nominal operating conditions, the sparse sampling system 108 transmits output signals from the primary carrier signal converter 112 and the secondary modulation signal converter 114 to the scan input 106 of the scanning probe instrument 104 (e.g., an external scan input of a scanning electron microscope, a scan amplifier circuit, or a deflection coil circuit). In one embodiment, the sparse sampling system 108 is an integrated component of the analytical instrument and functions as the primary pattern generator for the scanning probe instrument 104. In another embodiment, the sparse sampling system 108 interfaces with the scan input 106, which is an external scan input provided by the scanning probe instrument manufacturer. For example, it is common to provide an external scan control input for external scan control of scanning electron microscopes, scanning transmission electron microscopes, and other scanning probe instruments for use with third-party pattern generators.

[0025] Sequential probe position coordinates are controlled by sparsely sampled system output signals communicated through a scan input 106 signal interface to position the probe of the scanning probe instrument 104. If the probe of the scanning probe instrument 104 is fixed, the scan input 106 actuates an XY or XYZ stage driver interface (not shown) to position an object scan coordinate proximate to the probe. At each scanning probe coordinate, the probe induces a response from the object 102 for the duration of the dwell time, e.g., secondary electrons in the case of a scanning electron microscope or an interaction force in the case of an atomic force microscope. The induced object response signals at each scanning probe coordinate are simultaneously detected by an object response detector 126 for the duration of the dwell time. The induced signal streams at each scan coordinate position are simultaneously sent via an object response signal converter 122 to the controller 116, which correlates the scanning probe position signals with the object response signals for the duration of the dwell time. In one embodiment, the signal streams of the object response signal converter 122 are sent in data packets to a memory buffer included in the controller 116. The controller 116 communicates the ordered set of data from the object-response signal converter 122 to the image reconstruction system 120 to reconstruct a unified representation of the sparsely sampled object 102. In some embodiments, the image reconstruction system 120 may include at least one ASIC having at least one processor, at least one non-transitory computer-readable medium, and any other suitable circuitry to perform image reconstruction according to the following embodiments.

[0026] In one embodiment of the image reconstruction system 120, an integrated representation of the sparsely sampled object 102 being detected may be reconstructed from the object response signal transducer 122 patterns collected through a suitable object response detector 126 using a restoration reconstruction method. One particular embodiment of the restoration reconstruction method may be Beta Process Factor Analysis (BPFA). In one embodiment of the image reconstruction system 120, an integrated representation of the sparsely sampled object 102 being detected is reconstructed from the object response signal transducer patterns collected through a suitable object response detector using a downsampling method. One embodiment of the downsampling reconstruction method locates a pixel element closest to a missing pixel element and assigns the same value of the closest element to the missing pixel element.

[0027] In another embodiment, the image reconstruction system 120 performs reconstruction of a missing pixel from a subsampled image by first measuring subsampled addressable pixel elements (e.g., all subsampled addressable pixel elements) within a one-pixel grid perimeter of the missing pixel in each direction (a two-dimensional XY grid or a three-dimensional XYZ grid). The missing pixel elements are graphically represented by circular markers in FIG. 13A. A missing pixel element with a two-dimensional search grid of one pixel is graphically represented in FIG. 13B, where pixels marked with an "X" are addressable pixel elements searched for subsampled values. If one subsampled pixel is found in the search grid, that value is assigned to the missing pixel. If two or more subsampled pixels are found in the search grid, the arithmetic mean of the subsampled pixel values ​​is assigned to the missing pixel. If no subsampled pixel value is identified within the one-pixel grid search, the measurement perimeter is expanded to a two-pixel radius in each coordinate direction, and the average pixel value of all subsampled pixels measured in the two-pixel radius is assigned to the missing pixel, as graphically represented in FIG. 13C. If no subsampled pixel value is identified within a two-pixel element radius, the perimeter is expanded to three pixels, as graphically represented in Figure 13D. A four-pixel perimeter is graphically represented in Figure 13E. The search grid perimeter continues to expand in addressable pixel element-by-pixel steps until at least one subsampled pixel value is identified. This particular reconstruction embodiment is referred to as adaptive real-time inpainting (ARTI). The adaptive aspect reflects the fact that the search radius is programmatically expanded until at least one subsampled pixel is identified within the search perimeter.

[0028] As the perimeter of the grid size increases, the distance to the center of the off-axis subsampled pixel element increases proportionally with respect to the axis originating at the center of the missing pixel element. The larger the perimeter of the grid size, the greater the likelihood of a higher degree of sparse subsampling.

[0029] In one embodiment of ARTI, if two or more subsampled pixels are found within the search grid, a weighted arithmetic mean of the subsampled pixel values ​​is assigned to the missing pixel. The weighting factor may be linear with respect to proximity to the missing pixel, for example, with closer subsampled pixel locations being given larger weighting factors. Specifically, the linear inverse distance weighting factor for a subsampled pixel is the inverse distance to the subsampled pixel divided by the sum of the inverse distances to all subsampled pixels within the search grid perimeter.

[0030] An example of a 90% sparsely subsampled scanning electron microscope (SEM) image acquired using an embodiment of the present disclosure utilizing a Hilbert-type space-filling curve is shown in Figure 14A. The acquired image reconstructed using ARTI is graphically represented in Figure 14B.

[0031] In another embodiment of the sparse sampling technique disclosed herein, a unified representation of the detected object is reconstructed from the sparse sampling pattern using a Fourier sparse domain based method.

[0032] Examples of how sparse sampling is performed are provided below in connection with FIGS. 2-9. FIG. 2 is a graphical representation of one embodiment of a sparsely sampled primary carrier signal path generated by the sparse sampling pattern generator 119. The representation 200 is scaled for visualization. A scan boundary 202 surrounds the individual probe location elements defined by a grid pattern 204. In practice, the location grid pattern may exceed 64 million elements. While the scan boundary 202 depicts a square, linear boundary, the scan boundary 202 may also include a quadrilateral or non-linear boundary. The illustrated primary carrier signal path 206 represents a continuous Hilbert-type space-filling curve. Circular markers 208 overlaid on the primary carrier signal path 206 represent programmatically defined discrete primary carrier signal values ​​along the path 206, which serve as reference values ​​for corresponding secondary modulation signal values. In the sparse sampling system 100, the carrier signal pattern may be constructed from other phase space-filling curves, including, but not limited to: Hilbert curve, Peano curve, Moore curve, Sierpinski curve, Lissajous curve, and their variations.

[0033] FIG. 3 is a graphical representation 300 of a primary carrier signal path 306 identical to path 206 shown in FIG. 2, but now represented as a dotted line, with the additional representation of a randomized secondary modulation signal path 308 (solid line). The vertices of each solid line segment 308 are shown in FIG. 3 and represent the XY coordinates of discrete sparsely sampled elements within an element of a sampling array, as defined by grid 304. Note that the dotted lines for carrier signal path 306 and the solid lines for modulation signal path 308 are virtual representations for visualization purposes. A set of coordinates identified by marker type 310 defines a set of sparsely sampled coordinates located within an element of grid 304. The total number of grid elements defined by grid 304 determines the size of the array (e.g., 1024 x 1024, 2048 x 2048, 4096 x 4096, 8192 x 8192) and the corresponding step size, expressed in signal amplitude or resolution interval.

[0034] FIG. 4 is a graphical representation 400 of one embodiment of discrete sparsely sampled coordinates, represented by a circular marker 410 with a scan boundary 402 and a scan grid array 404. The set of points 410 represents the union of the primary carrier path 206 of FIG. 2 and the randomized secondary modulation signal path 308 of FIG. 3. Each circular marker 410 in FIG. 4 represents a sparse sampling element with a programmable dwell time. By selecting a different random seed or a different randomization algorithm, the same primary carrier signal path (e.g., a Hilbert-type space-filling path in FIG. 2) can generate different sets of sparsely sampled coordinates with the same or different degrees of sparsity. The degree of sparsity can be adjusted through the disclosed approach in fractional percent increments from 0% to over 99% sparsity. The distribution of work delivered to the scanning probe through the action of the primary carrier signal path on the secondary modulation pattern can be adjusted by adjusting the scale of the primary carrier pattern, along with the maximum signal amplitude allowed by the secondary modulation signal. For example, in one embodiment of the sparse sampling system 108, the primary carrier signal has a signal amplitude of ±10 V, while the maximum amplitude of the secondary modulation signal is ±3 mV. In this embodiment, the secondary modulation signal can deviate by ±3 mV relative to the coincident position of the primary carrier signal to which it is referenced. If the maximum allowable secondary modulation signal is ±0.5 V, the secondary modulation signal can contribute a larger portion of the work to construct the same sparse sampling pattern. The signal amplitude voltage corresponds to physical deflection of the scanning probe instrument 104, stage movement, or both. By varying the maximum secondary modulation signal amplitude as referenced to the primary carrier signal and adjusting the maximum rate of change of both the primary carrier modulation signal and the secondary modulation signal, the sampling rate of the sparse sampling scanning probe system 100 can be varied to mitigate scanning artifacts, including, but not limited to, slew, distortion, and hysteresis.

[0035] Many variations are possible in the design of a suitable primary carrier signal path. FIG. 5 is a graphical representation 500 of a smoothed Hilbert-type space-filling curve path 502 superimposed on an unsmoothed Hilbert-type space-filling curve path 504. The smoothing operation on the smoothed Hilbert-type space-filling curve path 502 represents one variation of the aforementioned phase curve that can be used in embodiments of the sparse sampling system 108. In one embodiment of the sparse sampling system 108, the sparse sampling pattern generator 119 can be programmed to construct a smoothed or otherwise modified primary carrier pattern after the phase and space-filling curves that make up all or a portion of the primary carrier signal communicated to the scan input 106 of the scanning probe instrument to adjust the resulting rate of change of the XY or XYZ pattern. Adjusting the rate of change of the primary carrier pattern is one means of mitigating scanning artifacts such as, but not limited to, slew, distortion, and hysteresis.

[0036] One embodiment of the sparse sampling system 108 utilizes a sparse sampling pattern generator 119 to implement a signal pattern that includes a region of interest (ROI) with varying sparsity within the scan boundary and / or scan grid spacing. FIG. 6 is a graphical representation 600 that includes a scan boundary 602 that surrounds a primary carrier signal path 604 and an ROI 606 that includes a primary carrier signal path 608 scaled relative to the primary carrier path 604. The ROI 606 may represent a region relative to the geometric boundary where a different sample sparsity and / or a different scan grid spacing is desired (e.g., a higher pixel density within the ROI 606). Multiple ROIs may exist within a single scan boundary 602. The graphical representation 600 illustrates the ROI 606 as a simplified visualization that surrounds a primary carrier signal path 608 constructed using a non-uniformly scaled version of the primary carrier signal path 604. However, the primary carrier signal path 608 may be comprised of any suitable type of carrier signal path.

[0037] 7 is a graphical representation 700 of an XY sparse sampling embodiment including a set of randomized sparse sample points defined by a scan boundary 702, a serpentine primary carrier signal path 704, and a set of plot marker types 706. Graphical representation 700 is useful for illustrating basic features common to sparse sampling systems 108. In this example, primary carrier signal path 704 starts at location 708, traverses along primary carrier signal path 704, and completes at location 710. XY coordinate 712 represents any coordinate along primary carrier signal path 704. The area within circular boundary 716 represents the maximum amplitude of the XY secondary modulation signal (not shown), which is referenced to primary carrier XY coordinate 712 located at the geometric center of circular boundary 716 along primary carrier signal path 704. Sparse sample coordinates can be randomly generated within any element of sample grid 714 located within circular boundary 716. The current graphical representation 700 consists of a two-dimensional XY sparse sampling implementation; therefore, the sparse sampling system 108 allows two degrees of freedom in defining possible sparse sample locations within a circular region 714. Assuming a three-dimensional XYZ implementation of the sparse sampling system 108, an analogous representation of the two-dimensional circular region 714 would be a three-dimensional sphere (not shown). For a three-dimensional XYZ implementation of the sparse sampling system 108, there are three degrees of freedom in defining random sparse sampling scan coordinate locations. The degrees of freedom afforded by the sparse sampling system 108 for defining sparse sampling coordinates is an important distinguishing feature compared to other proposed sparse sampling systems.

[0038] Additionally, the graphical representation 700 helps illustrate that one embodiment of the sparse sampling system 108 operates given an initial set of sparse sample coordinates, such as a set of randomized sparse sample points defined by a set of coordinates that match a marker type 706. In embodiments where the sparse sample coordinates are initially given, the sparse sampling pattern generator 119 configures the primary carrier signal path 704 and secondary modulation pattern to conform to the a priori set of sparse sample coordinates.

[0039] The same set of sparse sample coordinates matching the set of plot marker types 706 in the graphical representation 700 can be generated by the sparse sampling system 108 using various primary carrier signal paths. In one embodiment of the sparse sampling system 108, a Hilbert-type primary carrier signal path is used to generate the same set of sparse sample coordinates matching the set of plot marker types 706 in the graphical representation 700 generated using the serpentine primary carrier signal path 704. Another very simple alternative primary carrier signal carrier embodiment for generating the same set of sparse sample coordinates matching the set of plot marker types 706 in the graphical representation 700 is represented by a 90-degree clockwise or counterclockwise rotation of the primary carrier signal path 704.

[0040] The purpose of invoking a particular primary carrier signal path to satisfy a given set of sparsely sampled coordinates includes, but is not limited to, mitigating scanning artifacts associated with sample charging when the target 102 is either insulating or semiconducting. Both the primary carrier signal path and the degree of sparsity affect sample charging in a charged particle scanning probe instrument and can be adjusted using the sparse sampling system 108.

[0041] One embodiment of the sparse sampling system 108 utilizes a carrier signal path created from a sequentially collected list of XY or XYZ coordinates, including a collection of XY or XYZ coordinates generated by a parametric equation. FIG. 8 is a graphical representation 800 of a continuous XY parametric equation for generating a primary carrier signal path 802. An over-scan boundary 804 encompasses the entire scan area, and a target scan boundary 806 (dotted line) represents a subregion of the primary carrier signal path 802. It is common practice for pattern generators employed in scanning probe instruments to incorporate an over-scan region defined by the area between the over-scan boundary 804 and the target scan boundary 806, with the purpose of excluding areas where the scan pattern may be non-ideal for reasons including, but not limited to, non-linear scanning behavior and non-uniform area coverage. The area within the target scan boundary 806 represents a more uniform area compared to the area between the over-scan boundary 804 and the target scan boundary 806. The primary carrier signal path 802 of representation 800 begins at XY coordinate 808, follows the successive primary carrier signal paths 802, and ends at XY coordinate 810. A specific XY parametric equation embodiment for generating the space-filling carrier signal path 802 is a smoothed form of parametric equation 1:

number

[0042] In an embodiment of the sparse sampling system 108 employing a continuous space-filling path, beam blanking may not be required along the scan path or along portions of the scan path. Beam blanking is a common element in charged particle systems that provides a means to nullify, or "blank," probe interactions with the target. Typically, in charged particle instruments, beam blanking components may include electrostatic deflection plates near the top of the column proximate a crossover location in the optical path. The beam blanking action deflects the probe (beam) to a position that prevents it from transmitting through the optical path and interacting with the target. High-speed beam blanking elements are a typical option available in charged particle embodiments of the scanning probe instrument 104, allowing for more rapid beam blanking, which corresponds to a more precise definition of dwell times at each scan coordinate. In other embodiments of the sparse sampling system 108 using one or more discrete or continuous primary carrier signal paths, beam blanking can be utilized as desired to mitigate scanning artifacts and spurious probe interactions with the target 102.

[0043] The sparse sampling system 108 can utilize a wide variety of suitable space-filling carrier signals. Carrier signal patterns can be programmed to generate any combination of continuous and non-overlapping patterns, continuous and non-intersecting patterns, continuous and intersecting patterns, or continuous and overlapping patterns. The sparse sampling system 108 can utilize carrier signal patterns programmed as discrete segments with any discontinuity. For example, this approach can be applied to trace and / or fill multiple distinct geometric regions or spatial features of the object being sensed. A particular example is a carrier signal path and a reference modulation signal path designed to generate a set of sparsely sampled coordinates that trace the path of embedding neurons in a biological matrix. Similarly, rectangular, triangular, circular, or other geometric patterns on the object being sensed can represent sparsely sampled regions.

[0044] The sparse sampling system 108 may utilize a carrier signal pattern constructed from an analytical space-filling curve that is programmatically modified to adjust for the overall sparse sampling pattern and / or pattern generator performance and / or interaction with the scanning analysis system. For example, the transmitted carrier signal pattern and the referenced secondary modulation signal pattern may be smoothed mathematically by software or by hardware to limit the rate of change of the signal so as not to exceed the performance limits of the scanning probe system to avoid scanning artifacts.

[0045] 9 is a schematic diagram 900 of a dual column scanning probe instrument that can include at least some embodiments of the present invention. A scanning electron beam column 902 and a focused ion beam column 904 are oriented so that there is a coincidence region between the scan region of the electron beam 946 and the scan region of the ion beam 948. A surface area of ​​the target 944 is shown orthogonal to the ion beam 948. In one embodiment, the target 944 is fixed to a movable stage (not shown) that allows movement in XYZ as well as rotation and tilt stages with a range sufficient to orient the surface area of ​​the target 944 orthogonal to the electron beam 946 or ion beam 948. The illustrated scanning electron beam column 902 includes an electron source 906, an extraction electrode 908, an anode 910, an electromagnetic collimating lens system 912, a spray aperture 914, an in-lens objective response signal detector 916, an electromagnetic lens coil body 918, an outer pole piece 920, an inner pole piece 922, electrostatic objective lens electrodes 924, 928, and a scanning probe coil 926. The focused ion column 904 includes an ion source 930, an extraction electrode 932, a condenser lens 934, a variable aperture 936, electrostatic deflection electrodes 938, 940, and an objective lens 942.

[0046] 8 is comprised of at least two pairs of XY compound signal converters 110 for simultaneously driving the scan coil 926 of the scanning electron column 902 and the scan deflection electrodes 938, 940 of the focused ion column 904. One embodiment of the sparse sampling system 108 includes an object-responsive detector 126 configured as a secondary ion detector and an object-responsive detector 126 configured as a backscattered electron detector operating simultaneously with the sparse sampling system 108. The additional object-responsive detectors 126 in the dual column scanning probe instrument 900 can include, but are not limited to, an in-lens secondary electron detector, an in-chamber secondary electron detector, an in-chamber backscatter detector, a secondary ion conversion detector, a fluorescence detector, an X-ray detector, a time-of-flight secondary ion mass spectrometer, an electrostatic electromagnetic mass spectrometer, and a quadrupole mass spectrometer.

[0047] In one embodiment of the sparse sampling system 108, multiple XY discrete layers or thin sections of the object being probed are sparsely sampled. The sparse sampling locations from each layer are programmatically randomized to generate a randomized sparse sampling volume in three dimensions, X, Y, and Z. For example, the sparse sampling system 108 drives the scanning deflection electrodes 938 and 940 to generate an ion milling process with the focused ion column 904 to expose a surface layer (e.g., also referred to as a slice or section defined by the probe-object interaction volume) of the new object 944. Simultaneously with or after the ion milling process, the sparse sampling system 108 drives the electron beam column 902 to obtain a sparsely sampled integrated representation of the surface area of ​​the new object 944 using one or more object-responsive detectors 126, with each sparsely sampled XY scan having a unique randomized sparse sampling pattern. The process of generating a new surface with the ion beam column 904 using the sparse sampling system 108 and acquiring sparse sampling with the electron beam column 902 is repeated to produce a stack of XY integrated representations generated by the image reconstruction system 120. In this embodiment, the sparse sampling is extended from two dimensions to three dimensions, and the maximum percentage of sparseness allowed for a successful image reconstruction system 120 is much higher than the maximum sparseness allowed for the corresponding two-dimensional individual layer. For example, if 90% sparseness is the maximum sparseness that produces an acceptable integrated representation using the image reconstruction system 120 for each individual XY scan layer, then 97% or more sparseness can produce an acceptable integrated representation using the image reconstruction system 120 from the same XY scan layer when processed as a randomized three-dimensional XYZ stack. Each depth layer signal pattern includes a programmably unique randomized XY sparse sampling pattern to optimize the sparse sampling processed by the image reconstruction system 120 as an XYZ layer stack, resulting in a sparser integrated reconstruction than could be obtained from each layer individually.

[0048] In one embodiment of the sparse sampling system 108, the sparse sampling operation may be repeated successively over the same region using the same sparse sampling pattern generated from 119 for each successive scan, or using a uniquely randomized pattern generated from 119 for each successive scan pattern, or any combination thereof. This embodiment is utilized, for example, to obtain successive XY scans during continuous or semi-continuous sensing of the sparsely sampled object 102.

[0049] In one particular embodiment of the sparse sampling system 108, the entire sparse sampling process, from the sparse sampling pattern generator 119 to the image reconstruction system 120, operates continuously and repeatedly as rapidly as the combined systems allow. Alternatively, the entire sparse sampling system 108 operates with a discrete time delay. Using the highest possible operating rate of the sparse sampling system 108, or a discrete delay operation of the sparse sampling system 108, the object 102 can be observed continuously or semi-continuously while it is being sparsely sampled. Observations of the object 102 being sparsely sampled may include, but are not limited to, changes due to mechanical movement (e.g., clockwork or gear movement) of all or a portion of the object being sensed, modifications induced by the action of a separate probe (e.g., a micromanipulator, laser ablation, focused ion beam, or broad beam ion milling), changes induced by an energy source (e.g., heating, cooling), changes due to chemical interaction with all or a portion of the object being sensed, or any combination thereof. Advantages of sparsely sampled observation of the object 102 during such changes include reduced sensing probe interaction with the object 102 being sensed (e.g., reduced electron dose, reduced sample charging) and increased signal acquisition speed of the object response detector 126 during near real-time observation.

[0050] The pattern of successive object response signal detectors 126 transmitted through the object response signal transducers 122 during sparsely sampled observation of the object 102 is utilized in one embodiment of the sparse sampling system 108 to reconstruct an integrated representation through the image reconstruction system 120, providing a record of changes over the observation period. For example, image reconstructions obtained from observations of the object 102 while it is being ion milled or mechanically deformed form a three-dimensional volume representation from a stack of reconstructed XY integrated representations. Alternatively, the image reconstruction system 120 can generate a single XYZ volume integrated representation from the entire three-dimensional sparse data array or from one or more three-dimensional array sparse data blocks consisting of a subset of the entire sparse data array. An exemplary embodiment is a tomographic reconstruction of a volume based on a three-dimensional array of sparsely sampled data, where the object response detector 126 is a backscattered electron detector used to acquire sparsely sampled object response signal transducer 122 data acquired from sequential focused ion beam milling or during simultaneous focused ion beam milling to generate volumetric object 102 response signal image data.

[0051] In one embodiment of the sparse sampling system 108, the sparse sampling percentage is adapted to reflect changes in the geometry or material properties of the object 102 under observation.

[0052] One embodiment of the sparse sampling system 108 utilizes a carrier signal path, a reference set of modulation signal coordinates, and a degree of sparsity to tailor analytical objectives based on data from object response detector 126 signals derived from sparsely sensed objects 102. The analytical objectives may include, but are not limited to, response signal strength (e.g., adjusting dwell time) and / or response signal spatial resolution (e.g., adjusting sparsity).

[0053] In one embodiment of the sparse sampling system 108, the encoding and indexing of the ADC signal data synchronized through a programmable logic controller can be compressed relative to the full sample signal data to save significant digital storage memory by storing the data in an ordered list rather than as an object array.

[0054] In one embodiment of the sparse sampling system 108, a fast Fourier transform (FFT) is calculated from the integrated representation from the image reconstruction system 120 using an object response signal converter 122 signal conveyed by an object response detector 126 signal derived from the sparsely sensed object 102. In a further embodiment of the sparse sampling system 108, the FFT calculated from the integrated representation is utilized in an autofocus and / or autoastigmatism correction method.

[0055] One embodiment of the sparse sampling system 108 generates an XY scan pattern that drives the scanning deflection electrodes 938 and 940 of the focused ion beam column 904 at a sparseness that corresponds to an ion dose below the static secondary ion mass analysis limit.

[0056] In one embodiment of the sparse sampling system 108, the sparse sampling pattern generator 119 incorporates scan distortion correction to correct for scan distortion errors inherent in both the scanning probe instrument and the sparse sampling system 108. Scan distortion errors resulting from both the scanning probe instrument and the sparse sampling system 108 are measured using appropriate geometric reference standards. The measured scan distortion error corrections are mapped back to the sparse sampling pattern generator 119 to generate a signal pattern with minimal scan distortion. Scan distortion errors exhibited by a particular scanning probe instrument 104 and sparse sampling system 119 can be compensated for in this manner such that a corrected scan is output by the sampling scan pattern generator 119. The described a priori distortion correction eliminates the need for post-processing corrections of the reconstructed integrated representation of the object 102.

[0057] One embodiment of the sparse sampling system 108 includes a point spread function deconvolution (PSFD) operation as part of the image reconstruction system 120 to combine sparse sampling reconstruction with PSFD. In this embodiment, the spatial resolution of the integrated representation is improved by including a PSFD operation, where PSF is a measurement or theoretical function corresponding to the scanning probe instrument 102.

[0058] 1-9 primarily relates to programmable XY scan pattern generation. However, as briefly mentioned above, the principles of the present disclosure also apply to programmable XYZ scan pattern generation. Embodiments related to programmable XYZ scan pattern generation are described in detail below.

[0059] In one embodiment, the sparse sampling system 108 of FIG. 1 is configured as an XYZ pattern generator, and the Z coordinate of the scan pattern coordinates of the object scan for each XYZ set of coordinates can be defined by operation of a set of rules of the form: i) a Z coordinate defined by a third programmable primary carrier signal and a third secondary programmable signal modulating the third primary carrier signal with a third randomized modulation, such as modulation along a carrier signal path of a three-dimensional space-filling curve, an example of a specific carrier signal path being a three-dimensional Hilbert space-filling curve 1200 as graphically represented in FIGS. 12A-12D; and ii) a Z coordinate of a set of scan pattern coordinates over the object scan region defined as discretized on at least two predefined Z-plane coordinates, as graphically represented in FIGS. iii) Z-coordinate scan pattern coordinates for each XYZ set of scan pattern coordinates for the object scan 1100 defined by randomized modulation from Z coordinates on at least two predetermined Z planes in either the positive or negative Z direction, as graphically represented in Figures 11A-11D. iv) Z locations are randomly distributed in either the positive or negative Z direction for each XYZ set of scan pattern coordinates for the object scan.

[0060] Reiterating from above, the sparse sampling system composite signal converter 110 is expandable to “N” signal converter elements. For example, the composite signal converter 110 includes primary, secondary, and tertiary signal converter elements. The sparse sampling system 108 is expandable as a triad of composite signal converters to form an XYZ pattern generator. The XYZ pattern generator configuration in the context of the present disclosure further indicates that the sparse sampling system 108 has the ability to generate signals via the sparse sampling pattern generator 119 and send predefined conditioned signals to the scan input 106 of the scanning probe instrument 104 via the controller 116 to adjust and control object scan signals corresponding to the X, Y, and Z directions of the scanning probe instrument 104, for example, using a rectangular reference frame. Generally, as previously mentioned, the sparse sampling system 108 incorporates a controller 116 (e.g., an FPGA) that stores the sequence of transmit signals generated by the sparse sampling pattern generator 119 and coordinates the sequential transmission of the X, Y, and Z output signals to the scanning probe instrument's scan input 106 via appropriate X, Y, and Z signal converters (e.g., DACs). It should be noted that embodiments of the present disclosure may be equivalently expressed in polar and / or spherical coordinates.

[0061] An embodiment of the present disclosure for a sparse sampling system 108 configured as an XYZ pattern generator describes specific ways in which Z coordinates can be derived to define a three-dimensional sparse or partially subsampled pattern according to a set of rules i) through iv). Rule i) is an extension of the XY modulation of a two-dimensional space-filling curve to an XYZ modulation along the path of a three-dimensional space-filling curve. Rule ii) discretizes the Z coordinate positions, which reduces scanning artifacts but may result in a less random sparse sampling matrix than rule i). Rule iii) combines discrete planes with modulation of each plane, which reduces scanning artifacts while providing improved statistical randomness in the sparse sampling matrix compared to rule ii). Rule iv) is the most common, in which the Z coordinate is defined by randomized values ​​within the range of the Z direction of the object scan. However, in a real physical system, rule iv) may limit the range of the Z direction random modulation to reduce scanning artifacts. Furthermore, random values ​​drawn from the entire range of possible Z coordinate values ​​may result in both large and small deviations from the previous Z coordinate. Large deviations in successive Z coordinate values ​​may exacerbate artifacts associated with dynamic systems, such as slew and hysteresis, while limiting the dynamic response of the system. By implementing a random modulation scheme on a discrete plane according to rule iii) or along a three-dimensional space-filling path according to rule i), the Z coordinate has random conditions localized to the plane or three-dimensional path defined by the degree of the modulation parameter.

[0062] The form of rule i) generates a carrier modulator signal pair for the Z coordinate in the same manner as it is applied to generate a carrier modulator signal pair for the XY coordinate in an embodiment of the sparse sampling system 108 configured as an X-Y two-dimensional pattern generator. This embodiment extends modulation along an XY space-filling curve carrier signal to modulation of an XYZ carrier signal along a three-dimensional space-filling curve, as graphically represented in FIGS. 12A-12D. FIG. 12A is a diagram of an object scan 1200 along the Z direction 1006 (labeled in FIG. 12B). FIG. 12B is a diagram of an object scan 1200 along the X direction 1002 (labeled in FIG. 12A). FIG. 12C is a diagram of an object scan 1200 along the Y direction 1004 (labeled in FIG. 12A), and FIG. 12D is a trimetric XYZ perspective view of the object scan 1200. The carrier signal does not explicitly correspond to any coordinate of the object scan, but the object scan coordinates may randomly correspond to one or more components of the carrier signal. The carrier signal generally determines how the scanning probe moves through the object scan area and can be thought of as a guiding center around which the randomized modulation is generated.

[0063] In one embodiment of rule i), one or more continuous portions of the three-dimensional space-filling curve of the object scan are subdivided into two or more three-dimensional subregions. Random modulations along the continuous portions of the three-dimensional space-filling curve are generated within each subregion. In general, each subregion of the object scan can be generated with a different degree of sparseness. The ability to predefine the degree of sparseness specific to each subregion represents an adaptive strategy for adjusting the sparseness or partial subsampling distribution of the object scan. However, for object scans with uniform sparseness across the entire object scan region, the degree of sparseness generated in each predefined subregion is the same for each subregion. The advantage of subdividing the object scan into two or more subregions is that it adjusts the local randomness of the sparse or partial subsampling signal defined by the sparse sampling pattern generator 119, even if the sparseness is the same for each subregion. The subregions may be linear, non-linear, or irregularly shaped.

[0064] Within each subregion, the random generator seed value can be fixed or random. A random generator initialized with the same fixed seed value for modulating the carrier signal component of an object scan will reproduce the same randomized sparse coordinates given the same carrier signal space-filling curve (e.g., 2D Hilbert), the same object scan dimensions (the scan pattern of the object scan has the same deflection or physical dimensions mapped to the object scan in each direction), the same size and shape of subregions, the same number of addressable pixels within each subregion (e.g., 4096 x 4096), and the same degree of sparseness within each subregion (e.g., 80%).

[0065] In an embodiment of the present disclosure where the random modulation of the object scan carrier signal is subdivided into two or more sub-regions, a fixed seed value determines the initial value of the random generator sequence.

[0066] In one embodiment of the present disclosure, a random generator initialized with a random seed value to modulate carrier signal components generates object scans with non-repeating sets of randomized sparse coordinates, even given the same carrier signal space-filling curve, the same object scan dimensions, subregions of the same size and shape, the same number of addressable pixels within each subregion, and the same degree of sparsity within each subregion (e.g., 80%).

[0067] In an embodiment of the present disclosure where the random modulation of the object scan carrier signal is subdivided into two or more sub-regions, the random seed value determines the initial value of the random generator sequence.

[0068] A form of rule ii) generates a fixed, predetermined set of Z-planes, as graphically represented in Figures 10A-10D. Figure 10A is a diagram of an object scan 1000 along the Z direction 1006 (labeled in Figure 10B). Figure 10B is a diagram of the object scan 1000 along the X direction 1002 (labeled in Figure 10A). Figure 10C is a diagram of the object scan 1000 along the Y direction 1004 (labeled in Figure 10A), and Figure 10D is a trimetric XYZ perspective view of the object scan 1000. The volume of the object scan 1000 is defined by the dimensions in the X direction 1002, the Y direction 1004, and the Z direction 1006. In the embodiment shown in Figures 10A-10D, 80% sparsity is discretely distributed in each of five Z-planes 1008, with two of the five planes overlapping the object scan boundary. In one embodiment, each Z plane 1008 in FIG. 10 represents a 50×50 array of addressable pixels, with 20% of the addressable pixels in the 50×50 array being subsampled on each Z plane 1008.

[0069] The minimum and maximum object scan dimensions depend in part on the scanning probe instrument. For example, a common standard for an XY object scan range on an SEM or scanning transmission electron microscope (STEM) corresponds to ±10 volts sent to the analog input of the scanning deflection coil via circuitry associated with the magnification range. Depending on the magnitude of the voltage, the voltage signal can correspond to physical dimensions on the object greater than a few millimeters or even submicron. A discrete Z plane in the case of an SEM / STEM can be operated, for example, by sending a Z coordinate generated by the sparse sampling pattern generator 119 to an input to the objective lens assembly of the scanning probe instrument 104, adjusted via the controller 116, to set the focal length to a value corresponding to the physical Z plane of the object scan. However, scanning probe instruments can also be configured to generate object scans using a static probe (e.g., a laser probe) coupled to an XYZ stage.

[0070] Expressed in generalized terms, the predetermined rule ii) includes Z coordinates of a set of scan pattern coordinates of the object scan that are discretized and defined on at least two predefined Z planes such that different non-overlapping subsets of Z coordinates lie on different Z planes of the at least two predefined Z planes, and therefore the same Z coordinate of the object scan may not appear on more than one discrete plane.

[0071] In one embodiment of the form of rule ii) depicted in FIG. 10, the object scan 1000 is sparsely or partially subsampled within each discrete Z plane 1008 such that each Z plane 1008 contains a unique set of XY-addressable pixel locations.

[0072] Expressed in general terms, in one embodiment of the form of rule ii), the X and Y coordinates form XY coordinate pairs, and each different predefined Z-plane of the at least two different predefined Z-planes comprises a different non-overlapping subset of the XY coordinate pairs.

[0073] There may be certain conditions where no XY coordinates are repeated in any of the discrete planes, i.e., for an equal number of addressable pixels in each discrete plane, the sum of the partial addressable pixels may not exceed the total number of addressable pixels in a single discrete plane. As an example of a variation of this embodiment, any one of the five Z planes 1008 may have a 40% proportion of addressable pixels, any two of the five discrete Z planes may have a 20% proportion of addressable pixels, and any two of the five discrete Z planes may have a 10% proportion of addressable pixels, with the number of addressable pixels being the same in each of the five Z planes. The total number of subsampled addressable pixels will be 100% of the possible addressable pixels in a single plane. This embodiment can be easily extended to "W" Z planes, where "W" is a positive integer greater than 1.

[0074] In another embodiment of the form of rule ii), the subsampled partially addressable pixels from any discrete Z-plane can vary from greater than 0% up to 100% of the possible addressable pixels. In this particular embodiment, the XY coordinates can be repeated on one or more discrete Z-planes. Varying the scan sparsity on each discrete pre-defined Z-plane represents an adaptive strategy for improving the sparse or partially subsampling distribution of the object scan. For example, the non-uniformity of the object can vary, or it is desirable to limit scanning probe interactions (e.g., object damage due to electron beam dose) by varying the sub-region sparsity of the object scan.

[0075] In one embodiment of a sparse sampling system 108 configured as an XYZ pattern generator where the Z coordinate is provided by rule ii) and implemented in an optical scanning probe instrument, the focus state defining the Z plane can be operated via a computer controlled process that uses the DAC output of the scanning system described by this disclosure.

[0076] In one embodiment of the sparse sampling system 108 configured as an XYZ pattern generator where the Z coordinate is given by rule ii) and implemented in an optical scanning probe instrument, the focus state defining the Z plane can be operated via a computer-controlled process using an application programmer interface (API) module.

[0077] In one embodiment of a form of rule ii), the predefined Z planes are not equidistant in the Z direction. This embodiment represents an adaptive strategy for improving the sparse or partial subsampling distribution of the object scan.

[0078] In one embodiment, one or more Z-planes have different numbers of addressable pixels. For example, one Z-plane may be represented by 1024 x 1024 addressable pixels, while another Z-plane of the same object scan may be represented by 4096 x 4096 addressable pixels. The ability to vary the number of addressable pixels in each pre-defined discrete Z-plane via this embodiment represents an adaptive strategy for improving the sparse or partial sub-sampling distribution of the object scan.

[0079] The form of rule iii) graphically represented in Figures 11A-11D randomly modulates the Z coordinates of the object scan discrete Z planes in either the positive or negative Z direction. Figure 11A is a diagram of an object scan 1100 along the Z direction 1006 (labeled in Figure 11B). Figure 11B is a diagram of the object scan 1100 along the X direction 1002 (labeled in Figure 11A). Figure 11C is a diagram of the object scan 1100 along the Y direction 1004 (labeled in Figure 11A), and Figure 11D is a trimetric XYZ perspective view of the object scan 1100. The XY coordinates of the subsampled object scans in Figures 11A-11D are identical to the XY coordinates of the subsampled object scans shown in Figures 10A-10D. The Z coordinates of the sub-sampling in Figures 11A-11D are randomly modulated in the positive or negative Z direction in each of the finite Z planes represented in Figures 10A-10D to improve randomization of the object scan.

[0080] In one embodiment of rule iii), the magnitude of the Z-direction modulation is limited to no more than half the distance to the next discrete Z plane in either Z direction. By limiting the magnitude of the random modulation to no more than half the distance to the next discrete Z plane, each Z plane has the same number of addressable pixels, reducing the chance of overlapping XYZ coordinates. The number and relative separation of the predetermined Z planes may be based on prior knowledge of the object structure, such as varying density or vergence along the Z direction of features of interest.

[0081] A practical application of rules ii) and iii) and their associated embodiments relates to the ease with which sparse or partial sub-sampling can be implemented using a scanning probe instrument. For example, in the particular case of STEM, the XY coordinates of a scan pattern for an object scan are generally determined by the deflection of the scanning probe (e.g., an electron beam deflected by a scanning coil). In particular embodiments, the Z coordinate position of the XYZ scan coordinates of the scan pattern for an object scan can be affected via the sparse sampling system 108 by sending a signal to focus the scanning probe, thereby concentrating the scanning probe area to correspond to a particular Z plane on the object.

[0082] In one embodiment of the sparse sampling system 108 configured as an XYZ pattern generator where the Z coordinate is provided by rules ii) and iii) and implemented in an optical scanning probe instrument, the discrete Z planes can be operated by an automated computer-controlled process.

[0083] In one embodiment of the sparse sampling system 108 configured as an XYZ pattern generator implemented in an optical scanning probe instrument, where the Z coordinate is provided by rules ii) and iii), the discrete Z planes can be manipulated by manual focus manipulation.

[0084] In one embodiment of the sparse sampling system 108 configured as an XYZ pattern generator, the Z coordinate may be provided by the motion of the scanning stage.

[0085] In one embodiment of the sparse sampling system 108 configured as an XYZ pattern generator, the Z coordinate may be provided by the action of a combination of a scanning stage and a lens system.

[0086] In the particular case of laser scanning confocal microscopes (LSCMs) and similar scanning stage instruments, the XYZ coordinates of the scan pattern of the object scan can be acted upon by signals transmitted by the sparse sampling system 108 at the input of an XYZ stage, for example a piezoelectric stage with nanometer or sub-nanometer step size resolution.

[0087] In one embodiment of the present disclosure according to rules ii) and iii), the discrete planes may be uniformly distributed along the Z dimension of the object scan.

[0088] In an embodiment of the present disclosure according to rules ii) and iii), the discrete Z-planes may be non-uniformly distributed. The subjective locations of the pre-defined discrete planes represent an adaptive strategy to improve the sparse or partial sub-sampling distribution of the object scan.

[0089] In one embodiment of rules ii) and iii), the XYZ coordinates can define a non-repeating set of XY positions distributed across each discrete Z plane. For example, an 80% sparsely sampled 4096x4096 scan allows for up to five discrete Z planes, each containing 819x819 XY pixels, with each discrete Z plane consisting of a unique, non-repeating sequence of XY pixel locations within the 4096x4096x5 sparse data cube. Thus, discrete Z plane 2 does not contain identical XY coordinates from Z plane 1, discrete Z plane 3 does not contain duplicate XY coordinates from Z planes 1 or 2, discrete Z plane 4 does not contain duplicate XY coordinates from Z planes 1, 2, or 3, and discrete Z plane 5 does not contain duplicate XY coordinates from Z planes 1, 2, 3, or 4. This particular embodiment is shown in Figures 10A-10D using a 50x50x5 pixel array for illustrative purposes.

[0090] In one embodiment of rules ii) and iii), two or more sets of sparsely sampled XY coordinates of an object scan can be sequenced or mixed in a predetermined or random manner to generate a set of predefined Z-planes. For example, a set of 80% sparse XY coordinates of an object scan can form a set of up to five Z-planes with unique, non-repeating XY coordinates, which may be repeated once to form ten Z-planes, or twice to generate fifteen Z-planes, etc.

[0091] A form of Rule iv) randomly modulates the Z coordinate of the scan pattern coordinates of the object scan across the Z dimension of the object scan. In one embodiment of the form of Rule iv), the first and second primary carrier signals comprise X and Y components of a two-dimensional space-filling curve, the first and second secondary X and Y signals randomly modulate the primary carrier signals to define the X and Y coordinates of the object scan, and the Z coordinate of the object scan is randomly modulated in the positive or negative Z direction. Rule iv) can be considered a specific combination of Rules ii) and iii), where the number of discrete planes is reduced to one. A practical application of Rule iv) is when the dimension of the object scan in the Z direction is sufficiently small and / or the scanning method in the Z direction is well responsive, so that the modulation does not introduce artifacts, including but not limited to slew, hysteresis, and position error. For example, in the case of rule iii), a predetermined number of Z planes equal to 5 will produce a faithful reconstructed representation of the object, and an object scan of another similar object but with 1 / 5 the Z dimension can be reconstructed from data acquired using the sparse sampling system 108 by rule iv).

[0092] In one embodiment of the sparse sampling system 108 configured as an XYZ pattern generator, the Z coordinates of rules i) through iv) may be effected by the operation of a lens system.

[0093] In one embodiment of the sparse sampling system 108 configured as an XYZ pattern generator, rules ii), iii) and iv) are implemented in an optical scanning probe instrument, and the Z coordinate is provided by the operation of a lens system to focus the probe on a specified Z plane or can modulate the Z coordinate through overfocus (below the plane) and / or underfocus (above the plane).

[0094] In one embodiment of the sparse sampling system 108 configured as an XYZ pattern generator, the lens system may be an optical optical system such as a laser scanning confocal microscope (LSCM).

[0095] In one embodiment of the sparse sampling system 108 configured as an XYZ pattern generator, the lens system may be an electrostatic, electromagnetic, or combined electrostatic-electromagnetic lens system, including a STEM lens system.

[0096] The illustrations of the embodiments described herein are intended to provide a general understanding of the structure of various embodiments. These illustrations are not intended to serve as a complete description of all of the elements and features of apparatus and systems that utilize the structures or methods described herein. Many other embodiments may become apparent to those skilled in the art upon reviewing this disclosure. Other embodiments may be utilized and derived from the present invention, such that structural and logical substitutions and changes may be made without departing from the scope of the invention. Moreover, these illustrations are merely representational and may not be drawn to scale. Certain proportions in the illustrations may be exaggerated, and other proportions may be reduced. Accordingly, the disclosure and the figures should be considered illustrative and not limiting.

[0097] One or more embodiments of the present disclosure may be referred to herein individually and / or collectively by the term "invention" merely for convenience and without any intention to limit the scope of the present application to any particular invention or inventive concept. Furthermore, although specific embodiments have been illustrated and described herein, it is to be understood that any subsequent configurations designed to achieve the same or similar purpose may be substituted for the specific embodiment shown. The present disclosure is intended to cover any and all subsequent adaptations or variations of the various embodiments. Combinations of the above embodiments, and other embodiments not specifically described herein, will be apparent to those skilled in the art upon reviewing the description.

[0098] The Abstract is provided to comply with 37 CFR § 1.72(b) and is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. Additionally, in the foregoing Detailed Description, various features are grouped together or described in a single embodiment for the purpose of streamlining the disclosure. The invention is not to be interpreted as reflecting an intention that the claimed embodiments employ more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive subject matter may be directed to less than all features of any of the disclosed embodiments.

[0099] The subject matter disclosed above should be considered illustrative and not limiting, and the appended claims are intended to cover all such modifications, enhancements, and other embodiments that fall within the true spirit and scope of the invention. Accordingly, to the maximum extent permitted by law, the scope of the invention should be determined by the broadest permissible interpretation of the following claims and their equivalents, and should not be restricted or limited by the foregoing detailed description.

Claims

1. 1. A system comprising: a sparse sampling system having a processor and a non-transitory computer-readable medium, the sparse sampling system comprising: a first programmable primary carrier signal and a first secondary programmable signal modulating the first programmable primary carrier signal via a first randomized modulation that defines an X coordinate of a set of scan pattern coordinates for the object scan; a second programmable primary carrier signal and a second secondary programmable signal modulating the second programmable primary carrier signal via a second randomized modulation that defines a Y coordinate of the set of scan pattern coordinates for the object scan; a third programmable output signal that defines a Z coordinate of the set of scan pattern coordinates for the object scan based on a predetermined rule; and configured to generate The sparse sampling system is further configured to transmit the generated first and second programmable primary carrier signals, the first and second secondary programmable signals, and the third programmable output signal to a scanning probe instrument.

2. 2. The system of claim 1, wherein the X and Y coordinates of the set of scan pattern coordinates of the object scan sample less than 100% of the addressable pixels of a given set of the object scan.

3. 2. The system of claim 1, wherein the X, Y, and Z coordinates of the set of scan pattern coordinates of the object scan sample less than 100% of the addressable pixels of a given set of the object scan.

4. 10. The system of claim 1, wherein each of the first and second programmable primary carrier signals comprises an XY component of a two-dimensional or three-dimensional space-filling curve.

5. the first programmable carrier signal is divided into different sub-portions, each of the different sub-portions being modulated by the first secondary programmable signal via the first randomized modulation to provide a different subset of X-coordinates that together define the X-coordinate of the set of scan pattern coordinates for the object scan; and 2. The system of claim 1, wherein the second programmable carrier signal is divided into different sub-portions, each of the different sub-portions being modulated by the second secondary programmable signal via the second randomized modulation to provide a different subset of Y coordinates that together define the Y coordinate of the set of scan pattern coordinates for the object scan.

6. 2. The system of claim 1, wherein the predetermined rule includes defining the Z coordinate of the set of scan pattern coordinates for the object scan by a third programmable primary carrier signal and a third secondary programmable signal that modulates the third programmable primary carrier signal via a third randomized modulation, the third programmable primary carrier signal and the third secondary programmable signal comprising the third programmable output signal.

7. the first programmable carrier signal is divided into different sub-portions, each of the different sub-portions being modulated by the first secondary programmable signal via the first randomizing modulation to provide a different subset of X-coordinates that together define the X-coordinates of the set of scan pattern coordinates for the object scan; the second programmable carrier signal is divided into different sub-portions, each of the different sub-portions being modulated by the second secondary programmable signal via the second randomizing modulation to provide a different subset of Y-coordinates that together define the Y-coordinate of the set of scan pattern coordinates of the object scan; and 7. The system of claim 6, wherein the third programmable carrier signal is divided into different sub-portions, each of the different sub-portions being modulated by the third secondary programmable signal via the third randomizing modulation to provide a different subset of Z coordinates that together define the Z coordinate of the set of scan pattern coordinates for the object scan.

8. 7. The system of claim 6, wherein the third programmable primary carry signal comprises a Z component of a three-dimensional space-filling curve.

9. The system of claim 8 , wherein the three-dimensional space-filling curve comprises a three-dimensional Hilbert space-filling curve.

10. 2. The system of claim 1, wherein the predetermined rule includes the Z coordinates of the set of scan pattern coordinates of the object scan defined to be discretized on at least two predefined Z planes such that different non-overlapping subsets of the Z coordinates lie on different Z planes of the at least two predefined Z planes.

11. 11. The system of claim 10, wherein the X and Y coordinates form X-Y coordinate pairs, and each different pre-defined Z-plane of the at least two different pre-defined Z-planes comprises a different non-overlapping subset of the X-Y coordinate pairs.

12. 2. The system of claim 1, wherein the predetermined rule includes defining Z coordinates to be randomly modulated in the positive or negative direction with respect to at least two predefined Z planes such that a first non-overlapping subset of the Z coordinates are randomly modulated in the positive or negative direction with respect to a first predefined Z plane of at least two predefined Z planes and a second non-overlapping subset of the Z coordinates are randomly modulated in the positive or negative direction with respect to a second predefined Z plane of the at least two predefined Z planes, wherein an amplitude of modulation is less than or equal to half of a distance between the first predefined Z plane and the second predefined Z plane.

13. 2. The system of claim 1, wherein the predetermined rule includes defining the Z coordinate of the set of scan pattern coordinates of the object scan by a third programmable primary carrier signal and a third secondary programmable signal that modulates the third programmable primary carrier signal via a third randomized modulation in a positive or negative Z direction.

14. 2. The system of claim 1, wherein the sparse sampling system comprises a controller and a signal converter coupled to the controller, the signal converter comprising at least one digital-to-analog converter (DAC), and the controller configured to provide the first and second programmable primary carrier signals and the first and second secondary programmable signals to the signal converter.

15. 2. The system of claim 1, wherein the sparse sampling system comprises a controller configured to provide the first and second programmable primary carrier signals and the first and second secondary programmable signals to digital signal inputs on the scanning probe instrument.

16. 2. The system of claim 1, wherein the Z coordinate of the set of scan pattern coordinates for the object scan is defined by a third programmable primary carrier signal and a third secondary programmable signal that modulates the third programmable primary carrier signal via a third randomized modulation, the sparse sampling system comprising a controller and a signal converter coupled to the controller, the signal converter comprising at least one digital-to-analog converter (DAC), and the controller configured to provide the third programmable primary carrier signal and the third secondary programmable signal to the signal converter.

17. The system of claim 1 , wherein the sparse sampling system comprises a controller configured to provide a signal defining a Z coordinate of the scan pattern coordinate of the object scan to a digital signal input of the scanning probe instrument.

18. generating, with a sparse sampling system having a processor and a non-transitory computer readable medium, a first programmable primary carrier signal and a first secondary programmable signal that modulates the first programmable primary carrier signal via a first randomized modulation that defines an X coordinate of a set of scan pattern coordinates for scanning the object; generating, with the sparse sampling system, a second programmable primary carrier signal and a second secondary programmable signal that modulates the second programmable primary carrier signal via a second randomized modulation that defines a Y coordinate of the set of scan pattern coordinates for the object scan; generating, by the sparse sampling system, a third programmable output signal that defines a Z coordinate of the set of scan pattern coordinates of the object scan based on a predetermined rule; transmitting the first and second programmable primary carrier signals, the first and second secondary programmable signals, and the third programmable output signal generated by the sparse sampling system to a scanning probe instrument; A method comprising:

19. a first programmable primary carrier signal and a first secondary programmable signal modulating the first primary carrier signal via a first randomized modulation that defines an X coordinate of a set of scan pattern coordinates for scanning an object; a second programmable primary carrier signal and a second secondary programmable signal modulating the second primary carrier signal via a second randomized modulation that defines a Y coordinate of the set of scan pattern coordinates for the object scan; a third programmable output signal that defines a Z coordinate of the set of scan pattern coordinates for the object scan based on a predetermined rule; and a sparse sampling pattern generator configured to generate a sparse sampling pattern a controller coupled to the sparse sampling pattern generator and configured to provide the first and second programmable primary carrier signals, the first and second secondary programmable signals, and the third programmable output signal to a scan input of a scanning probe instrument configured to scan an object; at least one object signal response converter connected to the controller and configured to convert analog scanning response signals into digital scanning response signals and configured to receive scanning response signals from at least one object response detector that detects a response of the object to scan a probe signal directed at the object by the scanning probe device; a sampling and reconstruction system communicatively connected to the controller and configured to receive the digital scanning response signals from the controller and responsively reconstruct an integrated image of the object scanned by the scanning probe instrument from sampled object response signals; An apparatus comprising:

20. 20. The apparatus of claim 19, wherein the sampling reconstruction system is configured to reconstruct the integrated image using adaptive real-time inpainting.

21. a sampling reconstruction system having a processor and a non-transitory computer-readable medium, the sampling reconstruction system comprising: receiving a set of subsampled addressable pixel elements; and Reconstructing missing pixels from the set of subsampled addressable pixel elements by adaptive real-time inpainting. The system is configured as follows:

Images