Automated region selection for auto-sweep

Abstract

To provide improved approaches for selection of ROIs for use in automated focus and astigmatism control.SOLUTION: Sample regions of interest (ROIs) for use in autofocus procedures are identified based on a gradient image of the sample. ROIs with gradient values greater than a threshold are selected, and eigenvalues of the associated image matrices are determined. ROIs with suitable variation in eigenvalues such as at least two relatively large eigenvalues are associated with high contrast features orientated in multiple directions so that such ROIs are suitable for automatic focus and astigmatism correction. Suitable ROIs can also be identified based on a histogram of gradient orientations.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to autofocus and astigmatism control in charged particle beam imaging. [Background technology]

[0002] In charged-particle beam (CPB) imaging systems, automated procedures are available for the adjustment of focus and astigmatism of the optical column. In these methods, it is important to sweep the CPB over a suitable region of interest (ROI). In conventional approaches, the entire image field of view (FOV) is used, which can slow down the execution of automatic focus and astigmatism correction, which is particularly problematic when repeated execution of these automatic functions is required. In other approaches, ROIs that are suitable for use with the automatic functions are selected by the operator. With this manual approach, it is difficult to achieve consistency between users. In addition, the manual selection of ROIs is preferably performed by a user with training and experience in imaging physics. If the sample rotates and / or moves, or the stage drifts, it is likely that the suitable ROI is no longer swept for use by the automatic procedure and additional manual adjustments are required. For at least these reasons, an improved approach is needed for the selection of ROIs for use in automatic focus and astigmatism control. Summary of the Invention

[0003] A method and apparatus are disclosed that allows for the selection of suitable regions of interest (ROIs) for use in autofocus and astigmatism correction. To identify candidate ROIs, a gradient-based image is generated from a sample image. The candidate ROIs are evaluated based on eigenvalues ​​of an associated ROI image matrix. In some embodiments, a sample image is obtained and processed to obtain gradient magnitudes. A set of initial candidate ROIs is selected based on gradient magnitudes greater than a threshold. In a further embodiment, the sample image is processed to generate images associated with at least three image resolutions corresponding to an image pyramid, and the candidate ROIs are ranked based on eigenvalues ​​associated with corresponding image blocks in at least one of the three images of the image pyramid. Alternatively, a histogram of gradients can be used to evaluate the candidate ROIs.

[0004] The foregoing and other features and advantages of the disclosed technology will become more apparent from the following detailed description taken in conjunction with the accompanying drawings. [Brief description of the drawings]

[0005] [Figure 1] 1 illustrates an exemplary method for identifying a suitable region of interest (ROI) for use in autofocus and / or astigmatism adjustment. [Figure 1A] A typical four-level Gaussian pyramid is shown. [Figure 1B] 1 shows an ROI in an image at any level of the Gaussian pyramid in FIG. 1A. [Diagram 2] 13 shows an exemplary method for ranking ROIs for use in autofocus and astigmatism adjustment. [Diagram 3] 1 shows an exemplary dual beam system including auto focus and astigmatism adjustment based on the selected ROI. [Figure 4] 1 illustrates a representative computing environment for implementing the disclosed methods and apparatus. [Diagram 5] 1 shows an exemplary autofocus method using ranked ROIs. [Figure 6A] 1 is an image of a sample area having features that are suitable for selection as an ROI for use in the autofocus procedure. [Figure 6B] 1 is an image of a sample area having features that are suitable for selection as an ROI for use in the autofocus procedure. [Figure 6C] 1 is an image of a sample area having features that are suitable for selection as an ROI for use in the autofocus procedure. [Figure 6D] 13 is an image of a sample area that is less suitable for use as an ROI in the autofocus procedure. [Figure 6E] 13 is an image of a sample area that is less suitable for use as an ROI in the autofocus procedure. [Figure 7] A typical method for defining an ROI within an image is shown. [Figure 7A] We indicate the assignment of gradient orientations for use in generating a histogram of gradient orientation (HGO). [Figure 7B] We indicate the assignment of gradient orientations for use in generating a histogram of gradient orientation (HGO). [Figure 7C] We indicate the assignment of gradient orientations for use in generating a histogram of gradient orientation (HGO). [Figure 7D] 7B shows the HGO associated with the ROI in FIG. 7A. [Figure 7E] 1 shows an HGO associated with an ROI suitable for use in autofocus. [Figure 7F] 13 shows the HGO associated with an ROI that is unsuitable for use in autofocus. [Figure 8] We present an exemplary method for selecting and ranking candidate ROIs using HGO. [Figure 9]An exemplary method for selecting and ranking candidate ROIs using favorable and / or unfavorable criteria HGO is presented. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0006] Regions of interest (ROIs) for use in focus adjustment can preferably be imaged to have strong edges and high contrast with their background. ROIs for use in astigmatism correction can be imaged to have lines, edges, or linear structures in at least some directions, such as orthogonal corners. Typically, the selection of ROIs that can be imaged to provide such high contrast features aligned in different directions avoids sample areas with grainy textures that may have intensity variations in multiple directions but lack clear edges. Representative sample portions that are preferred for use in defining ROIs for autofocus procedures are shown in Figures 6A-6B. These images show features with strong edges and high contrast in multiple directions. Figure 6C shows another satisfactory sample area, but with lower contrast and edges in fewer directions. Figures 6D-6E show typical unsatisfactory areas with low contrast and / or grainy structures. An approach that allows for the automatic selection of suitable ROIs and thus a fully automated focus procedure is disclosed below.

[0007] General Terms and Considerations As used herein, "autofocus", "autofocusing", "autofocus procedure", and the like, unless otherwise indicated, refer to processor-based adjustment of a charged particle beam (CPB) optical column to establish focus and / or reduce astigmatism. Optical column generally refers to CPB lenses, deflectors, astigmatism correctors, and other CPB optical elements used for imaging. Region of interest (ROI) refers to a selected portion of a sample or a corresponding portion of a sample image. The size of the ROI is referred to as block size, and refers to the size or number of pixels of the sample area or associated image area. Images can be stored in a variety of formats, such as JPG, TIFF, BMP, or other formats, but in the following examples, a representation as a two-dimensional array is used. An image can also refer to a displayed view of a specimen, as presented on a display device or other visual display. In the examples, an image generally corresponds to a two-dimensional array of intensity values ​​I arranged in rows and columns, i.e., an array of values ​​I(n,m), where n, m are integers referring to row and column locations. For such an array, the image gradient values ​​are proportional to I(n+1,m)-I(n,m) and I(n,m+1)-I(n,m) in the row and column directions, respectively.

[0008] Thresholding an image or image portion refers to comparing image intensities within the image or image portion to a reference level, and typically ignoring intensities below the reference level.

[0009] In an embodiment, an image of the specimen or part thereof is processed to define a series of lower resolution images, typically each image in the series having half the resolution of the previous image in the series. This reduction in resolution can be obtained, for example, by discarding pixels in every other row and column. The series of images can be pictorially represented as levels of a pyramid, with the base level corresponding to the highest resolution image in the series and the apex level corresponding to the lowest resolution image in the series. A resolution difference by a factor of 2 is convenient, since it is easy to discard alternating rows and columns of pixels, but other factors such as 3, 4, or any other integer value can be used. Instead of simply discarding pixels, adjacent pixels in an image can be averaged and used to define the next image in the series. The series of images does not need to use a common factor between levels, but it is convenient to use a common reduction by a factor of 2. The images in the Gaussian pyramid are formed by applying a Gaussian blur defined by a number of pixels corresponding to the standard deviation, and then discarding pixels as discussed above. A region of interest (ROI) image is referred to as having a block size based on the number of rows and columns that is equivalent to the ROI area on the sample.

[0010] As used in this application and the claims, the singular forms "a," "an," and "the" include the plural forms unless the content clearly dictates otherwise. Furthermore, the term "includes" means "comprises." Furthermore, the term "coupled" does not exclude the presence of intermediate elements between the coupled items.

[0011] The systems, devices, and methods described herein should not be construed as limiting in any way. Instead, the present disclosure is directed to all novel and non-obvious features and aspects of the various disclosed embodiments, alone and in various combinations and subcombinations with each other. The disclosed systems, methods, and devices are not limited to any particular aspect or feature or combination thereof, and the disclosed systems, methods, and devices do not require that any one or more particular advantages exist or problems be solved. Any theory of operation is provided for ease of explanation, but the disclosed systems, methods, and devices are not limited to such theory of operation.

[0012] Although some operations of the disclosed methods are described in a particular sequential order for convenient presentation, it should be understood that this style of description encompasses reordering unless a particular ordering is required by specific language set forth below. For example, operations described in sequence may in some cases be reordered or performed simultaneously. Furthermore, for the sake of simplicity, the accompanying drawings may not show the various ways in which the disclosed systems, methods, and apparatuses may be used in conjunction with other systems, methods, and apparatuses. Furthermore, the description may use terms such as "produce" and "provide" to describe the disclosed methods. These terms are high-level abstractions of actual operations that are performed. The actual operations that correspond to these terms will vary depending on the particular implementation and are readily discernible by those of ordinary skill in the art.

[0013] In some examples, values, procedures, or devices are referred to as "lowest," "best," "minimum," etc. Such descriptions are intended to indicate that a selection may be made from among many functional alternatives used, but it will be understood that such a selection is not necessarily better than, less than, or otherwise preferred to other selections.

[0014] Examples are described with reference to orientations indicated as "above," "below," "upper," "lower," etc. These terms are used for convenient description only and do not imply a particular spatial orientation.

[0015] Example 1. ROI using Gaussian pyramid Referring to FIG. 1, an exemplary method 100 for identifying suitable ROIs for use in autofocus of a CPB instrument includes acquiring an image of a sample with the instrument at 102 and processing the acquired image at 104 to generate a corresponding image based on the gradient of the acquired image. Typically, the gradient image is based on the gradient magnitude (|∇I(x,y)|, where I(x,y) is the image intensity as a function of position (x,y) and is generally represented as I(m,n), where m,n are non-negative integers indicating row and column locations. The gradient magnitude image may be thresholded at 106 to generate a thresholded gradient magnitude image. Thresholding may be performed, for example, by normalizing the gradient magnitudes in the gradient magnitude image using the maximum gradient magnitude and then thresholding the gradient magnitudes by at least 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.8, 10, 12, 14, 16, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, This can be done by selecting a suitable threshold level, such as a normalized gradient image magnitude greater than 7, 0.8, or 0.9. At 108, candidate ROIs are identified based on the thresholded gradient image and may include some or all ROIs in the thresholded gradient image. The candidate ROI block size can be selected accordingly. In some cases, a particular image may lack sufficient gradient magnitude and no candidate ROIs are identified at 109. In other embodiments, gradient components in two directions are evaluated.

[0016] The candidate ROI may or may not be suitable for the autofocus procedure and may be further evaluated as follows: At 110, a pyramid, such as a Gaussian pyramid, is generated based on the acquired image having N stacked images, which may be labeled 0, 1, ..., N-1, where N is a positive integer. The image associated with level 0 may be the original acquired image, and each higher level in the pyramid is smaller by a factor of 2, i.e., for an acquired image of 4096 x 4096 pixels, the stacked images in the Kth level of the pyramid may be 4096 / 2 for K=0, ..., N-1. K ×4096 / 2 K This results in a pixel image.

[0017] A representative pyramid 149 for N=3, i.e., a four-level pyramid with images 150-153, is shown in FIG. 1A. Image 150 is the acquired image, and images 151, 152, 153 are images corresponding to resizing by factors of 2, 4, and 8, respectively. FIG. 1B shows a representative image 160 associated with one of the pyramid levels showing ROIs 161, 162. This image 160 is shown as a block size of 8 pixels by 8 pixels for ease of illustration, although larger block sizes are typically used. Images at each level of the pyramid can be obtained by applying a Gaussian blur and then discarding pixels to produce the appropriate resolution. The Gaussian blur is typically based on a standard deviation of 2 to 10 pixels.

[0018] At 112, starting from the top image, a pyramid level (i.e., the Kth image) is selected. A portion of the Kth image corresponding to some or all of the candidate ROIs is selected. The selected portion may be represented by an M×M matrix, where M is an integer associated with the block size of the ROI. For each selected candidate ROI, eigenvalues ​​of the associated M×M matrix from the Kth level image are determined. At 116, the eigenvalues ​​are used to select a Kth set of candidate ROIs that may include some or all of the ROIs identified at 108 or selected from a previous evaluation of an image from a higher level in the Gaussian pyramid. The above procedure is repeated with images from other levels of the Gaussian pyramid determined at 118. If additional levels are to be processed, the level counter K is decremented at 120 and the process returns to 114. If images from all levels have been processed, the final set of ROIs as a generally ranked set is output at 122 for delivery to the autofocus procedure. The final set of ROIs may include some, none, or all of the candidate ROIs identified at 108. As mentioned above, if a suitable set of ROIs cannot be found based on the identified candidate images, it may be necessary to manually select ROIs for use in autofocus.

[0019] Example 2. ROI evaluation using eigenvalues 2 shows a method 200 for processing an image at a selected level of a Gaussian pyramid to select and rank ROIs at the selected level. At 202, a candidate ROI associated with the selected level is input. The candidate ROI can be represented as a square matrix of intensity values. At 204, eigenvalues ​​of the candidate ROI are determined, and at 206, the eigenvalues ​​are rank-ranked to a size L KIt is sorted from the maximum to the minimum based on this. In 208, the ratio R1 of the eigenvalue having the second largest magnitude L2 to the eigenvalue having the largest magnitude L1, that is, R1 = L2 / L1, is obtained. In 210, the ratio R2 of the sum of the magnitudes of the two largest eigenvalues to the sum of the magnitudes of all eigenvalues, that is,

Number

[0020] When the ratios R1 and R2 are large enough, such as greater than 0.2, 0.3, 0.4, 0.5, 0.6, or 0.7, the associated ROI is shown as having strong edges that are approximately orthogonal and may have high-contrast corners. If in 212 it is determined that the ratios R1 and R2 are not large enough, for example, if additional candidate ROIs are available based on the image gradient, different candidate regions are selected in 220. When the ratios R1 and R2 are large enough, in 214, the ratio R M is the ratio of the sum of the magnitudes of the M largest eigenvalues to the sum of the magnitudes of all eigenvalues, that is,

Number

[0021] In addition, in 215, the ratio R3 = L J / L1 is determined as a measure of the similarity of the N largest eigenvalues. In 216, it is determined whether the N largest eigenvalues are similar enough, for example, whether the ratio R3 = L J / L1 is greater than 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, or 0.9 for all 2 < J < 36 of the largest eigenvalues. Also, for example, the ratio R MFor 2 < M < 36, it is also determined whether the set of eigenvalues of the largest magnitude occupies a significant portion of the variability within the image, as indicated by being greater than, for example, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, or 0.9. If these conditions are met, the associated ROI is shown to have strong edges oriented in multiple directions having a corner or circular shape in the corresponding gradient magnitude image. At 216, candidate ROIs whose eigenvalues meet the above conditions are added, at 217, to the selected set of ROIs used in the autofocus procedure and, at 218, are stored. Candidate regions having one or more suitable values of ratio R1, R2, R3, or R M can also be selected even if other ratios are not within the preferred range. At 220, another candidate ROI can be selected and the process is repeated until all or sufficient ROIs at the selected level have been processed. Method 200 typically starts from the lowest resolution image (the highest level in the pyramid) and is executed for each level of the Gaussian pyramid. In some cases, the ROI is processed at only one or some levels and only some candidate ROIs are evaluated. This procedure is applied to each level of the image pyramid and the candidate ROIs output from the Kth level function as input to the (K - 1)th level. Some or all levels of the image can be used and not all, but only the selected candidate ROIs are evaluated.

[0022] As discussed above, various ranges can be used for ratios R1, R2, R3, and R M but typical values such as 0.5, 0.7, and 0.9 are used. In general, the selected ROIs can be ranked based on some or all of these ratios such that each ROI can be associated with values R1, R2, R3, and R M . The ranking of the ROIs can be stored along with pointers to the associated ROI image blocks such as the rows and columns associated with the pixels having the largest gradient magnitude.

[0023] Determining eigenvalues ​​corresponds to principal component analysis (PCA), which can model variation within an ROI and identify significant variation. In PCA, ROIs with sharply defined features in multiple directions can be found using their relatively large eigenvalues ​​as discussed above.

[0024] In this embodiment, a particular ratio is used to evaluate the ROI for suitability for autofocus. In other embodiments, an average of some or all of the eigenvalue magnitudes and other eigenvalues ​​may be used in evaluating the ROI's suitability for autofocus.

[0025] Example 3. Dual beam system 3, a dual beam system 300 includes a system controller 302 coupled to an ion beam source 304 and an electron beam source 306 that generate an ion beam 305 and an electron beam 307, respectively. Respective scanners 312, 314 are positioned to direct a scanned ion beam 313 and a scanned electron beam 315, respectively, toward a specimen 320. In some applications, images are obtained based on the scanned electron beam 315, and the scanned ion beam 313 is used for specimen modification. However, images can be obtained with either the scanned ion beam 313 or the scanned electron beam 315, or both. In some cases, the imaging system includes only one of an electron beam source and an ion beam source. For many biochemical specimens, only the electron beam is required and the ion beam component is not used.

[0026] The specimen 320 is fixed to a stage 322 that is coupled to a stage controller 324 that is coupled to the system controller 302. The stage 322 may typically provide one or more of translation, rotation, or tilt as directed by the system controller 302. A beam 326 responsive to the scanned ion beam 313 or scanned electron beam 315 is directed to an electron or ion detector 328 that is coupled to system electronics 330 that may include one or more analog-to-digital converters (ADCs), digital-to-analog converters (DACs), amplifiers, and buffers for control of the detector 328 and processing (amplification, digitization, buffering) signals associated with the detector 328. In other embodiments, photon detectors are used that generate electrical signals that are further processed by the system electronics. In most practical applications, at least one ADC is used to generate digitized detector signals that may be stored in one or more tangible computer-readable media, such as an image, shown as image storage 332. In other embodiments, image storage is remote, such as via a communications connection, such as a wired or wireless network connection. The beam 326 can be a scattered portion of the scanned ion beam 313, the scanned electron beam 315, secondary electrons, ions, or neutral atoms. An optical imaging device 351, such as a camera, is coupled to generate an image of the specimen 320, for example, to provide visual feedback to a technician.

[0027] The system controller 302 is coupled to a memory 335 that stores processor-executable instructions for image processing and acquisition, such as autofocus 336, ROI identification and ranking 340, including gradient and eigenvalue calculation and ranking, selection and usage updating of ROIs in response to an autofocus request 338, and provides a graphical user interface (GUI) 342 for various functions, including parameters associated with selection of ROIs for autofocus, and whether usage indicators should be provided to avoid possible beam damage and / or sample charging. Images (both CPB and optical) may be stored in the memory portion 332. Stage coordinates (including rotation) may be stored in the memory portion 332 as well. The system controller 302 sets image acquisition parameters and is in communication with the stage controller 324. Specimen images may be presented on a display 352, and system control and imaging parameters may be specified using internally stored values ​​from the memory 335 or provided by a user using one or more user input devices 350.

[0028] It will be appreciated that the layout of Figure 3 is for convenience of illustration and does not show the actual alignment of the various beam sources, optical imaging device 351, and CPB detector. Although a dual beam (ion / electron) system is shown, either or both can be used, and in many applications such as electron microscopes, only the electron beam is used for imaging.

[0029] Example 4. Representative Computing Environment FIG. 4 and the following discussion are intended to provide a brief, general description of an exemplary computing environment in which the disclosed technology may be implemented. In particular, some or all portions of this computing environment may be used in conjunction with the methods and apparatus described above, for example, to identify and rank ROIs for use in autofocus operations. Although not required, the disclosed technology will be described in the general context of computer-executable instructions, such as program modules, executed by a personal computer (PC). Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. Furthermore, the disclosed technology may be implemented with other computer system configurations, including handheld devices, tablets, multiprocessor systems, microprocessor-based or programmable consumer electronics, network PCs, minicomputers, mainframe computers, etc. The disclosed technology may also be practiced in distributed computing environments in which tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices. As used herein, a processor refers to a CPU as well as other logical processing devices such as FPGAs, ASICs, and CPLDs.

[0030] With reference to FIG. 4, an exemplary system for implementing the disclosed techniques with a CPB appliance 499 includes a general-purpose computing device in the form of an exemplary conventional PC 400 including one or more processing units 402, a system memory 404, and a system bus 406 that couples various system components including the system memory 404 to the one or more processing units 402. The system bus 406 may be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. The exemplary system memory 404 includes a read only memory (ROM) 408 and a random-access memory (RAM) 409. The PC 400 includes a basic input / output system (BIOS) 412, which contains the basic routines that help transfer information between elements within the PC 400.

[0031] The exemplary PC 400 further includes one or more storage devices 430, such as a hard disk drive for reading from and writing to a hard disk, a magnetic disk drive for reading from and writing to a removable magnetic disk, and an optical disk drive for reading from and writing to a removable optical disk (such as a CD-ROM or other optical media). Such storage devices may be connected to the system bus 406 by a hard disk drive interface, a magnetic disk drive interface, and an optical drive interface, respectively. The drives and associated computer-readable media provide non-volatile storage of computer-readable instructions, data structures, program modules, and other data for the PC 400. Other types of computer-readable media can store data accessible by a PC and can be used in the exemplary operating environment, such as magnetic cassettes, flash memory cards, digital video disks, CDs, DVDs, RAM, ROM, and the like.

[0032] Several program modules may be stored in the storage device 430, including an operating system, one or more application programs, other program modules, and program data. A user may enter commands and information into the PC 400 through one or more input devices 440, such as a keyboard and a pointing device such as a mouse. For example, a user may enter a command to initiate ROI selection or to initiate autofocus or to select automatic ROI selection and autofocus. Other input devices may include digital cameras, microphones, joysticks, game pads, satellite receivers, scanners, and the like. These and other input devices are often connected to the one or more processing units 402 through a serial port interface coupled to the system bus 406, but may be connected by other interfaces such as a parallel port, game port, universal serial bus (USB), or a wired or wireless network connection. A monitor 446 or other type of display device may also be connected to the system bus 406 through an interface such as a video adapter and may display, for example, one or more ROI images, gradient images, or other processed images. A monitor 446 can also be used to assist with CPB equipment control, to confirm successful focus and astigmatism control, and other functions. Other peripheral output devices, such as speakers and a printer 450, may be included.

[0033] The PC 400 may operate in a networked environment using logical connections to one or more remote computers, such as a remote computer 460. In some embodiments, one or more network or communication connections 450 are included. The remote computer 460 may be another PC, a server, a router, a network PC, or a peer device or other common network node, and typically includes many or all of the elements described above in relation to the PC 400, although only a memory storage device 462 is illustrated in FIG. 4. The personal computer 400 and / or the remote computer 460 may be part of a logical local area network (LAN) and a wide area network (WLAN). A WAN (Network Area Network) can be connected to a network.

[0034] 4, memory 490 (or a portion of this or other memory) stores processor-executable instructions for autofocus, ROI identification, selection, and ranking for use in autofocus, including instructions for computing gradients and eigenvalues, determining ratios as discussed above, generating Gaussian pyramids, and computing and evaluating histograms of gradient orientations (HGOs). In addition, memory 490 can store ROIs and / or locations and block sizes along with rankings so that the autofocus procedure can select ROIs and, if necessary, indicate how many times a selected ROI has been used.

[0035] Example 5. Autofocus with Selected ROI 5, an exemplary method 500 includes receiving an autofocus request in a CPB imaging system at 502. At 504, the ratios R1, R2, R3, and R4 discussed above are calculated. MAn ROI is selected for use in autofocus from a set of ROIs stored in memory 509 based on one or more ROI ranks, such as one or more or a combination of, and / or the number of previous uses (Use#) of a particular ROI. The ROIs may be stored as a location and block size of the ROI from a common image, rather than as image data. In addition, a usage indicator may be provided for each ROI so that the number of uses of each ROI may be limited to avoid damaging or charging the ROI. At 506, autofocus is performed using the selected ROI, and at 508, the usage indicator associated with the selected ROI is updated. At 510, a subsequent request for autofocus is awaited in response to instrument drift, elapsed time, a change in specimen requiring a set of ROIs based on a new specimen, or a user request. Upon receiving an autofocus request, an ROI is again selected at 504, and method 500 continues as discussed above. If no additional autofocus requests are received, autofocus ends at 512. In method 500, autofocus does not require operator intervention, and even if autofocus is initiated by an operator, autofocus using an ROI does not require additional attention after the request. As a result, autofocus can be initiated at any time that does not interfere with use.

[0036] Example 6. ROI using Histogram of Gradient Orientation (HGO) HGO can also be used to evaluate candidate ROIs for use in focus and astigmatism correction. Generally, a preferred ROI has gradients of suitable magnitude in multiple directions, such as a uniform distribution of gradients. As discussed above, in a first step, the ROI can be evaluated to determine gradient magnitude, such as by comparison with a threshold, and ROIs that do not have sufficient gradient magnitude generally are less suitable for focus and astigmatism adjustment and need not be processed further. As shown in FIG. 7, candidate ROIs are obtained by selecting an image 700 and scanning a selected image area template 702 across the image 700 to generate ROIs 704, 706, 708. In this example, the selected image area template is scanned rightward from ROI 704 in the direction indicated by 703 to define ROIs 706, 708. The scan can be vertical or horizontal, and the scan step size is generally 1 / 32 to 1 / 4 of the number of pixels in a row or column of the ROI. For example, for an ROI that is 256x256 pixels, step sizes of 2, 4, 8, 16, 32, or 64 pixels can be used. In Figure 7, ROIs 704, 706, 708 are shown slightly vertically offset so that the overlap in Figure 7 does not obscure their depiction. In practice, such offsets are not used. However, the image area template can be scanned in any direction using any standard, non-standard, periodic, random, or other step size.

[0037] FIG. 7A shows an ROI 712 that includes four rows and four columns of pixels, such as representative pixel 714. Typically, more pixels are used in an ROI, but the example of FIG. 7A allows for convenient illustration. Each pixel of the ROI 712 is assigned a gradient direction from a set of gradient directions 715, indicated by arrows. In this example, the gradient is based on nearest neighbor gradients, and there are eight possible gradient directions. The number of pixels associated with each of the directions (indicated as arrows) is provided above the ROI 712. The associated HGOs are shown in FIG. 7D.

[0038] FIG. 7B shows the intensity I 22and the eight nearest neighboring pixels (with intensity I 11 ~I 13 , I 21 , I 23 , and I 31 ~I 33 ) and the center pixel 720. Eight gradient directions based on the difference associated with each of these nearest neighbor pixels are shown as arrows. These assignments are used to form the HGO, however, in an exemplary embodiment, pixels with gradients having small magnitudes are not used in forming the histogram. Additional or alternative directions can be used, as shown in FIG. 7C, where the 12 directions associated with the next nearest neighbor pixels of the center pixel 720 are shown as corresponding arrows. The HGO can be based on one of both of these sets of gradient directions, or the gradient directions can be based on additional pixels.

[0039] Figure 7E shows a target HGO 750 for an ROI where the number of pixels associated with each of the gradient directions is substantially the same and indicates a suitable candidate ROI. Figure 7F shows an HGO 752 where the distribution of gradients is concentrated along a single gradient direction and is associated with a candidate ROI that is less suitable for autofocus.

[0040] Example 7. Representative HGO Method Referring to FIG. 8, an exemplary method 800 includes selecting an ROI size at 802 and selecting an initial ROI location at 804. An ROI image is obtained at 806 and gradient components for each pixel within the ROI are calculated at 808. The ROIs are evaluated at 810 and ROIs associated with gradient magnitudes below a threshold are rejected for use in autofocus and astigmatism control and a different ROI is selected at 806 for evaluation. Otherwise, an associated HGO is calculated at 811 and compared to a uniform or other preferred reference HGO at 812. If the distance of the HGO for the selected ROI from the reference HGO is determined to be sufficient at 814, the ROI is added to a set of ROIs for focus and astigmatism control at 816 and then the ROI is selected again at 806. If the HGO distance is not sufficient at 814, the ROI is selected again at 806. If the reference HGO is amorphous, i.e., all gradient directions appear with similar frequency, then the ROI associated with a smaller distance is preferred, whereas for reference HGOs associated with one or several gradient directions, a larger distance is preferred. Although in FIG. 8 (and other embodiments), multiple candidate ROIs are evaluated and multiple candidate ROIs are selected, in some cases, only a single preferred ROI is used and multiple candidate ROIs need not be evaluated.

[0041] Example 8. HGO Comparison The HGO can be evaluated using various approaches. In one embodiment, the HGO can be displayed for evaluation by a technician. In this case, it may be useful to display the HGO associated with the suitable candidate ROI and / or the unsuitable candidate ROI as a reference HGO to aid in visual comparison. For example, the HGO for the candidate ROI can be displayed with the acceptable HGO and / or the unacceptable HGO as shown in Figures 7D-7E. The technician can then determine whether the candidate ROI is acceptable based on, for example, one or more of the shape, distribution, and peak difference of the HGO associated with the candidate ROI. The visual comparison can also be based on a display of the candidate HGO and the reference HGO superimposed on each other.

[0042] Although visual methods of comparing HGOs can be used, approaches that do not require the input of a technician are generally preferred. In one embodiment, statistical values ​​of the candidate HGOs are calculated, such as standard deviation and / or skewness. Relatively larger values ​​of standard deviation and higher values ​​of skewness indicate favorable candidate ROIs. Candidate ROIs can also be evaluated based on the Kolmogorov-Smirnov test, which is used to determine the difference between the HGO associated with the candidate ROI and the reference HGO. A chi-square test can also be used to compare the frequencies in different histogram bins to compare HGOs. The so-called Earth Mover's Distance (EMD), also referred to as the Wasserstein metric, can be used. In another embodiment, the Kullback-Leibler Divergence (KLD) can be used to evaluate the difference between the HGOs. Depending on whether favorable (acceptable) HGO or unfavorable (unacceptable) HGO is used as the reference in the comparison, higher or lower values ​​of the test value can be used to select the candidate ROI, and the associated test value can be used to rank the selected ROIs.

[0043] Example 8. Representative HGO Comparison Method Referring to FIG. 9, a representative method 900 includes selecting a candidate ROI for evaluation at 902. At 904, an HGO associated with the selected candidate ROI is obtained, and at 906, a comparison method is selected. At 908, a comparison with a non-preferred reference HGO is performed, and at 910, based on the comparison, the selected ROI is identified as suitable and added to the set of final ROIs with or without ranking, or is identified as unsuitable and not added to the set of final ROIs. Alternatively, at 906, a comparison with a preferred reference HGO is selected, and at 912, a comparison is performed. At 910, based on the comparison, the selected ROI is identified as suitable and added to the set of final ROIs with or without ranking, or is identified as unsuitable and not added to the set of final ROIs. As shown, a comparison of the HGO to two or more reference HGOs, one or more preferred reference HGOs and / or one or more unpreferred reference HGOs, can be used to determine ROI candidate suitability. Another candidate ROI can be selected and evaluated as described above at 914. If no further ROIs are to be evaluated, then at 916 the method ends.

[0044] Disclosure Provisions Clause 1 is a method that includes obtaining a sample image, processing the sample image to obtain gradient magnitudes, and selecting a set of initial candidate regions of interest (ROIs) in the sample image associated with gradient magnitudes greater than a threshold.

[0045] Clause 2 includes the subject matter of clause 1, and further includes ranking each of the initial candidate ROIs based on an associated gradient magnitude.

[0046] Clause 3 includes the subject matter of any of clauses 1-2, and further provides for evaluating the initial candidate ROIs based on a histogram of gradient orientations (HGO) associated with each of the initial candidate ROIs.

[0047] Clause 4 includes the subject matter of any of clauses 1-3, and further specifies that the gradient orientation in the HGO is based on a gradient orientation determined from a nearest neighbor pixel associated with each pixel in the candidate ROI.

[0048] Clause 5 includes the subject matter of any of clauses 1-4, and further specifies that evaluating the initial candidate ROIs based on a histogram of gradient orientations (HGO) is performed by comparing the HGO of each ROI to a uniform HGO.

[0049] Clause 6 includes the subject matter of any of clauses 1-5, and further includes processing the sample image to generate images associated with at least three image resolutions corresponding to the image pyramid, and ranking the initial candidate ROIs based on eigenvalues ​​associated with corresponding image blocks in at least one of the three images of the image pyramid.

[0050] Clause 7 includes the subject matter of any of clauses 1 to 6, and further includes selecting a first set of candidate ROIs from the initial set of candidate ROIs based on ranking the initial set of candidate ROIs obtained using the lowest resolution image of the image pyramid.

[0051] Clause 8 includes the subject matter of any of clauses 1-7, and further includes selecting a second set of candidate ROIs based on ranking a first set of candidate ROI sets obtained using an image of the image pyramid having a resolution other than the lowest resolution.

[0052] Clause 9 includes the subject matter of any of clauses 1-8, further comprising: processing the sample image to generate a Gaussian image pyramid including images associated with at least three image resolutions; and for each image in the Gaussian image pyramid, starting with the lowest resolution image and ending with the highest resolution image, ranking candidate ROIs from the initial set of candidate ROIs based on eigenvalues ​​associated with corresponding image blocks from each image in the Gaussian image pyramid; The ranking based on the lower resolution image of the Gaussian image pyramid is used to select candidate ROIs for processing with the next higher resolution image of the Gaussian image pyramid.

[0053] Clause 10 includes the subject matter of any of clauses 1-9, and further provides that the ranking is based on eigenvalue magnitude.

[0054] Clause 11 includes the subject matter of any of clauses 1-10, and further provides that the ranking is based on magnitudes of the N eigenvalues ​​having the largest magnitudes, where N is an integer less than 36 and greater than 2.

[0055] Clause 12 includes the subject matter of any of clauses 1-11 and further provides that the ranking is based on at least one of: a ratio of the magnitudes of the N eigenvalues ​​with the largest magnitudes to the sum of the magnitudes of all eigenvalues, a ratio of the sum of the magnitudes of the two eigenvalues ​​with the largest magnitudes to the sum of the magnitudes of all eigenvalues, a ratio of the magnitudes of the eigenvalues ​​with the largest magnitudes, and a ratio of each of the N eigenvalues ​​with the largest eigenvalues ​​to the average of the N eigenvalues ​​with the largest eigenvalues.

[0056] Clause 13 includes the subject matter of any of clauses 1-12, and further provides that the ranking is based on a ratio of the sum of the magnitudes of the N eigenvalues ​​having the largest magnitudes to the sum of the magnitudes of all of the eigenvalues.

[0057] Clause 14 includes the subject matter of any of clauses 1 to 13, and provides that at least three associated image resolutions are greater than or equal to M / 2 K , M is the size of the sample image, and K is an integer greater than 0 and less than 10.

[0058] Clause 15 includes the subject matter of any of clauses 1-14, and further includes selecting a final set of candidate ROIs based on eigenvalues ​​associated with image blocks in the highest resolution image, each candidate ROI in the final set of candidate ROIs being selected such that a ratio of the sum of the magnitudes of the N eigenvalues ​​with the largest magnitudes to the sum of the magnitudes of all eigenvalues ​​or the sum of the magnitudes of all eigenvalues ​​except the N eigenvalues ​​with the largest magnitudes is greater than 0.7, and a ratio of the magnitudes of the two eigenvalues ​​with the largest magnitudes is between 0.5 and 2.

[0059] Clause 16 includes the subject matter of any of clauses 1 to 15, and further provides that the block sizes of the image blocks selected for each image of the Gaussian image pyramid are different.

[0060] Clause 17 includes the subject matter of any of clauses 1-16, further specifying that the Gaussian image pyramid is obtained using a Gaussian kernel having a standard deviation corresponding to between 3 and 10 image pixels.

[0061] Clause 18 includes the subject matter of any of clauses 1-17, and further includes adjusting a charged particle beam (CPB) optical column based on at least one image associated with a ROI in the final set of candidate ROIs.

[0062] Clause 19 includes the subject matter of any of clauses 1-18, further including performing a series of adjustments of the CPB optical column based on a plurality of different ROIs from the final set of candidate ROIs.

[0063] Clause 20 includes the subject matter of any of clauses 1-19 and further provides that the plurality of different ROIs are selected in sequence of descending ranking, starting with the highest ranking and continuing in descending ranking order.

[0064] Clause 21 is a charged particle beam (CPB) imaging system including a CPB optical column operable to direct a CPB toward a sample and generate an image of at least a portion of the sample; and a system controller coupled to the CPB optical column and operable to shape and direct the CPB toward the sample, the system controller being configured to process the sample image to obtain a gradient magnitude image; to select a set of initial candidate regions of interest (ROIs) in the sample image associated with portions of the gradient magnitude image having gradient magnitudes greater than a threshold; to rank each of the initial candidate ROIs and based on the ranking, select an ROI from the initial candidate ROIs; and to adjust focus and astigmatism based on the selected ROI.

[0065] Clause 22 includes the subject matter of any of clauses 21-22 and further specifies that the system controller is configured to evaluate the candidate ROI based on images from a set of images defined as an image pyramid.

[0066] Clause 23 includes the subject matter of any of clauses 21-22 and further specifies that the system controller is configured to evaluate candidate ROIs based on eigenvalues ​​associated with the ROIs within images of the image pyramid.

[0067] Clause 24 includes the subject matter of any of clauses 21-23, and further provides that the ranking is determined based on a ratio of magnitudes of eigenvalues ​​associated with the ROIs within an image of the image pyramid.

[0068] Clause 25 includes the subject matter of any of clauses 21-24, wherein the system controller is configured to evaluate each candidate ROI based on an associated histogram of gradient orientations (HGO).

[0069] Clause 26 includes the subject matter of any of clauses 1-25, further providing that the gradient orientation in the HGO is based on a gradient orientation determined from a nearest neighbor pixel associated with each pixel in the candidate ROI.

[0070] Clause 27 includes the subject matter of any of clauses 1-26, and further specifies that evaluating the initial candidate ROIs based on a histogram of gradient orientations (HGO) is performed by comparing the HGO of each ROI to a uniform HGO.

[0071] Considering the many possible embodiments to which the principles of the disclosed technology may be applied, it should be understood that the illustrated embodiments are merely preferred examples. The scope of the invention is defined by the following claims. We therefore claim as our invention all that comes within the scope and spirit of these claims.

Claims

1. 1. A method comprising: Obtaining a sample image; processing the sample images to obtain gradient magnitudes; selecting a set of initial candidate regions of interest (ROIs) within the sample image that are associated with gradient magnitudes greater than a threshold.

2. The method of claim 1 , further comprising: ranking each of the initial candidate ROIs based on the associated gradient magnitude.

3. The method of claim 1 , further comprising: evaluating the initial candidate ROIs based on a histogram of gradient orientations (HGO) associated with each of the initial candidate ROIs.

4. The method of claim 3 , wherein the gradient orientation in the HGO is based on a gradient orientation determined from a nearest neighbor pixel associated with each pixel in a candidate ROI.

5. The method of claim 3 , wherein evaluating the initial candidate ROIs based on the histogram of gradient orientations (HGO) is performed by comparing the HGO of each ROI to a uniform HGO.

6. processing the sample image to generate images associated with at least three image resolutions corresponding to an image pyramid; ranking the initial candidate ROIs based on eigenvalues ​​associated with corresponding image blocks in at least one of the three images of the image pyramid; The method of claim 1 further comprising:

7. 7. The method of claim 6, further comprising: selecting a first set of candidate ROIs from the initial set of candidate ROIs based on the ranking of the candidate ROIs among the initial set of candidate ROIs obtained using the lowest resolution image of the image pyramid.

8. The method of claim 7, further comprising selecting a second set of candidate ROIs based on a ranking of the first set of candidate ROIs obtained using an image of the image pyramid having a resolution other than the lowest resolution.

9. processing the sample image to generate a Gaussian image pyramid including images associated with at least three image resolutions; and for each image of the Gaussian image pyramid, starting with a lowest resolution image and ending with a highest resolution image, ranking candidate ROIs from the set of initial candidate ROIs based on eigenvalues ​​associated with corresponding image blocks from each image of the Gaussian image pyramid, wherein the ranking based on the lower resolution images of the Gaussian image pyramid is used to select candidate ROIs for processing with the next higher resolution image of the Gaussian image pyramid. The method of claim 1.

10. The method of claim 9 , wherein the ranking is based on eigenvalue magnitude.

11. 10. The method of claim 9, wherein the ranking is based on the magnitudes of the N eigenvalues ​​having the largest magnitudes, where N is an integer less than 36 and greater than 2.

12. The ranking is the ratio of the sum of the magnitudes of the N eigenvalues ​​with the largest magnitudes to the sum of the magnitudes of all the eigenvalues, the ratio of the sum of the magnitudes of the two eigenvalues ​​having the largest magnitudes to the sum of the magnitudes of all eigenvalues; the ratio of the magnitudes of the eigenvalues ​​having the largest magnitudes; and 10. The method of claim 9, wherein the ratio of each of the N eigenvalues ​​having the largest eigenvalues ​​to an average of the N eigenvalues ​​having the largest eigenvalues ​​is based on at least one of:

13. The method of claim 12 , wherein the ranking is based on a ratio of a sum of magnitudes of the N eigenvalues ​​with the largest magnitudes to the sum of magnitudes of all eigenvalues.

14. The images associated with at least three image resolutions are Ｋ 12. The method of claim 11, wherein M is a size of the sample image and K is an integer greater than 0 and less than 10.

15. selecting a final set of candidate ROIs based on eigenvalues ​​associated with image blocks in the full resolution image, wherein each candidate ROI in the final set of candidate ROIs comprises: a ratio of the sum of the magnitudes of the N eigenvalues ​​having the largest magnitudes to the sum of the magnitudes of all eigenvalues ​​or to the sum of the magnitudes of all eigenvalues ​​except the N eigenvalues ​​having the largest magnitudes is greater than 0.7; and The method of claim 11, wherein the ratio of the magnitudes of the two largest magnitude eigenvalues ​​is selected to be between 0.5 and 2.

16. The method of claim 15 , wherein the block sizes of the image blocks selected for each image of the Gaussian image pyramid are different.

17. The method of claim 15, wherein the Gaussian image pyramid is obtained using a Gaussian kernel with a standard deviation corresponding to between 3 and 10 image pixels.

18. The method of claim 15 , further comprising adjusting a charged particle beam (CPB) optical column based on at least one image associated with an ROI in the final set of candidate ROIs.

19. The method of claim 18 , further comprising: making a series of adjustments to the CPB optical column based on a plurality of different ROIs from the final set of candidate ROIs.

20. The method of claim 19 , wherein the plurality of different ROIs are selected starting with the highest ranking and continuing in a sequence of descending ranking.

21. 1. A charged particle beam (CPB) imaging system, comprising: a CPB optical column operable to direct the CPB towards the sample and generate an image of at least a portion of the sample; a system controller coupled to the CPB optical column and operable to shape and direct the CPB toward the sample, To process the sample image to obtain the gradient magnitude image, selecting a set of initial candidate regions of interest (ROIs) within the sample image associated with portions of the gradient magnitude image having gradient magnitudes greater than a threshold; ranking each of the initial candidate ROIs; and a system controller configured to select an ROI from the initial candidate ROIs based on the ranking, and adjust focus and astigmatism based on the selected ROI.

22. 22. The CPB imaging system of claim 21, wherein the system controller is configured to evaluate candidate ROIs based on images from a set of images defined as an image pyramid.

23. 23. The CPB imaging system of claim 22, wherein the system controller is configured to evaluate candidate ROIs based on eigenvalues ​​associated with the candidate ROIs in the images of the image pyramid.

24. 23. The CPB imaging system of claim 22, wherein the ranking is determined based on a ratio of magnitudes of eigenvalues ​​associated with the candidate ROI in the images of the image pyramid.

25. 22. The CPB imaging system of claim 21, wherein the system controller is configured to evaluate each candidate ROI based on an associated histogram of gradient orientations (HGO).

26. 26. The CPB imaging system of claim 25, wherein the gradient orientation in the HGO is based on a gradient orientation determined from a nearest neighbor pixel associated with each pixel in the candidate ROI.

27. 26. The CPB imaging system of claim 25, wherein evaluating the initial candidate ROIs based on the histogram of gradient orientations (HGO) is performed by comparing the HGO of each ROI to a uniform HGO.

Images