Free run operation of fabry perot variable etalon hyperspectral imaging

Abstract

A system includes a processor configured to initiate etalon free run motion; initiate free run image frame capture; determine whether a full set of band passes has been captured; and in response to the full set of band passes having been captured, construct a data cube. The system further includes a memory coupled to the processor and configured to provide the processor with instructions.

Description

FREE RUN OPERATION OF FABRY PEROT VARIABLE ETALON HYPERSPECTRAL IMAGINGCROSS REFERENCE TO OTHER APPLICATIONS

[0001] This application claims priority to U.S. Provisional Patent Application No. 63 / 670,629 entitled FREE RUN OPERATION OF FABRY PEROT VARIABLE ETALON HYPERSPECTRAL IMAGING filed July 1 , 2024 which is incorporated herein by reference for all purposes.BACKGROUND OF THE INVENTION

[0002] The development of multi and hyperspectral imaging technologies have introduced new possibilities in optical based detection and identification. However, technologies such as pushbroom grating, and acousto-optic tunable filters (AOTFs) have been limited in their ability to capture the data in a timely manner with the required spectral resolution and spatial uniformity and resolution. For example, push-broom gratings systems have been limited to applications where either the subject is either stationary or moving relatively slowly in a uniform vector. The protracted image acquisition times on the time scale of several seconds to minutes have made it impractical for use in high-speed production processes or for monitoring stochastic and dynamic processes. Multi-spectral systems based on mosaiced band filters offer the acquisition rates for large fields of view, but the number of bands is limited and cannot be changed dynamically and are often lower spectral resolution. In addition, for either technology, one must tradeoff between spatial in spectral resolution for many of these technologies.

[0003] For example, push-broom gratings systems have been limited to applications where either the subject is either stationary or moving relatively slowly in a uniform vector. The protracted image acquisition times on the time scale of several seconds to minutes have made it impractical for use in high-speed production processes or for monitoring stochastic and dynamic processes common in semiconductor fabrication.BRIEF DESCRIPTION OF THE DRAWINGS

[0004] Various embodiments of the invention are disclosed in the following detailed description and the accompanying drawings.

[0005] Figure 1 is a diagram illustrating an embodiment of a system for hyperspectral imaging.

[0006] Figure 2 is a diagram illustrating an embodiment of a Fabry -Perot etalon of a system for hyperspectral imaging.

[0007] Figure 3 is a flow diagram illustrating an embodiment of a process for a system for hyperspectral imaging.

[0008] Figure 4 is a flow diagram illustrating an embodiment of a process for indicating to drive an etalon free running motion.

[0009] Figure 5 is a flow diagram illustrating an embodiment of a process for determining a look up table.

[0010] Figure 6A is diagram illustrating an embodiment of an object with an overlay.

[0011] Figure 6B is a diagram illustrating an embodiment of different spectra.

[0012] Figure 7 is a diagram illustrating an embodiment of a system for making a calibration measurement for a hyperspectral imaging system.

[0013] Figure 8 is a flow diagram illustrating a process for calibrating a hyperspectral imager for free-run operation.

[0014] Figure 9 is a flow diagram illustrating an embodiment of a process for calibrating an etalon physical spacing or gap positions.DETAILED DESCRIPTION

[0015] The invention can be implemented in numerous ways, including as a process; an apparatus; a system; a composition of matter; a computer program product embodied on a computer readable storage medium; and / or a processor, such as a processor configured to execute instructions stored on and / or provided by a memory coupled to the processor. In this specification, these implementations, or any other form that the invention may take, may be referred to as techniques. In general, the order of the steps of disclosed processes may be altered within the scope of the invention. Unless stated otherwise, a component such as a processor or a memory’ described as being configured to perform a task may be implemented as a general component that is temporarily- configured to perform the task at a given time or a specific component that is manufactured toperform the task. As used herein, the term 'processor’ refers to one or more devices, circuits, and / or processing cores configured to process data, such as computer program instructions.

[0016] A detailed description of one or more embodiments of the invention is provided below along with accompanying figures that illustrate the principles of the invention. The invention is described in connection with such embodiments, but the invention is not limited to any embodiment. The scope of the invention is limited only by the claims and the invention encompasses numerous alternatives, modifications and equivalents. Numerous specific details are set forth in the following description in order to provide a thorough understanding of the invention. These details are provided for the purpose of example and the invention may be practiced according to the claims without some or all of these specific details. For the purpose of clarity, technical material that is known in the technical fields related to the invention has not been described in detail so that the invention is not unnecessarily obscured.

[0017] A hyperspectral system is disclosed. The system comprises a processor and a memory. The processor is configured to initiate etalon free run motion, initiate free run image frame capture, determine whether a full set of band passes has been captured, and construct a data cube in response to the full set of band passes having been captured. The memory is coupled to the processor and configure to provide the processor with instructions.

[0018] In some embodiments, the disclosed Fabry-Perot variable etalon based hyperspectral imaging system with its multi-band pass modes of operation improves performance and fl exi bi 1 i t in terms of rapid full frame or area image acquisition, dynamically adjustable band selection, and high spatial and spectral resolution. The system addresses limitations of other hyperspectral systems in terms of spectral response range, speed of acquisition, while preserving spectral and spatial resolution. This is achieved using a free running frame image acquisition scheme that enables identification of features for rapidly changing scenes (e.g., stemming from fast-moving objects or dynamic phenomena). In this approach, the Fabry-Perot etalon is also varied in a free run fashion by the computer (e.g., a continuous sequence of positions, continuous motion, or resonant motion) while the free running frames are captured for multiple spectral bands. This approach enables a rapid spectral scanning system unconstrained by time delays associated with the stopping of the etalon mirror motion during frame acquisition.

[0019] Figure 1 is a diagram illustrating an embodiment of a system for hyperspectral imaging. In the example shown, the system utilizes front-staring imaging comprising a Fabry-Perot etalon, a broadband illumination system, an imaging area sensor, a set of focusing optics, a sample fixture, and a data acquisition and processing computer platform. The light from the illuminatedsample projects an image via the focusing optics through the Fabry-Perot etalon which modulates its mirror separation distance to enable the sensing of fdtered light from the sample on the imaging area sensor. For example, sample 102 is positioned in field of view 120 of hyperspectral imager 100. Field of view 120 comprises a desired sampling area which can include an entire sample or an area of interest of the sample. In some embodiments, the system utilizes translation stages (e.g.. a computer-controlled scanning motorized stage, a piezo-actuated stage, etc ). An operator can initiate the image / data capture via a user interface of computer 126. After the user initiates image / data capture, computer 126 indicates to illuminator 130 to illuminate sample 102 and to hyperspectral imager 100 to take data while mirror actuator driver 128 as instructed by computer 126 moves Fabry-Perot etalon 122 to different gaps 112 between mirror 108 and mirror 110. The visual, near infrared (VNIR) system depending on the Fabry-Perot interferometer, its coatings, and the application has about a few hundred bands (e g., 200, 300, 400, 500, 600 bands) or between N and M bands (e.g., between N=100 and M=400 bands) using about a few hundred gaps (e.g., 100, 150, 200, 250, 277, 300, 350, 400, 450,500, 550, 600 gaps) or between K and L gaps (e.g., K=100 and L=300). The gap spacings are in the tens to hundreds of nm. For the VNIR system, Fabry- Perot etalon 122 is coated for transmission in the UV, VNIR, SWIR or beyond bands of light. Wideband illumination from illuminator 130 is reflected off sample 102 and light reflected within field of view 120 travels on path 124 towards sensor array 104 (e.g.. a CMOS sensor, a CCD sensor, another solid-state imaging area sensor, or any other type of appropriate area sensor).

[0020] In some embodiments, an RGB or Bayer pattern filter color or multi-channel sensor is used to collect multiple band-passes with responsiveness in the visible (400 to 700 nm) through the NIR (700 to 1,000 nm), and in some cases extended to the SWIR (1,000 nm to 2,500 nm) or beyond to generate a hyperspectral data cube, whereby each exposed frame in the series represents the image at a single band-pass.

[0021] In some embodiments, a monochrome sensor is utilized.

[0022] In some embodiments, path 124 enters hyperspectral imager 100 via objective 116, passes through relay lens 114 is filtered by Fabry-Perot etalon 122 before passing through relaylens 106 to sensor array 104. In some embodiments, the system utilizes one or more optics that provide a different field of view and / or magnification (e.g., multiple objectives for objective 116). Computer 126 controls image data taking over a series of gap positions and stores sensor 104 data. The data from sensor 104 is analyzed by computer 126. Analysis of the data includes applying a calibration using a mathematical matrix transform to the data in the frames of images from sensor 104 to construct a data cube. Machine learning algorithms that have been trained on known samplesare then applied to the data cube to group or classify regions of the image of interest based on their respective spectra and in some cases morphology and / or geometry. Training samples for the machine learning algorithms are comprised of a sufficiently large population of specimens, materials or subj ects with and without an attribute, component or characteristic of interest. The performance of the model is statistically validated by testing against a smaller proportion of other samples to assess its ability to identify or measure that attribute, component or characteristic of interest. In practical use, this is often manifested as false color overlay or numerical output in the case of metrology applications. A number of algorithms or machine learning models can be utilized for this purpose. In some embodiments, there is a pre-processing of the data to remove noise, to segment or augment data, to mathematically transform, or to otherwise modify data prior to the application of such models. In some embodiments, a composite image is generated with markers or flags indicating their presence and displayed to a user via computer 126.

[0023] Figure 2 is a diagram illustrating an embodiment of a Fabry-Perot etalon of a system for hyperspectral imaging. In some embodiments, Fabry-Perot etalon shown in Figure 2 comprises mirror 108 and mirror 110 of Fabry-Perot 122 of Figure 1. In the example show n, mirror 200 and mirror 202 can be positioned relative to each other to create an adjustable gap between mirror 200 and mirror 202 using piezo actuator 204, piezo actuator 206. and piezo actuator 208. In some embodiments, piezo actuator 204, piezo actuator 206, and piezo actuator 208 have their length adjusted by applying a voltage across the piezo material. In order to keep mirror 200 and mirror 202 parallel to each other (e.g., the plane of mirror 200 is parallel to the plane of mirror 202), drive voltages to piezo actuator 204, piezo actuator 206, and piezo actuator 208 have to be calibrated for each desired gap distance. For example, mirror 200 and mirror 202 are illuminated using a uniform flat laser beam across the full mirror faces and the laser beam that is transmitted through the etalon of the two mirrors is imaged onto a sensor. The flatness can be measured using the uniformity of the transmitted light. Once the drive voltages (e.g., for the static situation) have been determined for each gap, the drive voltages for each gap are stored in a look up table (e.g., gap index number (e.g., associated with gap of x nm), voltage for piezo 1 (e.g., piezo 204), voltage for piezo 2 (e.g., piezo 206), and voltage for piezo 3 (e.g., piezo 208)). In some embodiments, in order to achieve the free running motion for the mirrors an offset drive voltage for each gap position for each piezo is needed for the moving piezo to be in the correct position at the next time interval, so that when setting a next target position using the voltage for each piezo an offset (e.g., an offset stored for each gap for each piezo) is included in the drive voltage (e.g., gap index voltage for piezo 1 plus offset voltage for piezo 1, gap index voltage for piezo 2 plus offset voltage for piezo 2, and gap index voltage for piezo 3 plus offset voltage for piezo 3). In some embodiments, these offset valuesfor each gap are stored in the look up table (e.g., gap index number (e.g., associated with gap of x nm), voltage for piezo 1, offset voltage for piezo 1, voltage for piezo 2, offset voltage for piezo2, and voltage for piezo 3, and offset voltage for piezo 3).

[0024] Figure 3 is a flow diagram illustrating an embodiment of a process for a system for hyperspectral imaging. In some embodiments, the process of Figure 3 is implemented using the system of Figure 1. In the example shown, in 300 a sample is received in a sample holder. For example, a sample is placed in the field of view where the imaging system objective lens is normal to the sample plane, and its field of view matches the area of interest. In some embodiments, a set of objective lenses are employed to capture the area or region of interest at the required spatial resolution. In some embodiments, the selected objective lens has a field of view that matches a desired sampling area. In some embodiments, the position of the sample can be adjusted using one or more sample positioners to manipulate the relative position of the sample and the system for hyperspectral imaging (e.g., moving the sample in the x, y, or z directions where x, y are perpendicular to the imaging axis of the hyperspectral imaging system(e.g., in the plane of the sample) and z is along the axis of the hyperspectral imaging system (e.g., normal to the plane of the sample). In some embodiments, the one or more sample positioners are controlled from a computer. In 302, an indication is received to capture data. For example, the operator initiates the image and data capture via a user interface from a computer system. In 304, etalon free run motion is initiated. For example, the mirror separation distance in the Fabry-Perot etalon is continuously varied by the computer. In 306, free run image frame capture is initiated. For example, the computer captures continuous single frame exposures of multiple band intensity by the sensor and the image frames are saved digitally on a storage medium. In 308, it is determined whether a full set of band passes have been captured. For example, the computer system determines whether the mirror has moved through the entire range of motion and frames are captured such that the full spectral range of interest is captured. In response to determining a full set of band passes have not been captured, control passes to 308. In response to determining a full set of band passes have been captured, control passes to 310. In 310, calibration is applied and a data cube is constructed. For example, a mathematical matrix transform is applied to the data in the frames of images, either to each frame as acquired or to all frames at once at the end of the collection cycle, such that a calibration is applied, and a data cube is constructed (e.g., the calibrated data is organized and stored into a data structure). In some embodiments, the data cube, also known as a hypercube. is a three-dimensional representation of the captured data whereby the two dimensions of the image’s spatial components (x, y coordinates) are augmented by the w avelength in the third dimension. In some embodiments, the number of band-passes required in each data cube varies depending upon the resolution of thespectral signature required to differentiate between samples that need to be detected. For example, for closely similar populations, this can be typically hundreds of band-passes. Such data cubes can be very large files, typically 1 GB each. In 312, classifiers are applied, and metrics are determined. For example, machine learning tools for classification and regression or mathematical characterization are then applied to identify or measure components in the image based on their spectral, geometric, and morphological characteristics to address any number of applications including medical and life science, industrial inspection, defense, and others.

[0025] In 314, it is determined whether to end data capture. For example, an indication is received from a user to end continuous data capture. In response to determining to not end data capture, control passes to 316. In 316, it is indicated to set a next gap to start capture of a next full set of band passes, and control passes to 308. In some embodiments, because the goal of this method is the continuous or streaming acquisition of data, the cycle repeats with the last full set of gap frames being constructed into data-cubes and classified while the next set is being collected and so forth until the operator stops it. In response to determining to end data capture, control passes to 318. In 318, etalon free run motion is stopped. For example, the computer system indicates to a piezo actuator driver to stop motion. In 320, free run image frame capture is stopped.

[0026] In some embodiments, the data cube is analyzed using a combination of one or more supervised machine learning clustering, classification, and regression algorithms (e.g., principal component analysis, K-Means, spectral angle mapping that is first trained on sample data cubes of components, or any other appropriate algorithm).

[0027] In some embodiments, principal component analysis (PCA) is used to simplify complexity in high-dimensional data while retaining trends and patterns by transforming data into fewer dimensions to summarize features.

[0028] In some embodiments, K-Means is an iterative clustering algorithm used to classify data into a specified number of groups by beginning with an initial set of randomly determined cluster centers. Each pixel in the image is then assigned to the nearest cluster center by distance and each cluster center is then recomputed as the centroid of all pixels assigned to the cluster. This process repeats until a desired threshold is achieved.

[0029] In some embodiments, spectral angle mapping (SAM) compares a given spectra to a known spectrum, treating both as vectors and calculating the ‘"spectral angle" between them and grouping them with respect to a threshold based on that angle.

[0030] In various embodiments, classification is based on morphology (i.e., shape) and / or intensify' images (i.e., distribution of brightness). In various embodiments, classification utilizes ahybrid deep-learning framework, long-short term memory (LSTM) network, deep residual network (ResNet), and / or one-dimensional convolutional neural networks (1D-CNN).

[0031] In some embodiments, once trained, these algorithms can be used to recognize, identify, and mathematically group spectral and spatial characteristics associated with specific characteristics of interest in a sample. Such characteristics include, for example, spectral profiles, spectral signatures, the shape and size of the component, etc.

[0032] In some embodiments, the above algorithms have demonstrated the ability to distinguish a variety of components.

[0033] In some embodiments, one or more neural network models are used to detect, identify, and / or classify images- for example, a convolutional neural network (CNN), a regionbased convolutional neural network (R-CNN), a Fast R-CNN model, a Faster R-CNN model, a You Only Look Once (YOLO) model from the family of YOLO models, or any other appropriate neural network model.

[0034] In various embodiments, a bounding box model is used to localize and / or detect images of potential components. In various embodiments, the bounding box model comprises one or more of a Tensor Flow model (e.g., a Tensor Flow application programming interface (API)), a convolutional neural network, a region-based convolutional neural network (R-CNN), a Fast R- CNN model, a Faster R-CNN model, a You Only Look Once (YOLO) model from the family of YOLO models, an EdgeBoxes model, or any other appropriate bounding box model.

[0035] In some embodiments, the bounding box model is trained using the intersection over union (loU) method (i.e., the intersection of the computed bounding box with the bounding box for ground truth) with the loU value set to an appropriate value corresponding to true positives, false positives, and false negatives (e.g., the loU value is set to 0.5, 0.75, 0.9, 0.95, or any appropriate value). An loU value of 1 implies that the predicted and the ground-truth bounding boxes perfectly overlap. For example, using an loU value of > 0.5, the object detection is classified as a true positive (TP). If the loU value is < 0.5, then the object detection is classified as a false positive (FP). If the ground truth is present in the image and the model failed to detect the object, the object detection is classified as a false negative (FN). A true negative (TN) occurs for every part of the image where an object was not predicted: however, since this is not useful for object detection, TN is ignored for the purpose of measuring model performance during training.

[0036] In some embodiments, precision and recall values are calculated and plotted (e.g., as precision and recall or ‘PR graphs’) to evaluate the performance of the bounding box model (e g.,calculated from the number of true positives (TP), false positives (FP), and false negatives (FN) achieved while training the bounding box model). For example, precision and recall are calculated using the formulas Precision = TP / (TP + FP) and Recall = TP / (TP + FN). In some embodiments, the performance of the bounding box model is evaluated using the mean average precision (mAP) value (e.g., using a multi-point interpolated average precision). In some embodiments, the point of operation for the performance of the bounding box model on the generated PR graphs is determined empirically from test data used during training the system (e.g., the bounding box model is set to operate at greater than 50%, 60%, 68%, 72%, 80%, or any other appropriate level of recall). In some embodiments, the point of operation for bounding box model on the generated PR graphs is adjusted after further model training and / or evaluation under operational conditions (e.g., to change the number of generated bounding boxes that require further evaluation).

[0037] In some embodiments, one or more neural network models are used to classify images of potential image components of interest. For example, the neural network model used to classify images of potential components, inclusions or defects (i.e., ‘the classifier model’) assigns a class label to the one or more possible components outlined by a bounding box. In some embodiments, the classifier model uses sharp edge detection. In some embodiments, the classifier model uses machine learning approaches. For example, the classifier model uses one or more neural network models to classify possible components that have been detected in the bounding box model. In various embodiments, the one or more neural network models used by the classifier model are the same, different, or a combination of same or different models as used by the bounding box model. In some embodiments, class labels comprise characteristics of the input image (e.g., texture, shape, or color aspects) that are summarized by numerical vectors (i.e., feature vectors).

[0038] In some embodiments, the one or more neural network models are trained using hierarchical feature learning (i.e., ‘layered learning’). For example, the first layers of the neural network model are trained to leam representations of basic (low-level) features, such as edges. Further down the network, layers output representations of more specific features, corresponding to parts of the object. After the last layer, the representations are high-level and allow a clear separation between different objects with the network having learned to distinguish among the different classes.

[0039] In some embodiments, the one or more neural network models are trained using hyperspectral data (e.g., hyperspectral data from potential components detected by a bounding box model). In some embodiments, one or more neural netw ork models are trained to match theobserved hyperspectral data with a library of hyperspectral data corresponding to known components. In some embodiments, the output of the one or more neural network models provides the class label of an identified image component (e.g., to a system user). In various embodiments, the provided class label of an identified component includes related metadata (e g., time and date, a level of confidence, the number and / or density of the identified component, a plot or graph of the associated hyperspectral data, etc ).

[0040] In the disclosed system, a composite image is generated from the processed grouped data. In some embodiments, the composite image provides an indication of the presence of component (e.g., providing an immediately actionable indication). In some embodiments, a threshold based on statistical likelihood is applied to provide a binary answer or response (e.g., false color image presentation, “go / no go?’ flags or warnings, etc.).

[0041] In some embodiments, the system utilizes machine learning clustering algorithms trained on hyperspectral images of known component of interest.

[0042] In some embodiments, the algorithms utilize both component spectral and spatial (e.g., shape and size) characteristics in their training.

[0043] In various embodiments, wherein the component or features presented are sparsely distributed methods for anomaly detection and targeting at the pixel and sub-pixel scale based on statistical and machine learning algorithms are used to rapidly locate and identify individual cells. In various embodiments, high-speed targeting algorithms utilized in defense and remote sensing / surveillance applications are used to automate the targeting and identification of components in a sparsely populated field of view. For example, airborne remote-sensing signal processing techniques, machine-learning techniques for anomaly detection, and / or orthogonal projection techniques are used to improve the speed and accuracy of the classification algorithms.

[0044] In some embodiments, the UI will provide an option to allow the user to train the algorithm for a component of interest through a guided process (e.g., an application wizard).

[0045] In various embodiments, the instrument is integrated into an in-line inspection system wherein samples are collected for analysis.

[0046] In some embodiments, no accommodations need to be made for the Fabry-Perot etalon stopping, settling, or pausing and / or any coordination between the Fabry-Perot etalon motion and the triggering of acquisition of a sensor frame.

[0047] In some embodiments, it is critical for the mirrors in the Fabry-Perot etalon maintain parallelism to one another throughout the image acquisition process regardless of position ormotion, the rate of transit of the moving mirror elements must be tuned to the mechanical specifics of a given device.

[0048] In some embodiments, this is accomplished through a feedback process where the maximum transit range segments and rates are set by a comparison of the resultant spectra with a known ground truth (i.e., spectrometer, monochromator, etc.).

[0049] In some embodiments, this feedback process is used to establish the mirror gap position indices required to derive the wavelength calibration for each multiple band pass mode of the etalon.

[0050] In some embodiments, the sensor exposure must be commensurate with its frame rate and mirror motion so as to acquire an intensity image for a stable established set of band passes at any given position. In some embodiments, the sensor exposure and etalon mirror motion are controlled and adjusted electronically via firmware and embedded micro-controller or processor.

[0051] Figure 4 is a flow diagram illustrating an embodiment of a process for indicating to drive an etalon free running motion. In some embodiments, the process of Figure 4 is used to implement a process after 304 of Figure 3. In the example shown, in 400 an indication is received for etalon free run motion. For example, a computer provides an indication that the etalon is to start or continue free run motion. In 402, a next drive signal is obtained for the set of positioners. For example, drive signals are obtained for the three piezo positioners for setting the gap between two etalon mirrors (e.g., a look up table is referenced related to a next desired position). In 404, the next drive signals are provided to a set of positioners. For example, the next drive signals are provided to an actuator driver to position the mirror positioners (e.g., three drive signal values are provided to position the mirrors in parallel to each other at a desired gap spacing). In 406, it is determined whether an indication has been received to stop free run motion. For example, the computer provides an indication to stop free run motion (e.g., after data for desired band passes has been collected). In response to determining that an indication has not been received to stop etalon free run motion, control passes to 402. In response to determining that an indication has been received to stop etalon free run motion.

[0052] Figure 5 is a flow diagram illustrating an embodiment of a process for determining a look up table. In some embodiments, the look up table determined in the process of Figure 5 is used for indicating a position to a set of positioners in 402 of Figure 4. In the example shown, in 500 a request is received for next drive signals. For example, during etalon free run motion, a set of drive signals is needed to position the mirrors in parallel at specific gap separations. In 502, a stored look up table index is retrieved, a look up table index is incremented, and the new look up table index isstored. For example, a last stored index is retrieved from memory, the last stored index value is incremented to make a new index value and this new index value is stored. In 504, drive signals associated with the new look up table index are provided. For example, using the new look up table index, the look up table is used to obtain drive signals (e.g., values for the positioners such as piezo actuators setting the gap separation between two etalon mirrors). In 506. it is determined whether the new look up table index is a last look up table index. For example, it is determined whether the index points to the end of the look up table. In response to the new look up table index not being a last look up table index, the process ends. For example, the look up table still has viable next entries for a next request. In response to the new look up table index being a last look up table index, control passes to 508. In 508, the stored look up table index is reset to start value. For example, the stored look up table is set to the value prior to a first look up table entry so that when the look up table index is incremented on the next request, the look up table retrieves the first entryin the look up table.

[0053] Figure 6A is diagram illustrating an embodiment of an object with an overlay. In some embodiments, a hyperspectral imager has been used to image the object and the data used to classify the object. In the example shown, object 600 with different object 601 coupled to it is shown at different times as the object moves (e.g., object 600 shows a person outline as he / she moves - object 602, object 604, object 606, and object 608 showing the person outline during a walking motion at different times). Uniform overlay for object 600 is distinguished from object 601 using the spectrum related to each.

[0054] Figure 6B is a diagram illustrating an embodiment of different spectra. In some embodiments, a hyperspectral imager has measured an object (e.g., object 600 of Figure 6A) and another object (e.g., object 601 of Figure 6A). In the example shown, spectra 610 shows a response versus wavelength ( ) for a first object (e.g., object 600 of Figure 6A). Spectra 612 shows a response versus wavelength (A) for a second object (e.g., object 601 of Figure 6A). The first object and the second object are displayed (e.g., as in Figure 6A) with different overlays to visually distinguish the objects.

[0055] Figure 7 is a diagram illustrating an embodiment of a system for making a calibration measurement for a hyperspectral imaging system. In some embodiments, FPI 122 of Figure 1 is used to implement hyperspectral imager 706 of Figure 7. In the example shown, broadband source 700 generates light that is passed through monochromator 702 to filter the light to a single wavelength of light. This single w avelength of light is split using beam splitter 704 to be measured by spectrometer 722 for a precise measurement of the w avelength and to illuminatehyperspectral imager 706. Hyperspectral imager 706 comprises relay lens 708, etalon 710 comprising mirror 712 and mirror 714 separated by gap 716, relay lens 718, and image sensor 720 In some embodiments, the measurement of the wavelength by spectrometer 722 is done sequentially without beam splitter 704 by disposing spectrometer 722 in the location of hyperspectral imager 706 after hyperspectral imager 706 measurement. A response of hyperspectral imager 706 can then be made for multiple gap spacings. A full calibration is time consuming as it requires setting monochromator 702 at a specific wavelength and then for that specific wavelength measuring a full set of gap spacings. This measurement is then repeated for another specific wavelength until a full set of wavelengths has each been measured at a full set of gap spacings.

[0056] In some embodiments, computer system 724 is used for controlling and / or coordinating the calibration measurement. For example, computer system 724 is able to indicate turning broadband source 700 on or off, indicate selecting a wavelength via monochromator 702, indicate setting a gap spacing for hyperspectral imager 706, indicate acquiring an image using image sensor 720, indicate measuring a wavelength or spectra using spectrometer 722, receive data from hyperspectral imager 706 (e.g., image data, gap data, etc.), from spectrometer 722 (e.g., wavelength or spectra data, etc.), from monochromator 702 (e.g., wavelength setting data, etc.), from broadband source 700 (e.g.. on / off status, etc.), display an interface, receive commands, or any other appropriate function for computer system 724.

[0057] Figure 8 is a flow diagram illustrating a process for calibrating a hyperspectral imager for free-run operation. In some embodiments, the process of Figure 8 is used to calibrate hyperspectral imager 706 of Figure 7. In the example shown, in 800 a broadband source is turned on. In 802, a monochromator source is set to ground truth band corresponding to transmission peak gap n=l. In 804, etalon gap is set to target position c = n+x, where x is minimum offset. In 806, one or more frames is / are acquired associated with the etalon gap. A spectrum is generated by applying a reconstruction algorithm. In 810, it is determined whether the spectrum is within acceptable tolerance to the ground truth. In response to the spectrum not being within acceptable tolerance to ground truth, control passes to 812. In 812, the etalon gap is set to target position c = c+1 and control passes to 806. In response to the spectrum being within acceptable tolerance to ground truth, control passes to 814. In 814. it is determined whether n corresponds to a maximum band number. In response to n not corresponding to a maximum band number, control passes to 81 . In 816, the system goes to the next gap, n+1, and control passes to 806. In response to n corresponding to a maximum band number, control passes to 818. In 818. it is indicated that calibration is complete, and the process ends.

[0058] In some embodiments, ‘n’ comprises an index representing a gap position for a wavelength band of interest for which the instrument is being calibrated to. For a VNIR system, 400 nm would be the first band and “n”, say 1, would the gap where it would be found. In this free- run mode of operation where the mirror is made to move continuously, it turns out programming the driver to go to “n” doesn’t yield the best spectrum. Telling it to overshoot by some distance “x” does. However, once it reaches "‘n” you move on to the next band / gap, say n2 by telling it to go to n2+x and so forth. Determining what “x” is, is the iterative process described above. The calibration process is complete once that offset “x” has been determined for all gaps / bands. These values are stored in the look up table (LUT) and applied at run-time.

[0059] Figure 9 is a flow diagram illustrating an embodiment of a process for calibrating an etalon physical spacing or gap positions. In some embodiments, the gap positions being calibrated are for an etalon of a hyperspectral imager (e g., hyperspectral imager 100 of Figurel). In the example shown, in 900 a first gap position is selected. In 902, piezo positioner(s) is / are adjusted to first position estimates. In 904, parallelness is measured. For example, a uniform laser wave front is imaged through the etalon, and the resulting transmission of the wave is observed for uniformity. In 906, it is determined whether the etalon mirrors are parallel. In response to the mirrors not being parallel, control passes to 902. In response to the mirrors being parallel, control passes to 908. In 908, piezo position settings are stored. For example, the drive signals for the three piezo positioners are stored in a look up table. In some embodiments, the drive signals for the three piezo positioners are stored as fitted functions or as sets of parameters to a functional form. In 910, it is determined whether there are more gap positions. In response to determining that there are more gap positions, control passes to 912. In 912, a next gap position is selected, and control passes to 902. In response to determining that there are no more gap positions, the process ends.

[0060] Although the foregoing embodiments have been described in some detail for purposes of clarity of understanding, the invention is not limited to the details provided. There are many alternative ways of implementing the invention. The disclosed embodiments are illustrative and not restrictive.

Claims

CLAIMS1. A system, comprising: a processor configured to: initiate etalon free run motion; initiate free run image frame capture; determine whether a full set of band passes has been captured; and in response to the full set of band passes having been captured, construct a data cube; and a memory coupled to the processor and configured to provide the processor with instructions.

2. The system of claim 1, wherein the processor is further configured to receive an indication to capture data.

3. The system of claim 1, wherein the processor is further configured to apply a calibration to construct the data cube in response to the full set of band passes having been captured.

4. The system of claim 1, wherein the processor is further configured to apply a classifier using the data cub in response to the full set of band passes having been captured.

5. The system of claim 1, wherein the processor is further configured to determine a metric using the data cube in response to the full set of band passes having been captured.

6. The system of claim 1, wherein the processor is further configured to determine whether to end data capture.

7. The system of claim 6, wherein the processor is further configured to indicate to set a next gap to start capture of a next full set of band passes in response to determining not to end data capture.

8. The system of claim 6, wherein the processor is further configured to indicate to stop etalon free run motion in response to determining to end data capture.

9. The system of claim 6, wherein the processor is further configured to indicate to stop free run image frame capture in response to determining to end data capture.

10. The system of claim 1, wherein the etalon free run motion comprises providing a set of drive signals to piezo actuators of the etalon.

11. The system of claim 1, wherein the set of drive signals for the piezo actuators of the etalon are stored as fitted functions.

12. The system of claim 1, wherein the set of drive signals for the piezo actuators of the etalon are stored in a look up table.

13. The system of claim 12, wherein the look up table includes drive signals for three piezo actuators for a plurality of positions.

14. The system of claim 13, wherein the drive signals for the piezo actuators position the etalon mirrors in parallel at a plurality7of gaps.

15. The system of claim 12, wherein the look up table includes an offset for a position of the plurality of positions.

16. The system of claim 1, wherein the free run image frame capture is not synchronized to the etalon free run motion.

17. The system of claim 1, wherein the full set of band passes comprises bands between N and M bands.

18. The system of claim 17, wherein the full set of band passes are taken using gaps between K and L.

19. A method, comprising: initiating etalon free run motion; initiating free run image frame capture; determining, using a processor, whether a full set of band passes has been captured; and in response to the full set of band passes having been captured, constructing a data cube.

20. A computer program product embodied in a non-transitoiy computer readable medium and comprising computer instructions for: initiating etalon free run motion; initiating free run image frame capture; determining, using a processor, whether a full set of band passes has been captured; and in response to the full set of band passes having been captured, constructing a data cube.

Images