Spectrometer image enhancement

Abstract

A time-series of image frames of spectra is obtained, the spectrum being imaged onto an image sensor by an optical system of a spectrometer, and the frames being recorded while the spectrum moves rela

Description

30 07 24 TECHNICAL FIELD

[0001] The disclosure relates to methods and systems for enhancing the quality of images obtained using an optical spectrometer, specifically although not exclusively, for enhancing resolution. BACKGROUND

[0002] Scientific instruments for obtaining spectral images may include a complex arrangement of movable components, sensors, input and output ports, energy sources, and consumable components. As a result, obtained spectral images are sensitive to changes in the operating conditions of the scientific instrument, for example changes in temperature. Image drift caused, for example, by different parts of the scientific instrument being in a condition of non-equilibrium, can deteriorate image quality and the analytic data of an Inductive Coupled Plasma Optical Emission Spectrometer (ICP-OES). Therefore, many solutions exist to mitigate this drift. Typically, these solutions involve one or more of: thermal decoupling of an optical tank of the scientific instrument from the larger heat sources of the scientific instrument; active thermal stabilization (by means of thermostatic heating or cooling of the optical tank); or post-processing methods that re-align the drifted data in order to generate a spectral image not dependent on operating conditions of the scientific instrument. Other factors detrimental to the spectral image quality include low resolution, dark noise (noise arising from electron patterns) and dead pixels in the spectral images are typically addressed by improving hardware. As such, there is a general need for improving image quality in spectrometers that is cost-efficient and sustainable. SUMMARY

[0003] Disclosed herein are scientific instrument support systems, as well as related methods, computing devices, and computer-readable media. For example, in some embodiments, there is provided, a computer-implemented method of obtaining a digital image of a spectrum from a time-series of images frames of the spectrum imaged onto an image sensor by an optical system of a spectrometer. For example, in some embodiments, the spectrometer is an Inductively Coupled Plasma Optical Emission Spectrometer (ICP-OES). The method comprises obtaining the time-series of image frames of respective spectra recorded while the imaged spectrum moves relative to the image sensor. The method further comprises up-sampling each image frame to generate a respective up-sampled image frame, transforming the up-sampled image frames to align the recorded spectra and ; and combining the transformed image frames to generate the digital image. Up-sampling each image frame may comprise computing additional pixels in between pixels of the sensor by interpolating the array of pixels to generate an interpolated array of pixels. By up-sampling an image frame with additional pixels in between the captured pixels, a higher resolution digital image can be achieved when the up-sampled image frames are combined and the spectrum moves by sub-pixel fractions (or multiples of sub-pixel fractions) relative to the image sensor. The generated digital image may have less or no blank sections resulting from dead pixels of the sensor, since combining the transformed image frames fills in the dead pixels on any given image frame using information from other image frames due to the movement of the 30 07 24 spectrum across the sensor. For example, this can enable high quality digital images to be obtained using relatively inexpensive charge injection device (CID) equipment and can reduce the rate at which sensor chips are rejected due to dead pixels. Additionally, combining the transformed imaged frames, in some embodiments, enables a slight expansion of the detector area which may permit capturing of signals (such as peaks) that would have otherwise “fallen off the detection area. Typically, image drift caused, for example, by different parts of the scientific instrument being in a condition of thermal nonequilibrium, is considered to deteriorate the image quality and the analytic data of, for example, an Inductive Coupled Plasma Optical Emission Spectrometer (ICP-OES) and is hence considered undesirable and to be suppressed. By contrast, the provided method exploits image drift in order to improve the quality of the obtained digital image and thereby loosens the installation constraints related to temperature regulation of the optics. In some embodiments, these constraints may be dispensed with entirely.

[0004] In some embodiments, the method may further comprise recording the time-series of image frames while the imaged spectrum moves relative to the image sensor. In other embodiments, the timeseries is pre-recorded and stored in memory and the method accesses the recorded time series in memory. For example, in some embodiments, the time-series may be or have been recorded on an Inductively Coupled Plasma Optical Emission Spectrometer (ICP-OES).

[0005] In some embodiments, transforming the image frames to align the recorded spectra comprises translating the image frames using a linear translation. In some embodiments transforming the image frames to align the recorded spectra comprises rotating the image frames. More generally, the transforming may comprise any affine or non-affine transformation suitable to align the images, for example a combination of rotation, translation and / or scaling, or the application of a deformation field.

[0006] In some embodiments, the image frames are aligned to a pre-determined image frame in the time-series of image frames, for example, the first image frame in the time-series of image frames. Of course, it is understood that there are other ways to align the image frames not by aligning the frames to a specific frame but instead by mutually aligning the frames and furthermore, that any appropriate method of aligning the image frames may be used.

[0007] In some embodiments, combining the transformed digital image frames comprises averaging corresponding array values of each aligned image frame to generate one array from a number of arrays equal to a number of image frames in the time-series of digital image frames. For example, in some embodiments, the movement of the spectrum may be due to thermal expansion of at least one of the optical components and / or thermal contraction of at least one of the components. For example, the thermal expansion and / or contraction due to temperature variation may be due to normal operation of the spectrometer to obtain a digital image. This enables the benefits discussed above to be achieved with normal operation of a spectrometer (i.e., the method requires no additional or different steps to be taken when capturing the image frames).

[0008] In some embodiments, the movement of the spectra in the image frames is actively induced. For example, the thermal expansion and / or contraction may be due to heating or cooling of the optical system by a temperature control system. This may occur in addition to any variation due to normal operation. In some embodiments, at least some of the movement may be caused by a motion 30 07 24 of at least one actuator coupled to one of the components in the optical system, wherein the actuator may be a motor or piezoelectric actuator. This may induce additional randomness in the drift which may further increase the effective resolution of the digital image due to better sampling of the positioning of features of the image relative to the sensor pixels.

[0009] In some embodiments, there is provided, a system comprising one or more processors and one or more memories having stored thereon computer-readable instructions configured to cause the one or more processors to perform operations comprising one or more of the steps described above. In some embodiments, the system may further comprise a spectrometer, such as an optical emission spectrometer, for example, an ICP-OES, for obtaining the time series of images.

[0010] In some embodiments, there is provided, a computer-readable medium comprising instructions, that, when executed by one or more data processing apparatus, cause the one or more data processing apparatus to perform operations comprising one or more of the steps described above.

[0011] In some embodiments, there is provided, computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out one or more of the steps described above.

[0012] The scientific instrument support embodiments disclosed herein may achieve improved performance relative to conventional approaches. For example, the need to mitigate drift is avoided and drift is in fact exploited in order to improve the resolution of the digital image. The embodiments disclosed herein thus provide improvements to scientific instrument technology (e.g., improvements in the computer technology supporting such scientific instruments, among other improvements).

[0013] The embodiments disclosed herein may achieve better quality data relative to conventional approaches. For example, conventional approaches are limited by the resolution of the image sensor, whereas the disclosed embodiments can achieve a resolution higher than that of the image sensor. Furthermore, the approaches address a number of technical problems and limitations, including dead pixels and image drift caused, for example, by different parts of the scientific instrument being in a condition of thermal non-equilibrium.

[0014] Various ones of the embodiments disclosed herein may improve upon conventional approaches to achieve the technical advantages of improved resolution by up-sampling and combining image frames. Such technical advantages are not achievable by routine and conventional approaches, and all users of systems including such embodiments may benefit from these advantages (e.g., by assisting the user in the performance of a technical task, such as spectroscopy, by means of a guided human-machine interaction process). The technical features of the embodiments disclosed herein are thus decidedly unconventional in the field of analytical chemistry, as are the combinations of the features of the embodiments disclosed herein. The computational and user interface features disclosed herein do not only involve the collection and comparison of information but apply new analytical and technical techniques to change the operation of processing image frames obtained from a spectrometer to obtain higher quality information from the image frames. The present disclosure thus introduces functionality that neither a conventional computing device, nor a human, could perform.

[0015] Accordingly, the embodiments of the present disclosure may serve any of a number of technical purposes, such as controlling a specific technical system or process; determining from 30 07 24 measurements how to control a machine; digital image enhancement or analysis; reducing the amount of sensor data to be processed; or providing a faster processing of sensordata. In particular, the present disclosure provides technical solutions to technical problems, including but not limited to a problem of how to provide better quality data from a spectrometer without changing the equipment.

[0016] The embodiments disclosed herein thus provide improvements to analytical chemistry technology (e.g., improvements in the computer technology supporting analytical chemistry, among other improvements). BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Embodiments will be readily understood by the following detailed description in conjunction with the accompanying drawings. To facilitate this description, like reference numerals designate like structural elements. Embodiments are illustrated by way of example, not by way of limitation, in the figures of the accompanying drawings.

[0018] FIG. 1 is a diagram of an optical system of an example Inductive Coupled Plasma Optical Emission Spectrometer 100 for obtaining optical emission spectra in accordance with some embodiments.

[0019] FIG. 2 is an example optical emission spectrum of an intensity distribution captured by an image sensor.

[0020] FIG. 3 is a block diagram of an example scientific instrument support module 300 performing support operations, in accordance with various embodiments.

[0021] FIG. 4 is a flow diagram of an example method 400 of obtaining a digital image from a timeseries of image frames of a spectral image formed by an optical system of a spectrometer.

[0022] FIG. 5A shows an example of up-sampling image matrices 504 A, B and n corresponding to image frames A, B and n to produce up-sampled image matrices 508 A, B and n.

[0023] FIG. 5B shows an example of aligning up-sampled image matrices 508 A and n to image matrix 508 B.

[0024] FIG. 6 is an example of a graphical user interface 600 that may be used in the performance of some, or all of the support methods disclosed herein in accordance with various embodiments.

[0025] FIG. 7 is a block diagram of an example computing device 700 that may perform some or all of the scientific instrument support methods disclosed herein in accordance with various embodiments.

[0026] FIG. 8 is a block diagram of an example scientific instrument support system 800 in which some or all of the scientific instrument support methods disclosed herein may be performed, in accordance with various embodiments. DETAILED DESCRIPTION

[0027] In the following detailed description, reference is made to the accompanying drawings that form a part hereof wherein like numerals designate like parts throughout, and in which is shown, byway of illustration, embodiments that may be practiced. It is to be understood that other embodiments may be utilized, and structural or logical changes may be made, without departing from the scope of the present disclosure. Therefore, the following detailed description is not to be taken in a limiting sense. 30 07 24

[0028] Various operations may be described as multiple discrete actions or operations in turn, in a manner that is most helpful in understanding the subject matter disclosed herein. However, the order of description should not be construed as to imply that these operations are necessarily order dependent. In particular, these operations may not be performed in the order of presentation. Operations described may be performed in a different order from the described embodiment. Various additional operations may be performed, and / or described operations may be omitted in additional embodiments.

[0029] For the purposes of the present disclosure, the phrases "A and / or B" and "A or B" mean (A), (B), or (A and B). For the purposes of the present disclosure, the phrases "A, B, and / or C" and "A, B, or C" mean (A), (B), (C), (A and B), (A and C), (B and C), or (A, B, and C). Although some elements may be referred to in the singular (e.g., “a processing device”), any appropriate elements may be represented by multiple instances of that element, and vice versa. For example, a set of operations described as performed by a processing device may be implemented with different ones of the operations performed by different processing devices.

[0030] The description uses the phrases "an embodiment," “various embodiments,” and "some embodiments," each of which may refer to one or more of the same or different embodiments. Furthermore, the terms "comprising," "including," "having," and the like, as used with respect to embodiments of the present disclosure, are synonymous. When used to describe a range of dimensions, the phrase "between X and Y" represents a range that includes X and Y. As used herein, an “apparatus” may refer to any individual device, collection of devices, part of a device, or collections of parts of devices. The drawings are not necessarily to scale.

[0031] FIG. 1 is a diagram of an example of a scientific instrument 100 for obtaining spectral images in accordance with various embodiments. The example scientific instrument shown in FIG. 1 is an optical emission spectrometer, for example an Inductive Coupled Plasma Optical Emission Spectrometer (ICP-OES). The ICP-OES 100 may be a component in a scientific support system 800, for example, as described below with reference to FIG. 8. Scientific instruments, such as the ICP-OES 100 may comprise a complex arrangement of movable components, sensors, input and output ports, energy sources, and consumable components. As a result, obtained spectral images are sensitive to changes in the operating conditions of the scientific instrument, for example changes in temperature. Emission spectrometers operate by exciting atoms and ions to emit electromagnetic (EM) radiation at wavelengths characteristic of a particular element. A spectrum of frequencies of EM radiation is emitted due to an electron making a transition from a high energy state to a lower energy state. The ICP-OES 100 may comprise a plasma chamber 102. The plasma chamber 102 may be connected to a radio frequency (RF) source and a source of gas, for example argon gas. The argon gas may be ionized inside an oscillating RF field produced by the RF source to develop and maintain plasma inside the plasma chamber 102. The ICP-OES 100 may be of an echelle-based optical design to produce a two-dimensional optical emission spectrum.

[0032] Specifically, the ICP-OES may comprise an echelle diffraction grating 114, a prism 112 and multiple focusing mirrors 104,106,110,116. Collectively, these components provide an optical system 124. Light from the plasma chamber 102 enters the ICP-OES 100 and is selectively focused using 30 07 24 multiple focusing mirrors, for example a first 104 and a second 106 mirror. The focused light may be passed through an entrance slit 108 and into the prism 112 using the mirror 110. The prism 112 may separate the light by wavelength. The echelle diffraction grating 114 may diffract the separated light from the prism 112 into multiple diffraction orders, creating a high-resolution 2D spectrum known as an echellogram or echelle spectrum. After passing through these optical elements, the mirror 116 may collect and focus the spectrum onto an image sensor, for example the camera 118 in order to generate an image frame. FIG. 2 depicts an example image frame 200, in accordance with various embodiments. Multiple peaks are seen as white dots of varying intensities.

[0033] Image frames, such as the image frame 200 shown in FIG. 2 are sensitive to changes in the operating conditions of the scientific instrument, for example changes in temperature. Image drift caused, for example, by different parts of the scientific instrument being in a condition of non-equilibrium, can deteriorate the image quality. For example, changes in temperature may cause drift of spectral peaks in the image frame.

[0034] The optical system 124 is enclosed in a housing. One or more heating pads 122 may be located on an outside surface of the housing to allow the temperature of the optical system 124 to be controlled. A temperature sensor 120 may indicate the temperature of, for example, an environment of the ICP-OES 100. In one embodiment, the temperature sensor 120 may indicate the temperature of the ICP-OES 100, or the optical system 124 of the ICP-OES 100 comprising the optical components. In some embodiments, a actuator 128 may be coupled to one or more of components of the optical system. For example, in some embodiments, the actuator 128 may be coupled to one or more focusing mirrors of the multiple focusing mirrors. For example, the actuator 128 may be coupled to the mirror 116. The actuator may be any appropriate component for inducing motion in the optical system. For example, the actuator may be a motor or piezoelectric actuator.

[0035] The number, identity and location of the components described above with reference to FIG.1 are used illustratively and it is understood that other configurations and / or optical components may be used. Although the embodiments are described with reference to an inductive coupled plasma optical emission spectrometer, they are equally applicable to other types of spectral images obtained with other types of spectrometers. For example, the methods described herein may be used for similar spectroscopic techniques such as inductively coupled plasma mass spectrometry (ICP-MS) and atomic absorption spectrometry (AAS) and, indeed, may be applied to any method of obtaining spectral images, where the spectral images are dependent on an operating condition of the spectrometer. Various embodiments include the use of spectroscopic techniques such as Rayleigh, Raman or Mass spectrometry.

[0036] FIG. 3 is a block diagram of an example scientific instrument support module 300 performing support operations, in accordance with various embodiments.

[0037] The scientific instrument support module 300 may be implemented by circuitry (e.g., including electrical and / or optical components), such as a programmed computing device. The logic of the scientific instrument support module 300 may be included in a single computing device or may be distributed across multiple computing devices that are in communication with each other as appropriate. Examples of computing devices that may, singly or in combination, implement the scientific instrument 30 07 24 support module 300 are discussed herein with reference to the computing device 700 of FIG. 7, and examples of systems of interconnected computing devices, in which the scientific instrument support module 300 may be implemented across one or more of the computing devices, is discussed herein with reference to the scientific instrument support system 800 of FIG. 8.

[0038] The scientific instrument support module 300 may comprise a data acquisition logic 302, a transformation logic 304 and a merge logic 306. As used herein, the term “logic” may include an apparatus that is to perform a set of operations associated with the logic. For example, any of the logic elements included in the support module 300 may be implemented by one or more computing devices programmed with instructions to cause one or more processing devices of the computing devices to perform the associated set of operations. In a particular embodiment, a logic element may include one or more non-transitory computer-readable media having instructions thereon that, when executed by one or more processing devices of one or more computing devices, cause the one or more computing devices to perform the associated set of operations. As used herein, the term “module” may refer to a collection of one or more logic elements that, together, perform a function associated with the module. Different ones of the logic elements in a module may take the same form or may take different forms. For example, some logic in a module may be implemented by a programmed general-purpose processing device, while other logic in a module may be implemented by an application-specific integrated circuit (ASIC). In another example, different ones of the logic elements in a module may be associated with different sets of instructions executed by one or more processing devices. A module may not include all of the logic elements depicted in the associated drawing; for example, a module may include a subset ofthe logic elements depicted in the associated drawing when that module is to perform a subset ofthe operations discussed herein with reference to that module.

[0039] The data acquisition logic 302 may be configured to acquire a time-series of image frames of a spectrum imaged onto an image sensor by an optical system of a spectrometer while the imaged spectrum moves relative to the image sensor. The spectrum movement may be due to normal operation ofthe spectrometer, due to imposed movement ofthe optical system ora combination therein. In some embodiments, the data acquisition logic 302 may be configured to obtain a time-series of image frames from a database of image frames. In some embodiments, the data acquisition logic 302 may be configured to record a time-series of image frames using the ICP-OES 100.

[0040] The transformation logic 304 may be configured to align the time-series of image frames by transforming the image frames. For example, in some embodiments, the transformation logic 304 may be configured to align the image frames by determining a base image frame, representing the image frames as a matrix of pixels or sub-pixels, using phase correlation to obtain an offset between each image frame and the base image frame and applying a transformation based on the offset in order to remove the offset. This is described in more detail below with reference to FIG. 4.

[0041] The merge logic 306 may be configured to merge the transformed image frames to generate a digital image. For example, in some embodiments, merging the transformed image frames to generate a digital image may comprise averaging the intensity across the aligned pixels or sub-pixels to generate the digital image. The logic blocks 302 to 304 of support module 300 may perform any ofthe operations described below with reference to FIG. 4. 30 07 24

[0042] FIG. 4 is a flow diagram of an example of a method 400 of obtaining a digital image from a time-series of image frames of a spectral image. The spectral image may be formed by an optical system of a spectrometer wherein the optical system comprises an image sensor having an array of pixels defining a sensor resolution, in accordance with various examples. The method 400 uses a time-series of image frames of a first resolution to generate a merged image of a second, higher resolution by aligning the image frames in the time series, for example aligning each image frame with a single image frame in the time series. In some embodiments, the method comprises estimating a per pixel contribution of each image frame to the merged image, while in other embodiments the aligned image frames are combined with equal weight, for example averaged. In the present disclosure the term “sensor resolution” refers to the number of pixels in an image frame of a spectral image obtained from the image sensor. For example, a sensor resolution may be described using the pixel dimension, such as 1920 x 1080 pixels, describing the number of horizonal and vertical pixels in an image frame of a spectral image. When displayed electronically, the sensor resolution may therefore be thought of as a pixel density, since different resolutions may be displayed on the same area of a user interface. For example, if viewed on a consistently sized screen, the higher the pixel resolution (the more pixels) the higher the density of pixels displayed on the screen and therefore the detail of the image shown on the electronic screen.

[0043] The method 400 may use image frames 200 obtained using an optical system comprising, for example, the Inductive Coupled Plasma Optical Emission Spectrometer (ICP-OES) 100 described above with reference to FIG. 1. The steps of method 400 may be performed by the scientific instrument support module 300 described in FIG. 3.

[0044] Although the operations of the method 400 may be illustrated with reference to particular embodiments disclosed herein (e.g., a graphical user interface (GUI) 600 discussed herein with reference to FIG. 6, the computing devices 700 discussed herein with reference to FIG. 7, and / or the scientific instrument support system 800 discussed herein with reference to FIG. 8), the method 400 may be used in any suitable setting to perform any suitable support operations. Operations are illustrated once each and in a particular order in FIG. 4, but the operations may be reordered and / or repeated as desired and appropriate (e.g., operations may be performed in parallel, as suitable).

[0045] The method 400 comprises first second, third and fourth operations at, respectively, 402, 404, 406 and 408

[0046] At 402, first operations may be performed. The first operations may comprise obtaining a timeseries of image frames of respective spectra imaged onto an image sensor by an optical system of a spectrometer recorded while the imaged spectrum moves relative to the image sensor. Step 402 may be controlled, for example, by the data acquisition logic 302. For example, in some embodiments, the spectrometer may be the ICP-OES 100 described above with reference to FIG. 1, the image sensor may be the camera 118 and the image frames may be image frames 200 shown in FIG. 2. The first operations 402 may comprise acquiring the image frames by requesting or receiving image frames from any appropriate device capable of storing the image frames. For example, the data acquisition logic 302 may request or receive image frames stored in memory of a device in communication with the ICP-OES 100 or a computing device 700 with memory on which multiple image frames are stored. 30 07 24 For example, in some embodiments, step 402 may comprise acquiring a time-series of image frames while the imaged spectrum moves relative to the image sensor due to un-imposed thermal drift resulting from conventional operation of the spectrometer to acquire image frames (also described herein as normal operation of the spectrometer). The data acquisition logic 302 may be configured to acquire and store image frames. The image frames may comprise a collection of pixels with pixel values reflecting the captured intensity distribution. The pixel values may be grey-scale scalar values or vector values representing color information, for example, representing the image frames as a matrix of pixel values.

[0047] At step 404, second operations may be performed. The second operations, at 404, may be performed, for example, by the transformation logic 304. The second operations may comprise, subsequent to the first operations and before the third operations, representing each image frame as a matrix of pixels and up-sampling the matrix to generate a matrix of sub-pixels, each point in the new matrix corresponding to a multiple of a fraction of a pixel, with the spacing between sub-pixels being a fraction of the spacing between the pixels. This is illustrated in FIG. 5A. FIG. 5A shows image frames A, B and n with a captured image of a peak 502. The peak 502 drifts with the spectrum from image frames 504A to 504n. On the left of FIG. 5A, image frames A, B and n are represented by a 3 x 3 matrix of pixels, 504A, 504B and 504n, respectively. For illustrative purposes the 3 x 3 matrix of pixels 504A to 504n represents the pixels captured by the camera 118, however, it is understood that in reality the matrix captured by the camera 118 is much larger and the described method applies to any captured matrix of pixels. Furthermore, on the right of FIG. 5A, image frames A, B and n are represented by an up-sampled matrix of 5 x 5 sub-pixels, filling in a sub-pixel between each of the pixels. For example, in some embodiments, the matrix of sub-pixels may be created by adding sub-pixels in between each pixel in order to interpolate the captured matrix of pixels 504A to 504n to obtain a new matrix of sub-pixels 508A to 508n. This process is referred to herein as up-sampling the captured matrix of pixels. It is understood that the matrices of obtained pixels 504A, 504B and 504n may be up-sampled by any appropriate method such as linear interpolation as described above or, for example, kernel regression. Notably, FIG. 5A shows the peak position is captured by just one sub-pixel in each up-sampled matrix 508A, 508B and 508C, for example, each peak is captured by one value in the matrix of sub-pixels which has a different intensity value to neighboring intensity values. The same peak is captured by two pixels in the matrix 504B, for example, the same sized peak would be represented by two values in the matrix of pixels 504B. Although both matrices show the same peak, the peak in the up-sampled matrix appears smaller and (for example) white whereas the peak in matrix 504B appears larger and (for example) grey (the one-pixel white peak appears spread out in lower resolution). Of course, in reality peaks will be represented by multiple pixels and different areas of the peak will appear in different intensities, but the one pixel peak has been used as an illustrative example of improved resolution from up-sampling.

[0048] At 406, third operations may be performed. The third operation may comprise transforming the image frames to align the recorded spectra. The second operations may be performed, for example, by the transformation logic 304.

[0049] The third operations at 406 are performed on the up-sampled matrices, specifically aligning the up-sampled image frames A, B and n and the sub-pixel values are moved, for example shifted, in 30 07 24 units of sub-pixels in order to align the image. This is illustrated in FIG. 5B. FIG. 5B shows an example of aligning up-sampled matrices 508 A, B and n by applying a linear transformation, specifically a translation, to image frames A and n to align them to image frames B. The respective translations for each of image frames A and n are illustrated by the dotted arrows in Figure 5B.

[0050] In some embodiments, aligning the time-series of image-frames may comprise representing each image frame as a matrix, wherein each value in the matrix corresponds to a pixel value at a coordinate of the image frame. For example, an image frame with pixel dimensions 10 X 10 is represented as a 10 x 10 matrix, with each value of the matrix corresponding to the pixel intensity at each point in the image frame. In such embodiments, pixel values are shifted in units of pixels in order to align the image.

[0051] In some embodiments, the image frames may be aligned to a base image in the time series of image frames. For example, the image frames may be aligned to a first image in the time series. In some embodiments, the transformation logic 304 may align each image-frames to the base image by using phase correlation to obtain an offset between each image frame and the base image frame and applying a transformation based on the offset in order to remove the offset. Of course, it is understood that any image frame in the time-series may be used as the base image.

[0052] The transformation applied to the image frames may be any appropriate transformation to align the image frames. For example, in some embodiments, aligning the time-series of image-frames comprises applying a linear translation to the image frames. For example, in some embodiments, aligning the time-series of image-frames comprises applying a rotation to the image frames. For example, in some embodiments, aligning the time-series of image-frames comprises applying a deformation to the image frames. Of course, it is understood that any combination of the above transformations may be used and indeed that any appropriate transformation operation may be used to align the image frames.

[0053] At 408, fourth operations may be performed. The fourth operations may comprise combining the transformed image frames to generate the digital image. The fourth operations may be performed, for example, by the merge logic 306. As discussed above with reference to FIG.3, combining the transformed image frames may comprise averaging the intensity across the aligned sub-pixels to generate the digital image.

[0054] Step 408 comprises merging the image frames to generate a digital image by averaging the values of sub-pixels across aligned image frames. In such embodiments, an effective higher resolution digital image can be obtained when the up-sampled image frames are combined, and the spectrum moves by multiples of sub-pixels (for example 3 sub-pixels or 1.5 pixels) relative to the camera 118. Of course, it is understood that other examples of up-sampling may be used where one pixel is up-sampled to any appropriate number of sub-pixels. Put differently, the number of pixels may be increased by any appropriate amount (200%, 205%, 325% etc). In some examples 1 pixel may be up-sampled to 10 subpixels in which case a drift of 1.5 pixels in, say, an x direction would be equivalent to 15 sub-pixels in the x direction. In some examples 1 pixel may be up-sampled to 20 subpixels in which case a drift of 1.5 pixels would be equivalent to 30 sub-pixels. 30 07 24

[0055] The methods described herein exploit image drift caused by movement of components in the optical system to enhance digital image quality. In some embodiments it may be desirable to deliberately induce drift in addition to the drift resulting from normal operation of the ICP-OES 100. The deliberately induced drift may be induced by imposing thermal changes in the optical system or by mechanically moving of components in the optical system or both in order to enhance the digital image.

[0056] For example, in some embodiments, the first operations at 402 may comprise acquiring a timeseries of image frames while the imaged spectrum moves relative to the image sensor due to imposed thermal drift. The imposed thermal drift may be in addition to the drift resulting from normal operation of the ICP-OES 100 and may be exploited to further improve image quality. For example, in some embodiments, the data acquisition logic 302 may be configured to control the ICP-OES 100 to alter an operating condition of the ICP-OES 100 in order to impose thermal drift by controlling the heating pad 122 whilst acquiring time-series of images in order to induce a further drift in the images. For example, the data acquisition logic 302 may be configure control the heating pads to cycle the temperature between 25 and 42 degrees Celsius whilst acquiring image frames.

[0057] In some embodiments, the first operations at 402 may comprise acquiring a time-series of image frames of a spectrum imaged onto an image sensor by an optical system of a spectrometer while the imaged spectrum moves relative to the image sensor due to imposed drift as a result of any other imposed movement in the optical system. For example, in some embodiments, the data acquisition logic 302 may be configured to control the ICP-OES 100 to operate the actuator 128 coupled to a component in the optical system, for example, the mirror 116, to mechanically vibrate the mirror 116 or other component back and forth in a repeating or pseudo-random motion.

[0058] The inventors have found that by letting the analytical image move due to unmitigated thermal drift, actively induced thermal drift, other periodic or random movements in the optical system, such as movement caused by a motor or piezoelectric actuator as described above, or a combination thereof, and by acquiring a time series of image frames, it is possible to generate a high-resolution digital image from multiple low-resolution image frames.

[0059] FIG. 6 depicts an example GUI 600 that may be used in the performance of some, or all of the support methods disclosed herein, in accordance with various embodiments. As noted above, the GUI 600 may be provided on a display device (e.g., the display device 710 discussed herein with reference to FIG. 7) of a computing device (e.g., the computing device 700 discussed herein with reference to FIG. 7) of a scientific instrument support system (e.g., the scientific instrument support system 800 discussed herein with reference to FIG. 8), and a user may interact with the GUI 600 using any suitable input device (e.g., any of the input devices included in the other I / O devices 712 discussed herein with reference to FIG. 7) and input technique (e.g., movement of a cursor, motion capture, facial recognition, gesture detection, voice recognition, actuation of buttons, etc.).

[0060] The GUI 600 may include a data display region 602, a data analysis region 604, a scientific instrument control region 606, and a settings region 608. The particular number and arrangement of regions depicted in FIG. 6 is simply illustrative, and any number and arrangement of regions, including any desired features, may be included in a GUI 600. 30 07 24

[0061] The data display region 602 may display data generated by a scientific instrument (e.g,, the scientific instrument 610 discussed herein with reference to FIG. 6). For example, the data display region 602 may display information pertaining to the image frames, for example, the number of frames or the operating conditions of the optical system under which the frames are obtained.

[0062] The data analysis region 604 may display the results of data analysis (e.g., the results of analyzing the data illustrated in the data display region 602 and / or other data). For example, the data analysis region 604 may display information related to the drift between image frames or resolution information, for example of the frames or of the obtained digital image. In some embodiments, the data display region 602 and the data analysis region 604 may be combined in the GUI 600 (e.g., to include data output from a scientific instrument, and some analysis of the data, in a common graph or region).

[0063] The scientific instrument control region 606 may include options that allow the user to control a scientific instrument (e.g., the scientific instrument 810 discussed herein with reference to FIG. 8). For example, the scientific instrument control region 606 may include options to control the temperature of the optical system.

[0064] The settings region 608 may include options that allow the user to control the features and functions of the GUI 600 (and / or other GUIs) and / or perform common computing operations with respect to the data display region 602 and data analysis region 604 (e.g., saving data on a storage device, such as the storage device 704 discussed herein with reference to FIG. 7, sending data to another user, labeling data, etc.). Computing infrastructure

[0065] As noted above, the scientific instrument support module 200 may be implemented by one or more computing devices. FIG. 7 is a block diagram of a computing device 700 that may perform some or all of the scientific instrument support methods disclosed herein, in accordance with various embodiments. In some embodiments, the scientific instrument support module 200 may be implemented by a single computing device 700 or by multiple computing devices 700. Further, as discussed below, a computing device 700 (or multiple computing devices 700) that implements the scientific instrument support module 1000 may be part of one or more of the scientific instruments 810, the user local computing device 820, the service local computing device 810, or the remote computing device 840 of FIG. 8.

[0066] The computing device 700 of FIG. 7 is illustrated as having a number of components, but any one or more of these components may be omitted or duplicated, as suitable for the application and setting. In some embodiments, some or all of the components included in the computing device 700 may be attached to one or more motherboards and enclosed in a housing (e.g., including plastic, metal, and / or other materials). In some embodiments, some these components may be fabricated onto a single system-on-a-chip (SoC) (e.g., an SoC may include one or more processing devices 702 and one or more storage devices 704). In some embodiments, one or more of these components may be remote to the computing device 700. Additionally, in various embodiments, the computing device 700 may not include one or more of the components illustrated in FIG. 7, but may include interface circuitry (not shown) for coupling to the one or more components using any suitable interface (e.g., a Universal Serial 30 07 24 Bus (USB) interface, a High-Definition Multimedia Interface (HDMI) interface, a Controller Area Network (CAN) interface, a Serial Peripheral Interface (SPI) interface, an Ethernet interface, a wireless interface, or any other appropriate interface) . For example, the computing device 700 may not include a display device 710, but may include display device interface circuitry (e.g., a connector and driver circuitry) to which a display device 710 may be coupled.

[0067] The computing device 700 may include a processing device 702 (e.g., one or more processing devices). As used herein, the term "processing device" may refer to any device or portion of a device that processes electronic data from registers and / or memory to transform that electronic data into other electronic data that may be stored in registers and / or memory. The processing device 11002 may include one or more digital signal processors (DSPs), application-specific integrated circuits (ASICs), central processing units (CPUs), graphics processing units (GPUs), cryptoprocessors (specialized processors that execute cryptographic algorithms within hardware), server processors, or any other suitable processing devices.

[0068] The computing device 700 may include a storage device 704 (e.g., one or more storage devices). The storage device 704 may include one or more memory devices such as random access memory (RAM) (e.g., static RAM (SRAM) devices, magnetic RAM (MRAM) devices, dynamic RAM (DRAM) devices, resistive RAM (RRAM) devices, or conductive-bridging RAM (CBRAM) devices), hard drive-based memory devices, solid-state memory devices, networked drives, cloud drives, or any combination of memory devices. In some embodiments, the storage device 704 may include memory that shares a die with a processing device 702. In such an embodiment, the memory may be used as cache memory and may include embedded dynamic random access memory (eDRAM) or spin transfer torque magnetic random access memory (STT-MRAM), for example. In some embodiments, the storage device 704 may include non-transitory computer readable media having instructions thereon that, when executed by one or more processing devices (e.g., the processing device 702), cause the computing device 700 to perform any appropriate ones of or portions of the methods disclosed herein.

[0069] The computing device 700 may include an interface device 706 (e.g., one or more interface devices 706). The interface device 706 may include one or more communication chips, connectors, and / or other hardware and software to govern communications between the computing device 700 and other computing devices. For example, the interface device 706 may include circuitry for managing wireless communications for the transfer of data to and from the computing device 700. The term "wireless" and its derivatives may be used to describe circuits, devices, systems, methods, techniques, communications channels, etc., that may communicate data through the use of modulated electromagnetic radiation through a nonsolid medium. The term does not imply that the associated devices do not contain any wires, although in some embodiments they might not. Circuitry included in the interface device 706 for managing wireless communications may implement any of a number of wireless standards or protocols, including but not limited to Institute for Electrical and Electronic Engineers (IEEE) standards including Wi-Fi (IEEE 802.11 family), IEEE 802.16 standards (e.g., IEEE 802.16-2005 Amendment), Long-Term Evolution (LTE) project along with any amendments, updates, and / or revisions (e.g., advanced LTE project, ultra-mobile broadband (UMB) project (also referred to as "3GPP2"), etc.). In some embodiments, circuitry included in the interface device 706 for managing 30 07 24 wireless communications may operate in accordance with a Global System for Mobile Communication (GSM), General Packet Radio Service (GPRS), Universal Mobile Telecommunications System (UMTS), High Speed Packet Access (HSPA), Evolved HSPA (E-HSPA), or LTE network. In some embodiments, circuitry included in the interface device 706 for managing wireless communications may operate in accordance with Enhanced Data for GSM Evolution (EDGE), GSM EDGE Radio Access Network (GERAN), Universal Terrestrial Radio Access Network (UTRAN), or Evolved UTRAN (E-UTRAN). In some embodiments, circuitry included in the interface device 706 for managing wireless communications may operate in accordance with Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Digital Enhanced Cordless Telecommunications (DECT), Evolution-Data Optimized (EV-DO), and derivatives thereof, as well as any other wireless protocols that are designated as 3G, 4G, 5G, and beyond. In some embodiments, the interface device 706 may include one or more antennas (e.g., one or more antenna arrays) to receipt and / or transmission of wireless communications.

[0070] In some embodiments, the interface device 706 may include circuitry for managing wired communications, such as electrical, optical, or any other suitable communication protocols. For example, the interface device 706 may include circuitry to support communications in accordance with Ethernet technologies. In some embodiments, the interface device 706 may support both wireless and wired communication, and / or may support multiple wired communication protocols and / or multiple wireless communication protocols. For example, a first set of circuitry of the interface device 706 may be dedicated to shorter-range wireless communications such as Wi-Fi or Bluetooth, and a second set of circuitry of the interface device 706 may be dedicated to longer-range wireless communications such as global positioning system (GPS), EDGE, GPRS, CDMA, WiMAX, LTE, EV-DO, or others. In some embodiments, a first set of circuitry of the interface device 706 may be dedicated to wireless communications, and a second set of circuitry of the interface device 706 may be dedicated to wired communications.

[0071] The computing device 700 may include battery / power circuitry 708. The battery / power circuitry 708 may include one or more energy storage devices (e.g., batteries or capacitors) and / or circuitry for coupling components of the computing device 700 to an energy source separate from the computing device 700 (e.g., AC line power).

[0072] The computing device 700 may include a display device 710 (e.g., multiple display devices). The display device 710 may include any visual indicators, such as a heads-up display, a computer monitor, a projector, a touchscreen display, a liquid crystal display (LCD), a light-emitting diode display, ora flat panel display.

[0073] The computing device 700 may include other input / output (I / O) devices 712. The other I / O devices 712 may include one or more audio output devices (e.g., speakers, headsets, earbuds, alarms, etc.), one or more audio input devices (e.g., microphones or microphone arrays), location devices (e.g., GPS devices in communication with a satellite-based system to receive a location of the computing device 700, as known in the art), audio codecs, video codecs, printers, sensors (e.g., thermocouples or other temperature sensors, humidity sensors, pressure sensors, vibration sensors, accelerometers, gyroscopes, etc.), image capture devices such as cameras, keyboards, cursor control devices such as 30 07 24 a mouse, a stylus, a trackball, ora touchpad, barcode readers, Quick Response (QR) code readers, or radio frequency identification (RFID) readers, for example.

[0074] The computing device 700 may have any suitable form factor for its application and setting, such as a handheld or mobile computing device (e.g., a cell phone, a smart phone, a mobile internet device, a tablet computer, a laptop computer, a netbook computer, an ultrabook computer, a personal digital assistant (PDA), an ultra-mobile personal computer, etc.), a desktop computing device, or a server computing device or other networked computing component.

[0075] One or more computing devices implementing any of the scientific instrument support modules or methods disclosed herein may be part of a scientific instrument support system. FIG. 8 is a block diagram of an example scientific instrument support system 800 in which some or all of the scientific instrument support methods disclosed herein may be performed, in accordance with various embodiments. The scientific instrument support modules and methods disclosed herein (e.g., the scientific instrument support module 100 of FIG. 2 and the methods 5000, 6000, 7000 of FIG. 5, 6, 7) may be implemented by one or more of the scientific instruments 810, the user local computing device 820, the service local computing device 830, or the remote computing device 840 of the scientific instrument support system 800.

[0076] Any of the scientific instrument 810, the user local computing device 820, the service local computing device 830, or the remote computing device 840 may include any of the embodiments of the computing device 700 discussed herein with reference to FIG. 7, and any of the scientific instrument 810, the user local computing device 820, the service local computing device 830, or the remote computing device 840 may take the form of any appropriate ones of the embodiments of the computing device 700 discussed herein with reference to FIG. 7.

[0077] The scientific instrument 810, the user local computing device 820, the service local computing device 830, or the remote computing device 840 may each include a processing device 802, a storage device 804, and an interface device 806. The processing device 802 may take any suitable form, including the form of any of the processing devices 702 discussed herein with reference to FIG. 7, and the processing devices 802 included in different ones of the scientific instrument 810, the user local computing device 820, the service local computing device 830, or the remote computing device 840 may take the same form or different forms. The storage device 804 may take any suitable form, including the form of any of the storage devices 704 discussed herein with reference to FIG. 7, and the storage devices 804 included in different ones of the scientific instrument 810, the user local computing device 820, the service local computing device 830, or the remote computing device 840 may take the same form or different forms. The interface device 806 may take any suitable form, including the form of any of the interface devices 706 discussed herein with reference to FIG. 7, and the interface devices 806 included in different ones of the scientific instrument 810, the user local computing device 820, the service local computing device 830, or the remote computing device 840 may take the same form or different forms.

[0078] The scientific instrument 810, the user local computing device 820, the service local computing device 830, and the remote computing device 840 may be in communication with other elements of the scientific instrument support system 800 via communication pathways 808. The 30 07 24 communication pathways 808 may communicatively couple the interface devices 806 of different ones of the elements of the scientific instrument support system 800, as shown, and may be wired or wireless communication pathways (e.g., in accordance with any of the communication techniques discussed herein with reference to the interface devices 706 of the computing device 700 of FIG. 7). The particular scientific instrument support system 800 depicted in FIG. 8 includes communication pathways between each pair of the scientific instrument 810, the user local computing device 820, the service local computing device 830, and the remote computing device 840, but this “fully connected” implementation is simply illustrative, and in various embodiments, various ones of the communication pathways 808 may be absent. For example, in some embodiments, a service local computing device 830 may not have a direct communication pathway 808 between its interface device 806 and the interface device 806 of the scientific instrument 810, but may instead communicate with the scientific instrument 810 via the communication pathway 808 between the service local computing device 830 and the user local computing device 820 and the communication pathway 808 between the user local computing device 820 and the scientific instrument 810.

[0079] The scientific instrument 810 may include any appropriate scientific instrument, such as an optical emission spectrometer, for example the inductive coupled plasma optical emission spectrometer 1000 as shown in FIG. 1.

[0080] The user local computing device 820 may be a computing device (e.g., in accordance with any of the embodiments of the computing device 700 discussed herein) that is local to a user of the scientific instrument 810. In some embodiments, the user local computing device 820 may also be local to the scientific instrument 810, but this need not be the case; for example, a user local computing device 820 that is in a user’s home or office may be remote from, but in communication with, the scientific instrument 810 so that the user may use the user local computing device 820 to control and / or access data from the scientific instrument 810. In some embodiments, the user local computing device 820 may be a laptop, smartphone, or tablet device. In some embodiments the user local computing device 820 may be a portable computing device.

[0081] The service local computing device 830 may be a computing device (e.g., in accordance with any of the embodiments of the computing device 700 discussed herein) that is local to an entity that services the scientific instrument 810. For example, the service local computing device 830 may be local to a manufacturer of the scientific instrument 810 or to a third-party service company. In some embodiments, the service local computing device 830 may communicate with the scientific instrument 810, the user local computing device 820, and / or the remote computing device 840 (e.g., via a direct communication pathway 808 or via multiple “indirect” communication pathways 808, as discussed above) to receive data regarding the operation of the scientific instrument 810, the user local computing device 820, and / or the remote computing device 840 (e.g., the results of self-tests of the scientific instrument 810, calibration coefficients used by the scientific instrument 810, the measurements of sensors associated with the scientific instrument 810, etc.). In some embodiments, the service local computing device 830 may communicate with the scientific instrument 810, the user local computing device 820, and / or the remote computing device 840 (e.g., via a direct communication pathway 808 or via multiple “indirect” communication pathways 808, as discussed above) to transmit data to the 30 07 24 scientific instrument 810, the user local computing device 820, and / or the remote computing device 840 (e.g., to update programmed instructions, such as firmware, in the scientific instrument 810, to initiate the performance of test or calibration sequences in the scientific instrument 810, to update programmed instructions, such as software, in the user local computing device 820 or the remote computing device 840, etc.). A user of the scientific instrument 810 may utilize the scientific instrument 810 or the user local computing device 820 to communicate with the service local computing device 830 to report a problem with the scientific instrument 810 or the user local computing device 820, to request a visit from a technician to improve the operation of the scientific instrument 810, to order consumables or replacement parts associated with the scientific instrument 810, or for other purposes.

[0082] The remote computing device 840 may be a computing device (e.g., in accordance with any of the embodiments of the computing device 700 discussed herein) that is remote from the scientific instrument 810 and / or from the user local computing device 820. In some embodiments, the remote computing device 840 may be included in a datacenter or other large-scale server environment. In some embodiments, the remote computing device 840 may include network-attached storage (e.g., as part of the storage device 804). The remote computing device 840 may store data generated by the scientific instrument 810, perform analyses of the data generated by the scientific instrument 810 (e.g., in accordance with programmed instructions), facilitate communication between the user local computing device 820 and the scientific instrument 810, and / or facilitate communication between the service local computing device 830 and the scientific instrument 810.

[0083] In some embodiments, one or more of the elements of the scientific instrument support system 800 illustrated in FIG. 8 may not be present. Further, in some embodiments, multiple ones of various ones of the elements of the scientific instrument support system 800 of FIG. 8 may be present. For example, a scientific instrument support system 800 may include multiple user local computing devices 820 (e.g., different user local computing devices 820 associated with different users or in different locations). In another example, a scientific instrument support system 800 may include multiple scientific instruments 810, all in communication with service local computing device 830 and / or a remote computing device 840; in such an embodiment, the service local computing device 830 may monitor these multiple scientific instruments 810, and the service local computing device 830 may cause updates or other information may be “broadcast” to multiple scientific instruments 810 at the same time. Different ones of the scientific instruments 810 in a scientific instrument support system 800 may be located close to one another (e.g., in the same room) orfartherfrom one another (e.g., on different floors of a building, in different buildings, in different cities, etc.). In some embodiments, a scientific instrument 810 may be connected to an Internet-of-Things (loT) stack that allows for command and control of the scientific instrument 810 through a web-based application, a virtual or augmented reality application, a mobile application, and / or a desktop application. Any of these applications may be accessed by a user operating the user local computing device 820 in communication with the scientific instrument 810 by the intervening remote computing device 840. In some embodiments, a scientific instrument 810 may be sold by the manufacturer along with one or more associated user local computing devices 820 as part of a local scientific instrument computing unit 812.

[0084] In some embodiments, different ones of the scientific instruments 810 included in a scientific instrument support system 800 may be different types of scientific instruments 810; for example, one scientific instrument 810 may be an optical emission spectrometer. In some such embodiments, the remote computing device 840 and / or the user local computing device 820 may combine data from different types of scientific instruments 810 included in a scientific instrument support system 800. 30 07 24

Claims

13 05 251. A method of obtaining a digital image of a spectrum from a time-series of images frames of the spectrum imaged onto an image sensor by an optical system of a spectrometer, the method comprising: obtaining the time-series of image frames of respective spectra recorded while the imaged spectrum moves relative to the image sensor;up-sampling each image frame to generate a respective up-sampled image frame; transforming the up-sampled image frames to align the recorded spectra; and combining the transformed image frames to generate the digital image.

2. The method of claim 1, wherein combing the transformed digital image frames comprises averaging corresponding array values of each aligned image frame to generate one array from a number of arrays equal to a number of image frames in the time-series of digital image frames.

3. The method of any preceding claim, wherein the movement of the spectrum is due to thermal expansion of at least one of the optical components and / or thermal contraction of at least one of the components.

4. The method of any preceding claim, wherein the movement is caused by a motion of an actuator coupled to one of the components, for example a vibrating motion5. The method of any preceding claim dependent on claim 3, wherein the thermal expansion and / or contraction due to temperature variation is due to operation of the spectrometer.

6. The method of any preceding claim dependent on claim 3, wherein the temperature variation is due to heating or cooling of the optical system by a temperature control system.

7. The method of any preceding claim, wherein transforming the image frames to align the recorded spectra comprises translating the image frames in a linear translation.

8. The method of any preceding claim, wherein transforming the image frames to align the recorded spectra comprises rotating the image frames.

9. The method of claims 1 to 6, wherein transforming the image frames to align the recorded spectra comprises applying a deformation field to the image frames.

10. The method of any preceding claim, wherein the spectrometer is an Inductively Coupled Plasma Optical Emission Spectrometer (ICP-OES).

11. The method of any preceding claim further comprising:recording a time-series of image frames while the imaged spectrum moves relative to the imagesensor.

12. A system comprising one or more processors and one or more memories having stored thereon computer-readable instructions configured to cause the one or more processors to perform operations comprising the steps of any of claims 1-11.

13. The system of claim 12 further comprising a spectrometer for obtaining the time series of images.

14. The system of claim 13, wherein the spectrometer comprises an optical system, wherein the optical system comprises an image sensor having an array of pixels defining a sensor resolution.

15. A computer-readable medium comprising instructions, that, when executed by one or more data processing apparatus, cause the one or more data processing apparatus to perform operations comprising the steps of any of claims 1 -11; ora computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out the method of any of claims 1-11.

Images