Spectrometer image enhancement

Abstract

Scientific instrument support systems and related methods, computing devices, and computer readable media are disclosed herein. For example, in some embodiments, a computer-implemented method of acquiring a digital image of a spectrum from a time series of image frames of the spectrum imaged onto an image sensor by an optical system of a spectrometer is provided. The method comprises: acquiring a time sequence of image frames of a corresponding spectrum recorded when an imaging spectrum moves relative to an image sensor; transforming the image frame to align the recorded spectrum; and combining the transformed image frames to generate a digital image. In some embodiments, the method may further include upsampling each image frame to generate a corresponding upsampled image frame, and transforming the upsampled image frame to align the recorded spectrum. Upsampling each image frame may include generating an array of interpolated pixels by interpolating the array of pixels, thereby calculating additional pixels between the sensor pixels.

Description

Technical Field

[0001] This disclosure relates to methods and systems for enhancing the quality of images acquired using an optical spectrometer, specifically but not exclusively for enhancing resolution. Background Technology

[0002] Scientific instruments used to acquire spectral images can include complex arrangements of movable components, sensors, input and output ports, and power and consumable components. Therefore, the acquired spectral images are sensitive to changes in the operating conditions of the scientific instrument, such as temperature variations. For example, image drift caused by different components of a scientific instrument in a non-equilibrium state can degrade image quality and analytical data in an inductively coupled plasma optical emission spectrometer (ICP-OES). Consequently, there are numerous solutions to mitigate this drift. Typically, these solutions involve one or more of the following: thermal decoupling of the optical slot from a large heat source in the scientific instrument; active thermal stabilization (through isothermal heating or cooling of the optical slot); or post-processing methods that realign drift data to generate spectral images independent of the operating conditions of the scientific instrument. Other factors degrade spectral image quality include low resolution, dark noise (noise caused by electronic patterns), and dead pixels in the spectral image, which are typically addressed by hardware improvements. Therefore, there is a widespread need to improve image quality in spectrometers in a cost-effective and sustainable manner. Summary of the Invention

[0003] This document discloses scientific instrument support systems and related methods, computing devices, and computer-readable media. For example, in some embodiments, a computer-implemented method is provided for acquiring a digital image of a spectrum from a time series of image frames of a spectrum imaged onto an image sensor by an optical system of a spectrometer. For example, in some embodiments, the spectrometer is an inductively coupled plasma optical emission spectrometer (ICP-OES). The method includes: acquiring a time series of image frames of a corresponding spectrum recorded as the imaging spectrum moves relative to the image sensor. The method further includes: upsampling each image frame to generate a corresponding upsampled image frame; transforming the upsampled image frames to align the recorded spectrum; and combining the transformed image frames to generate a digital image. Upsampling each image frame may include: generating an interpolated pixel array by interpolating a pixel array, calculating additional pixels between pixels of the sensor. By upsampling image frames using additional pixels between captured pixels, a higher resolution digital image can be obtained when the upsampled image frames are combined and the spectrum moves relative to the image sensor at a sub-pixel fraction (or a multiple of a sub-pixel fraction). Because of the spectral shift across the sensor, combining transformed image frames uses information from other image frames to fill in failed pixels on any given image frame. Therefore, the resulting digital image may have fewer or no blank areas caused by failed sensor pixels. For example, this can acquire high-quality digital images using relatively inexpensive charge injection (CID) devices and reduce the rate at which sensor chips are rendered obsolete due to failed pixels. Furthermore, in some embodiments, combining transformed imaging frames can slightly expand the detector area, allowing the capture of signals (such as peaks) that might otherwise be "out of range" from the detection area. Image drift, caused by different components of scientific instruments being in thermal nonequilibrium, is generally considered to degrade image quality and analytical data in instruments such as inductively coupled plasma optical emission spectrometers (ICP-OES) and is therefore considered undesirable and should be suppressed. In contrast, the method provided in this disclosure utilizes image drift to improve the quality of the acquired digital image, thereby relaxing installation constraints related to optical temperature control. In some embodiments, these constraints can be completely eliminated.

[0004] In some embodiments, the method may further include recording a time series of image frames as the imaging spectrum moves relative to the image sensor. In other embodiments, the time series is pre-recorded and stored in memory, and the method accesses the time series recorded in memory. For example, in some embodiments, the time series may be, or has been, recorded on an inductively coupled plasma optical emission spectrometer (ICP-OES).

[0005] In some embodiments, transforming an image frame to align with a recorded spectrum includes translating the image frame using a linear translation. In some embodiments, transforming an image frame to align with a recorded spectrum includes rotating the image frame. More generally, the transformation may include any affine or non-affine transformation suitable for aligning the image, such as a combination of rotation, translation, and / or scaling, or the application of a deformation field.

[0006] In some embodiments, image frames are aligned with predetermined image frames in a time series of image frames (e.g., the first image frame in a time series of image frames). Of course, it should be understood that there are other ways to align image frames, not by aligning frames with specific frames, but by aligning frames with each other, and any suitable method for aligning image frames can be used.

[0007] In some embodiments, combining the transformed digital image frames includes averaging the corresponding array values ​​for each aligned image frame to generate an array from a plurality of arrays equal in number to the number of image frames in the time series of the digital image frames. For example, in some embodiments, the shift in the spectrum may be due to thermal expansion and / or thermal contraction of at least one of the optical components. For example, thermal expansion and / or thermal contraction caused by temperature changes may be due to normal operation of the spectrometer acquiring digital images. In this way, the advantages discussed above can be achieved during normal operation of the spectrometer (i.e., the method does not require additional or different steps when capturing image frames).

[0008] In some embodiments, the shift in the spectrum within an image frame is actively induced. For example, thermal expansion and / or contraction may be caused by heating or cooling of the optical system by a temperature control system. This may also occur in addition to any changes caused by normal operation. In some embodiments, at least some of the shift is caused by the movement of at least one actuator of one of the components coupled to the optical system, wherein the actuator may be a motor or a piezoelectric actuator. This may introduce additional randomness in the drift, which can further increase the effective resolution of the digital image due to better sampling of the positioning of image features relative to sensor pixels.

[0009] In some embodiments, a system is provided that includes one or more processors and one or more memories storing computer-readable instructions thereon, the computer-readable instructions being configured to cause the one or more processors to perform one or more of the operations described above. In some embodiments, the system may further include a spectrometer, such as an optical emission spectrometer (e.g., ICP-OES), for acquiring time series of images.

[0010] In some embodiments, a computer-readable medium including instructions that, when executed by one or more data processing devices, cause the one or more data processing devices to perform one or more of the operations described above.

[0011] In some embodiments, a computer program including instructions is provided that, when executed by a computer, causes the computer to perform one or more of the steps described above.

[0012] The scientific instrument support embodiments disclosed herein can achieve improved performance compared to conventional methods. For example, the need to mitigate drift is avoided, and drift is actually utilized to improve the resolution of digital images. Thus, the embodiments disclosed herein provide improvements to scientific instrument technology (e.g., improvements to the computer technology supporting such scientific instruments, and others).

[0013] The embodiments disclosed herein can achieve data of better quality than conventional methods. For example, conventional methods are limited by the resolution of image sensors, while the disclosed embodiments can achieve higher resolutions than image sensors can. Furthermore, these methods address many technical problems and limitations, including dead pixels and image drift, for example, problems caused by different components of scientific instruments in a state of thermal nonequilibrium.

[0014] The various embodiments disclosed herein improve upon conventional methods to achieve the technical advantage of increased resolution through upsampling and combining image frames. Such technical advantages cannot be achieved through conventional and traditional methods, and all users of systems including such embodiments can benefit from these advantages (e.g., by assisting users in performing technical tasks, such as spectroscopy, through guided human-computer interaction processes). Therefore, the technical features of the embodiments disclosed herein are clearly unconventional in the field of analytical chemistry, as are the combinations of features of the embodiments disclosed herein. The computational and user interface features disclosed herein involve not only the collection and comparison of information but also the application of new analytical and technical means to alter the processing of image frames acquired from spectrometers, thereby extracting higher quality information from the image frames. Therefore, this disclosure introduces functionality that is impossible for conventional computing devices and humans to perform.

[0015] Therefore, embodiments of this disclosure can be used for any of a variety of technical purposes, such as controlling a specific technical system or process; determining how to control a machine based on measurement results; digital image enhancement or analysis; reducing the amount of sensor data to be processed; or providing faster sensor data processing. Specifically, this disclosure provides technical solutions to technical problems, including but not limited to how to obtain higher quality data from a spectrometer without replacing the equipment.

[0016] Therefore, the embodiments disclosed herein provide improvements to analytical chemistry techniques (e.g., improvements in computer technology supporting analytical chemistry, and others). Attached Figure Description

[0017] The various embodiments will be readily understood by reading the following detailed description in conjunction with the accompanying drawings. For ease of description, the same reference numerals denote the same structural elements. The embodiments are shown in the accompanying drawings by way of example rather than by way of limitation.

[0018] Figure 1 is a diagram of the optical system of an exemplary inductively coupled plasma optical emission spectrometer 100 for acquiring optical emission spectra according to some embodiments.

[0019] Figure 2 is an exemplary optical emission spectrum of the intensity distribution captured by an image sensor.

[0020] Figure 3 is a block diagram of an exemplary scientific instrument support module 300 for performing support operations according to various embodiments.

[0021] Figure 4 is a flowchart of example method 400, which acquires a digital image from a time series of image frames of a spectral image formed by the optical system of a spectrometer.

[0022] Figure 5A shows an example of upsampling the image matrices 504 A, B and n corresponding to image frames A, B and n to generate upsampled image matrices 508 A, B and n.

[0023] Figure 5B shows an example of aligning upsampled image matrices 508A and n with image matrix 508B.

[0024] Figure 6 is an example of a graphical user interface 600 according to various embodiments, which can be used to perform some or all of the support methods disclosed herein.

[0025] Figure 7 is a block diagram of an exemplary computing device 700 that can perform some or all of the scientific instrument support methods disclosed herein, according to various embodiments.

[0026] Figure 8 is a block diagram of an exemplary scientific instrument support system 800 according to various embodiments, wherein some or all of the scientific instrument support methods disclosed herein may be performed. Detailed Implementation

[0027] In the following detailed description, reference is made to the accompanying drawings, which form a part of this document, wherein similar reference numerals always designate similar parts, and embodiments that can be practiced are illustrated in the drawings by way of illustration. It should be understood that other embodiments may be utilized and structural or logical changes may be made without departing from the scope of this disclosure. Therefore, the following detailed description should not be regarded as limiting.

[0028] Various operations can be described as multiple discrete actions or operations in a manner most conducive to understanding the subject matter disclosed herein. However, the order of description should not be construed as implying that these operations necessarily depend on a specific order. In particular, these operations may not be performed in the order presented. The described operations may be performed in a different order than the described embodiments. Various other operations may be performed, and / or the described operations may be omitted in additional embodiments.

[0029] For the purposes of this disclosure, the phrases “A and / or B” and “A or B” mean (A), (B), or (A and B). For the purposes of this disclosure, the phrases “A, B and / or C” and “A, B or C” mean (A), (B), (C), (A and B), (A and C), (B and C), or (A, B and C). Although some elements may be mentioned in the singular (e.g., “a processing device”), any suitable element may be represented by multiple instances of that element, and vice versa. For example, a set of operations described as being performed by a processing device can be implemented by different operations among those performed by different processing devices.

[0030] The phrases “embodiment,” “various embodiments,” and “some embodiments” as used in this specification may each refer to one or more embodiments of the same or different embodiments. Furthermore, the terms “comprising,” “including,” “having,” etc., as used with respect to embodiments of this disclosure are synonymous. When used to describe a range of dimensions, the phrase “between X and Y” indicates a range including both X and Y. As used herein, “apparatus” may refer to any single device, a collection of devices, a component of a device, or a collection of components of a device. The drawings are not necessarily drawn to scale.

[0031] Figure 1 is an illustration of an example of a scientific instrument 100 for acquiring spectral images according to various embodiments. Figure 1The exemplary scientific instrument shown is an optical emission spectrometer, such as an inductively coupled plasma optical emission spectrometer (ICP-OES). ICP-OES 100 can be a component of a science support system 800, as described with reference to Figure 8 below. Scientific instruments, such as ICP-OES 100, can include a complex arrangement of movable components, sensors, input and output ports, power and consumable components. Therefore, the acquired spectral images are sensitive to changes in the operating conditions of the scientific instrument, such as temperature variations. Emission spectrometers operate by exciting atoms and ions to emit electromagnetic (EM) radiation at wavelengths characteristic of specific elements. Electrons emit a spectrum of EM radiation when transitioning from a high-energy state to a low-energy state. ICP-OES 100 can include a plasma chamber 102. Plasma chamber 102 can be connected to a radio frequency (RF) source and a gas source (e.g., argon). Argon can be ionized within an oscillating RF field generated by the RF source to form and sustain a plasma within plasma chamber 102. The ICP-OES100 may employ an echelle grating-based optical design to generate a two-dimensional optical emission spectrum.

[0032] Specifically, the ICP-OES may include a etched diffraction grating 114, a prism 112, and multiple focusing mirrors 104, 106, 110, and 116. These components together constitute an optical system 124. Light from the plasma chamber 102 enters the ICP-OES 100 and is selectively focused using multiple focusing mirrors (e.g., the first mirror 104 and the second mirror 106). The converged light passes through an entrance slit 108 and enters the prism 112 using mirror 110. The prism 112 can separate the light by wavelength. The etched diffraction grating 114 can diffract the separated light from the prism 112 into multiple diffraction orders, producing a high-resolution 2D spectrum known as an etched grating pattern or etched spectrum. After passing through these optical elements, mirror 116 can collect the spectrum and focus it onto an image sensor (e.g., a camera 118) to generate an image frame. Figure 2 depicts an exemplary image frame 200 according to various embodiments. Multiple peaks appear as white dots of varying intensities.

[0033] Image frames (such as image frame 200 shown in Figure 2) are sensitive to changes in the operating conditions of scientific instruments, such as temperature changes. Image drift caused by different components of a scientific instrument in a non-equilibrium state can degrade image quality. For example, temperature changes may cause spectral peaks in the image frame to drift.

[0034] Optical system 124 is encapsulated in a housing. One or more heating pads 122 may be located on the outer surface of the housing to enable temperature control of optical system 124. Temperature sensor 120 may indicate, for example, the ambient temperature of ICP-OES 100. In one embodiment, temperature sensor 120 may indicate the temperature of ICP-OES 100 or the optical system 124 of ICP-OES 100 including optical components. In some embodiments, actuator 128 may be coupled to one or more components of the optical system. For example, in some embodiments, actuator 128 may be coupled to one or more focusing mirrors of a plurality of focusing mirrors. For example, actuator 128 may be coupled to mirror 116. The actuator may be any suitable component for inducing motion in the optical system. For example, the actuator may be a motor or a piezoelectric actuator.

[0035] The number, identification, and location of the components described above with reference to Figure 1 are illustrative, and it should be understood that other configurations and / or optical components may also be used. Although the embodiments are described with reference to an inductively coupled plasma optical emission spectrometer, the same applies to other types of spectral images acquired using other types of spectrometers. For example, the methods described herein can be used in similar spectroscopic techniques such as inductively coupled plasma mass spectrometry (ICP-MS) and atomic absorption spectrometry (AAS), and can indeed be applied to any method of acquiring spectral images, where the spectral images depend on the operating conditions of the spectrometer. Various embodiments include the use of spectroscopic techniques such as Rayleigh spectroscopy, Raman spectroscopy, or mass spectrometry.

[0036] Figure 3 is a block diagram of an exemplary scientific instrument support module 300 for performing support operations according to various embodiments.

[0037] The scientific instrument support module 300 can be implemented via circuitry (e.g., including electrical and / or optical components) (such as a programmable computing device). The logical components of the scientific instrument support module 300 can be included in a single computing device, or, depending on the situation, distributed across multiple computing devices communicating with each other. Examples of computing devices that can implement the scientific instrument support module 300 individually or in combination are discussed herein with reference to the computing device 700 of Figure 7, and examples of systems of interconnected computing devices are discussed herein with reference to the scientific instrument support system 800 of Figure 8, wherein the scientific instrument support module 300 can be implemented on one or more computing devices.

[0038] Scientific instrument support module 300 may include data acquisition logic unit 302, transformation logic unit 304, and merging logic unit 306. As used herein, the term "logic unit" may include means for performing a set of operations associated with the logic unit. For example, any of the logic elements included in support module 300 may be implemented by one or more computing devices programmed with instructions to cause one or more processing devices of the computing devices to perform an associated set of operations. In a particular embodiment, a logic element may include one or more non-transitory computer-readable media having instructions that, when executed by one or more processing devices of the one or more computing devices, cause the one or more computing devices to perform an associated set of operations. As used herein, the term "module" may refer to a collection of one or more logic elements that collectively perform functions associated with the module. Different logic elements in a module may take the same form or different forms. For example, some logic elements in a module may be implemented by a programmed general-purpose processing device, while other logic elements in the module may be implemented by an application-specific integrated circuit (ASIC). In another example, different logic elements in a module may be associated with different sets of instructions executed by one or more processing devices. A module may not include all the logical elements depicted in the associated figures; for example, when the module performs a subset of the operations discussed herein with reference to the module, the module may include a subset of the logical elements depicted in the associated figures.

[0039] Data acquisition logic unit 302 can be configured to acquire time series of image frames of the spectrum imaged onto the image sensor via the spectrometer's optical system as the imaging spectrum shifts relative to the image sensor. Spectral shift may be caused by normal operation of the spectrometer, forced movement of the optical system, or a combination of both. In some embodiments, data acquisition logic unit 302 can be configured to retrieve time series of image frames from a database of image frames. In some embodiments, data acquisition logic unit 302 can be configured to record the time series of image frames using an ICP-OES 100.

[0040] Transform logic unit 304 can be configured to align the time series of image frames by transforming the image frames. For example, in some embodiments, transform logic unit 304 can align the image frames by determining a base image frame, representing the image frame as a pixel or sub-pixel matrix, obtaining the offset between each image frame and the base image frame using phase correlation, and performing a transform based on the offset to eliminate the offset. This is described in more detail below with reference to FIG4.

[0041] The merging logic unit 306 can be configured to merge transformed image frames to generate a digital image. For example, in some embodiments, merging transformed image frames to generate a digital image may include averaging the intensity between aligned pixels or subpixels to generate the digital image. Logic blocks 302 to 304 of the support module 300 can perform any of the operations described below with reference to FIG4.

[0042] Figure 4 is a flowchart of an example of method 400, which acquires a digital image from a time series of image frames of a spectral image. According to various examples, the spectral image may be formed by the optical system of a spectrometer, wherein the optical system includes an image sensor with a pixel array having a defined sensor resolution. Method 400 uses a time series of first-resolution image frames to generate a merged image of a higher second resolution by aligning the image frames in the time series, for example, aligning each image frame to a single image frame in time series. In some embodiments, the method includes estimating the per-pixel contribution of each image frame to the merged image, while in other embodiments, the aligned image frames are combined with equal weights (e.g., averaged). In this disclosure, the term "sensor resolution" refers to the number of pixels in an image frame of a spectral image acquired from an image sensor. For example, sensor resolution can be described using pixel dimensions, such as 1920 x 1080 pixels, describing the number of horizontal and vertical pixels in an image frame of a spectral image. When displayed electronically, sensor resolution can be considered as pixel density because different resolutions can be displayed on the same area of ​​the user interface. For example, when viewed on a screen of uniform size, the higher the pixel resolution (the more pixels), the higher the pixel density displayed on the screen, and therefore the more image details are displayed on the electronic screen.

[0043] Method 400 may use image frames 200 acquired via an optical system, including, for example, an inductively coupled plasma optical emission spectrometer (ICP-OES) 100 described above with reference to FIG1. ​​The steps of method 400 may be performed by the scientific instrument support module 300 described in FIG3.

[0044] Although the operation of method 400 can be described with reference to specific embodiments disclosed herein (e.g., the graphical user interface (GUI) 600 discussed herein with reference to FIG. 6, the computing device 700 discussed herein with reference to FIG. 7, and / or the scientific instrument support system 800 discussed herein with reference to FIG. 8), method 400 can be used in any suitable setting to perform any suitable support operation. The operations are each illustrated once in FIG. 4 and in a specific order, but the operations can be reordered and / or repeated as needed and as appropriate (e.g., operations can be performed in parallel where appropriate).

[0045] Method 400 includes first, second, third, and fourth operations at 402, 404, 406, and 408, respectively.

[0046] At 402, a first operation can be performed. The first operation may include: acquiring a time series of image frames of a corresponding spectrum imaged onto an image sensor by the optical system of the spectrometer, the image frames being recorded as the imaging spectrum moves relative to the image sensor. Step 402 may be controlled by, for example, data acquisition logic unit 302. For example, in some embodiments, the spectrometer may be the ICP-OES 100 described above with reference to FIG. 1, the image sensor may be camera 118, and the image frame may be image frame 200 as shown in FIG. 2. The first operation 402 may include: acquiring the image frame by requesting or receiving the image frame from any suitable device capable of storing the image frame. For example, data acquisition logic unit 302 may request or receive the image frame stored in the memory of a device communicating with the ICP-OES 100 or a computing device 700 having a memory for storing multiple image frames.

[0047] For example, in some embodiments, step 402 may include: acquiring a time series of an image frame during routine operation of the spectrometer (also described herein as normal operation of the spectrometer) to acquire an image frame, as the imaging spectrum shifts relative to the image sensor due to unexposed thermal drift. Data acquisition logic unit 302 may be configured to acquire and store image frames. An image frame may include a set of pixels having pixel values ​​that reflect the captured intensity distribution. Pixel values ​​may be grayscale scalar values ​​or vector values ​​representing color information; for example, the image frame may be represented as a matrix of pixel values.

[0048] At step 404, a second operation can be performed. The second operation at 404 can be performed, for example, by the transformation logic unit 304. The second operation may include: after the first operation and before the third operation, representing each image frame as a pixel matrix and upsampling the matrix to generate a sub-pixel matrix, where each point in the new matrix corresponds to a multiple of a pixel fraction, and the spacing between sub-pixels is a fraction of the spacing between pixels. As shown in Figure 5A, Figure 5A shows image frames A, B, and n containing captured images of peak 502. Peak 502 drifts with the spectrum from image frame 504A to 504n. On the left side of Figure 5A, image frames A, B, and n are represented by 3x3 pixel matrices 504A, 504B, and 504n, respectively. For illustrative purposes, 3x3 pixel matrices 504A to 504n represent pixels captured by camera 118; however, it should be understood that the actual matrices captured by camera 118 are much larger, and the described method is applicable to any captured pixel matrix. Furthermore, on the right side of Figure 5A, image frames A, B, and n are represented by 5x5 subpixel upsampled matrices, with one subpixel padded between each pixel. For example, in some embodiments, subpixel matrices can be created by adding subpixels between each pixel to interpolate the captured pixel matrices 504A to 504n to obtain new subpixel matrices 508A to 508n. This process is referred to herein as upsampling the captured pixel matrix. It should be understood that the obtained pixel matrices 504A, 504B, and 504n can be upsampled by any suitable method, such as linear interpolation as described above or, for example, kernel regression. It is worth noting that Figure 5A shows that the peak location is captured by only one subpixel in each upsampling matrix 508A, 508B, and 508C. For example, each peak is captured by a single value in a subpixel matrix that has an intensity value different from its adjacent intensity values. The same peak is captured by two pixels in matrix 504B; for example, peaks of the same size will be represented by two values ​​in pixel matrix 504B. Although the two matrices show the same peak, the peak in the upsampling matrix is ​​smaller and appears (e.g.) white, while the peak in matrix 504B is larger and appears (e.g.) gray (a single white peak of a single pixel appears scattered at lower resolution). Of course, in practice, a peak will be represented by multiple pixels, and different regions of the peak will be displayed with different intensities, but the peak of a single pixel has been used as an illustrative example of improving resolution through upsampling.

[0049] At 406, a third operation can be performed. This third operation may include transforming the image frame to align the recorded spectrum. This second operation may be performed, for example, by the transformation logic unit 304.

[0050] A third operation is performed at position 406 on the upsampled matrix. Specifically, the upsampled image frames A, B, and n are aligned, and the subpixel values ​​are shifted, for example, translated in subpixel units to align the images. This is illustrated in Figure 5B. Figure 5B shows an example of aligning the upsampled matrices 508A, B, and n by applying a linear transformation (specifically, a translation) to image frames A and n to align them with image frame B. The dashed arrows in Figure 5B illustrate the corresponding translation of each of image frames A and n.

[0051] In some embodiments, aligning the time series of image frames may include representing each image frame as a matrix, where each value in the matrix corresponds to a pixel value at coordinates within the image frame. For example, an image frame with a pixel size of 10x10 is represented as a 10x10 matrix, where each value corresponds to the pixel intensity at each point in the image frame. In such embodiments, pixel values ​​are offset in pixels to align the image.

[0052] In some embodiments, image frames may be aligned with a base image in a time series of image frames. For example, an image frame may be aligned with the first image in the time series. In some embodiments, the transformation logic unit 304 may align each image frame with the base image by using phase correlation to obtain an offset between each image frame and the base image frame, and apply a transformation based on the offset to eliminate the offset. Of course, it should be understood that any image frame in the time series can be used as the base image.

[0053] The transformation applied to the image frame can be any suitable transformation performed to align the image frames. For example, in some embodiments, aligning the image frames in a time series includes applying a linear translation to the image frame. For example, in some embodiments, aligning the image frames in a time series includes applying a rotation to the image frame. For example, in some embodiments, aligning the image frames in a time series includes applying a deformation to the image frame. Of course, it should be understood that any combination of the above transformations can be used, and in fact, any suitable transformation operation can be used to align the image frames.

[0054] At 408, a fourth operation can be performed. This fourth operation may include combining the transformed image frames to generate a digital image. For example, this fourth operation may be performed by the merging logic unit 306. As discussed above with reference to FIG3, combining the transformed image frames may include averaging the intensity across aligned subpixels to generate a digital image.

[0055] Step 408 includes merging image frames to generate a digital image by averaging the subpixel values ​​across aligned image frames. In such embodiments, when upsampled image frames are combined, a higher resolution digital image can be obtained, and the spectrum shifts relative to camera 118 by multiple subpixels (e.g., 3 subpixels or 1.5 pixels). It should be understood, of course, that other examples of upsampling can also be used, where one pixel is upsampled to any suitable number of subpixels. In other words, the number of pixels can be increased by any suitable amount (200%, 205%, 325%, etc.). In some examples, one pixel can be upsampled to 10 subpixels, in which case a 1.5-pixel shift in the x-direction would be equivalent to 15 subpixels in the x-direction. In some examples, one pixel can be upsampled to 20 subpixels, in which case a 1.5-pixel shift would be equivalent to 30 subpixels.

[0056] The methods described herein utilize image drift caused by movement of components in the optical system to enhance digital image quality. In some embodiments, in addition to the drift generated by normal operation of the ICP-OES 100, it may be necessary to intentionally induce drift. To enhance the digital image, intentionally induced drift can be induced by applying thermal changes to the optical system or by mechanically moving components within the optical system, or both.

[0057] For example, in some embodiments, the first operation at 402 may include: acquiring a time series of image frames while the imaging spectrum shifts relative to the image sensor due to applied thermal drift. The applied thermal drift may be superimposed on drift generated during normal operation of the ICP-OES 100 and may be used to further improve image quality. For example, in some embodiments, the data acquisition logic unit 302 may be configured to control the ICP-OES 100 to change the operating conditions of the ICP-OES 100 to apply thermal drift by controlling the heating pad 122 while acquiring a time series of images to induce further drift in the images. For example, the data acquisition logic unit 302 may be configured to control the heating pad to cycle the temperature between 25 degrees Celsius and 42 degrees Celsius while acquiring image frames.

[0058] In some embodiments, the first operation at 402 may include: acquiring a time series of image frames of the spectrum imaged onto the image sensor by the spectrometer's optical system as an applied drift, caused by any other applied movement in the optical system, results in a shift of the imaging spectrum relative to the image sensor. For example, in some embodiments, the data acquisition logic unit 302 may be configured to control the operation of the ICP-OES 100 on actuators 128 coupled to components of the optical system (e.g., mirror 116), thereby mechanically vibrating mirror 116 or other components back and forth with repetitive or pseudo-random motion.

[0059] The inventors discovered that by moving the analyzed image due to unmitigated thermal drift, actively induced thermal drift, other periodic or random movements in the optical system (such as movements caused by motors or piezoelectric actuators as described above), or combinations thereof, and by acquiring time series of image frames, high-resolution digital images can be generated from multiple low-resolution image frames.

[0060] Figure 6 depicts an exemplary GUI 600 according to various embodiments, which can be used to perform some or all of the support methods disclosed herein. As described above, the GUI 600 can be located on the display device (e.g., the display device 710 discussed herein with reference to Figure 7) of a computing device (e.g., the computing device 700 discussed herein with reference to Figure 7) of a scientific instrument support system (e.g., the scientific instrument support system 800 discussed herein with reference to Figure 8), and the user can interact with the GUI 600 using any suitable input device (e.g., any input device included in the other I / O devices 712 discussed herein with reference to Figure 7) and input technologies (e.g., cursor movement, motion capture, facial recognition, gesture detection, speech recognition, button driving, etc.).

[0061] GUI 600 may include a data display area 602, a data analysis area 604, a scientific instrument control area 606, and a settings area 608. The specific number and arrangement of areas depicted in Figure 6 are for reference only, and any number and arrangement of areas (including any desired features) may be included in GUI 600.

[0062] Data display area 602 can display data generated by a scientific instrument (e.g., scientific instrument 610 discussed herein with reference to FIG. 6). For example, data display area 602 can display information related to image frames, such as the frame number or the operating conditions of the optical system acquiring these frames.

[0063] Data analysis area 604 can display the results of data analysis (e.g., the results of analysis of the data shown in data display area 602 and / or other data). For example, data analysis area 604 can display information related to drift between image frames or (e.g., resolution information of these frames or the acquired digital image). In some embodiments, data display area 602 and data analysis area 604 can be combined in GUI 600 (e.g., to include data output from scientific instruments and some analysis of the data in a public graphics or area).

[0064] The scientific instrument control area 606 may include options that allow a user to control a scientific instrument (e.g., scientific instrument 810 discussed herein with reference to FIG8). For example, the scientific instrument control area 606 may include options for controlling the temperature of an optical system.

[0065] The settings area 608 may include features and functions that allow users to control the GUI 600 (and / or other GUIs), and / or options for performing common computational operations on the data display area 602 and the data analysis area 604 (e.g., saving data to a storage device (such as storage device 704 discussed herein with reference to FIG7), sending data to other users, tagging data, etc.).

[0066] Computing infrastructure

[0067] As described above, the scientific instrument support module 200 can be implemented by one or more computing devices. Figure 7 is a block diagram of a computing device 700 that can perform some or all of the scientific instrument support methods disclosed herein, according to various embodiments. In some embodiments, the scientific instrument support module 200 can be implemented by a single computing device 700 or by multiple computing devices 700. Furthermore, as discussed below, the computing device 700 (or multiple computing devices 700) implementing the scientific instrument support module 1000 can be part of one or more of the scientific instrument 810 of Figure 8, the user local computing device 820, the service local computing device 810, or the remote computing device 840.

[0068] The computing device 700 of Figure 7 is shown as having multiple components, but any or more of these components may be omitted or repeated based on application and setup requirements. In some embodiments, some or all of the components included in the computing device 700 may be attached to one or more motherboards and encapsulated in a housing (e.g., including plastic, metal, and / or other materials). In some embodiments, some of these components may be fabricated onto a single system-on-a-chip (SoC) (e.g., the SoC may include one or more processing devices 702 and one or more storage devices 704). In some embodiments, one or more of these components may be located remotely from the computing device 700. Furthermore, in various embodiments, the computing device 700 may not include one or more of the components shown in Figure 7, but may include interface circuitry (not shown) for coupling to one or more components using any suitable interface (e.g., a Universal Serial Bus (USB) interface, a High Definition Multimedia Interface (HDMI) interface, a Controller Area Network (CAN) interface, a Serial Peripheral Interface (SPI) interface, an Ethernet interface, a wireless interface, or any other suitable interface). For example, computing device 700 may not include display device 710, but may include display device interface circuitry (e.g., connectors and driver circuitry) to which display device 710 may be coupled.

[0069] Computing device 700 may include processing device 702 (e.g., one or more processing devices). Herein, the term "processing device" can refer to any device or part of a device that processes electronic data from registers and / or memory to convert said electronic data into other electronic data that can be stored in registers and / or memory. Processing device 702 may include one or more digital signal processors (DSPs), application-specific integrated circuits (ASICs), central processing units (CPUs), graphics processing units (GPUs), cryptographic processors (dedicated processors that execute cryptographic algorithms within hardware), server processors, or any other suitable processing device.

[0070] Computing device 700 may include storage device 704 (e.g., one or more storage devices). Storage device 704 may include one or more memory devices, such as random access memory (RAM) (e.g., static RAM (SRAM) devices, magnetic RAM (MRAM) devices, dynamic RAM (DRAM) devices, resistive RAM (RRAM) devices, or conductive bridged RAM (CBRAM) devices), hard disk drive-based memory devices, solid-state memory devices, network drives, cloud drives, or any combination of memory devices. In some embodiments, storage device 704 may include memory sharing a die with processing device 702. In such embodiments, the memory may be used as cache memory and may include, for example, embedded dynamic random access memory (eDRAM) or spin-transfer torque magnetic random access memory (STT-MRAM). In some embodiments, storage device 704 may include a non-transitory computer-readable medium having instructions thereon that, when executed by one or more processing devices (e.g., processing device 702), cause computing device 700 to perform any suitable method or portion thereof of the methods disclosed herein.

[0071] Computing device 700 may include interface device 706 (e.g., one or more interface devices 706). Interface device 706 may include one or more communication chips, connectors, and / or other hardware and software to manage communication between computing device 700 and other computing devices. For example, interface device 706 may include circuitry for managing wireless communication used to transfer data with computing device 700. The term "wireless" and its derivatives can be used to describe circuits, devices, systems, methods, techniques, communication channels, etc., that can transmit data through a non-solid medium using modulated electromagnetic radiation. This term does not imply that the associated device does not contain any wires, although in some embodiments it may not contain any wires. The circuitry included in interface device 706 for managing wireless communications may implement any of a number of wireless standards or protocols, including but not limited to Institute of Electrical and Electronics Engineers (IEEE) standards, including Wi-Fi (IEEE 802.11 series), IEEE 802.16 standards (e.g., IEEE 802.16-2005 amendments), Long Term Evolution (LTE) projects, and any amendments, updates, and / or revisions (e.g., Advanced LTE projects, Ultra Mobile Broadband (UMB) projects (also known as “3GPP2”), etc.). In some embodiments, the circuitry included in interface device 706 for managing wireless communications may operate according to Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Universal Mobile Telecommunications System (UMTS), High Speed ​​Packet Access (HSPA), Evolved HSPA (E-HSPA), or LTE networks. In some embodiments, the circuitry included in interface device 706 for managing wireless communications may operate according to Enhanced Data GSM Evolution (EDGE), GSM EDGE Radio Access Network (GERAN), Universal Terrestrial Radio Access Network (UTRAN), or Evolved UTRAN (E-UTRAN). In some embodiments, the circuitry included in interface device 706 for managing wireless communications may operate based on Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Digital Enhanced Wireless Communication (DECT), Evolved Data Optimization (EV-DO) and its derivatives, as well as any other wireless protocol designated 3G, 4G, 5G, and higher. In some embodiments, interface device 706 may include one or more antennas (e.g., one or more antenna arrays) to receive and / or transmit wireless communications.

[0072] In some embodiments, interface device 706 may include circuitry for managing wired communications, such as electrical communication protocols, optical communication protocols, or any other suitable communication protocols. For example, interface device 706 may include circuitry supporting communications based on Ethernet technology. In some embodiments, interface device 706 may support both wireless and wired communications, and / or may support multiple wired communication protocols and / or multiple wireless communication protocols. For example, a first set of circuitry for interface device 706 may be dedicated to short-range wireless communications such as Wi-Fi or Bluetooth, while a second set of circuitry for interface device 706 may be dedicated to long-range wireless communications such as Global Positioning System (GPS), EDGE, GPRS, CDMA, WiMAX, LTE, EV-DO, etc. In some embodiments, a first set of circuitry for interface device 706 may be dedicated to wireless communications, while a second set of circuitry for interface device 706 may be dedicated to wired communications.

[0073] The computing device 700 may include a battery / power circuit 708. The battery / power circuit 708 may include one or more energy storage devices (e.g., batteries or capacitors) and / or circuitry for coupling components of the computing device 700 to a power source (e.g., AC line power) that is separate from the computing device 700.

[0074] Computing device 700 may include display device 710 (e.g., multiple display devices). Display device 710 may include any visual indicator, such as a head-up display, computer monitor, projector, touch screen display, liquid crystal display (LCD), light-emitting diode display, or flat panel display.

[0075] The computing device 700 may include other input / output (I / O) devices 712. Other I / O devices 712 may include, for example, one or more audio output devices (e.g., speakers, headphones, earphones, alarm clocks, etc.), one or more audio input devices (e.g., microphones or microphone arrays), positioning devices (e.g., GPS devices that communicate with satellite-based systems to receive the location of the computing device 700, as known in the art), audio codecs, video codecs, printers, sensors (e.g., thermocouples or other temperature sensors, humidity sensors, pressure sensors, vibration sensors, accelerometers, gyroscopes, etc.), image capture devices (such as cameras), keyboards, cursor control devices (such as mice, styluses, trackballs, or touchpads), barcode readers, quick-response (QR) code readers, or radio frequency identification (RFID) readers.

[0076] The computing device 700 can have any form factor suitable for its application and scenario, such as handheld or mobile computing devices (e.g., mobile phones, smartphones, mobile internet devices, tablets, laptops, netbooks, ultrabooks, personal digital assistants (PDAs), ultra-mobile personal computers, etc.), desktop computing devices, server computing devices, or other networked computing components.

[0077] One or more computing devices implementing any of the scientific instrument support modules or methods disclosed herein may be part of a scientific instrument support system. Figure 8 is a block diagram of an exemplary scientific instrument support system 800 according to various embodiments, wherein some or all of the scientific instrument support methods disclosed herein may be performed. The scientific instrument support modules and methods disclosed herein (e.g., scientific instrument support module 100 of Figure 2 and methods 5000, 6000, 7000 of Figures 5, 6, and 7) may be implemented by one or more of the scientific instrument 810, user local computing device 820, service local computing device 830, or remote computing device 840 of the scientific instrument support system 800.

[0078] Any of the scientific instrument 810, the user local computing device 820, the service local computing device 830, or the remote computing device 840 may include any embodiment of the computing device 700 discussed herein with reference to FIG. 7, and any of the scientific instrument 810, the user local computing device 820, the service local computing device 830, or the remote computing device 840 may take the form of any suitable embodiment of the computing device 700 discussed herein with reference to FIG. 7.

[0079] Scientific instrument 810, user local computing device 820, service local computing device 830, or remote computing device 840 may each include a processing device 802, a storage device 804, and an interface device 806. The processing device 802 may take any suitable form, including any of the processing devices 702 discussed herein with reference to FIG. 7, and the processing devices 802 included in different devices of scientific instrument 810, user local computing device 820, service local computing device 830, or remote computing device 840 may take the same or different forms. The storage device 804 may take any suitable form, including any of the storage devices 704 discussed herein with reference to FIG. 7, and the storage devices 804 included in different devices of scientific instrument 810, user local computing device 820, service local computing device 830, or remote computing device 840 may take the same or different forms. Interface device 806 may take any suitable form, including any of the interface devices 706 discussed herein with reference to FIG7, and the interface devices 806 included in different devices such as scientific instrument 810, user local computing device 820, service local computing device 830, or remote computing device 840 may take the same or different forms.

[0080] Scientific instrument 810, user local computing device 820, service local computing device 830, and remote computing device 840 can communicate with other elements of scientific instrument support system 800 via communication path 808. As shown, communication path 808 can communicatively couple interface devices 806 of different elements in scientific instrument support system 800 and can be a wired or wireless communication path (e.g., any communication technology discussed herein according to the interface device 706 of computing device 700 shown with reference to FIG. 7). The particular scientific instrument support system 800 depicted in FIG. 8 includes communication paths between each pair of devices in scientific instrument 810, user local computing device 820, service local computing device 830, and remote computing device 840; however, this “fully connected” implementation is merely illustrative, and various communication paths 808 may not exist in various embodiments. For example, in some embodiments, the serving local computing device 830 may not have a direct communication path 808 between its interface device 806 and the interface device 806 of the scientific instrument 810, but may instead communicate with the scientific instrument 810 via a communication path 808 between the serving local computing device 830 and the user local computing device 820 and a communication path 808 between the user local computing device 820 and the scientific instrument 810.

[0081] Scientific instrument 810 may include any suitable scientific instrument, such as an optical emission spectrometer, for example, Figure 1 The inductively coupled plasma optical emission spectrometer 1000 is shown.

[0082] User-local computing device 820 may be a user-local computing device of scientific instrument 810 (e.g., any embodiment of the computing device 700 discussed herein). In some embodiments, user-local computing device 820 may also be located locally to scientific instrument 810, but this is not always the case; for example, user-local computing device 820 located in a user's home or office may be remote from scientific instrument 810 but communicate with said scientific instrument, allowing the user to use user-local computing device 820 to control and / or access data from scientific instrument 810. In some embodiments, user-local computing device 820 may be a laptop, smartphone, or tablet device. In some embodiments, user-local computing device 820 may be a portable computing device.

[0083] The serving local computing device 830 can be a computing device that serves the physical locality of the scientific instrument 810 (e.g., any embodiment of the computing device 700 discussed herein). For example, the serving local computing device 830 can be a local device of the manufacturer of the scientific instrument 810 or a third-party service company. In some embodiments, the serving local computing device 830 can communicate with the scientific instrument 810, the user local computing device 820, and / or the remote computing device 840 (e.g., via a direct communication path 808 or via multiple “indirect” communication paths 808, as discussed above) to receive data regarding the operation of the scientific instrument 810, the user local computing device 820, and / or the remote computing device 840 (e.g., self-test results of the scientific instrument 810, calibration coefficients used by the scientific instrument 810, measurement results of sensors associated with the scientific instrument 810, etc.). In some embodiments, the service local computing device 830 may communicate with scientific instrument 810, user local computing device 820, and / or remote computing device 840 (e.g., via direct communication path 808 or via multiple “indirect” communication paths 808, as discussed above) to transfer data to scientific instrument 810, user local computing device 820, and / or remote computing device 840 (e.g., to update programming instructions (such as firmware) in scientific instrument 810, initiate the execution of test or calibration sequences in scientific instrument 810, update programming instructions (such as software) in user local computing device 820 or remote computing device 840, etc.). A user of scientific instrument 810 may use scientific instrument 810 or user local computing device 820 to communicate with service local computing device 830 to report problems with scientific instrument 810 or user local computing device 820, request on-site technician visits to improve the operation of scientific instrument 810, order consumables or replacement parts associated with scientific instrument 810, or for other purposes.

[0084] Remote computing device 840 may be a computing device located remotely from scientific instrument 810 and / or user local computing device 820 (e.g., any embodiment of the computing device 700 discussed herein). In some embodiments, remote computing device 840 may be included in a data center or other large-scale server environment. In some embodiments, remote computing device 840 may include network-attached storage (e.g., as part of storage device 804). Remote computing device 840 may store data generated by scientific instrument 810, perform analysis on data generated by scientific instrument 810 (e.g., according to programming instructions), facilitate communication between user local computing device 820 and scientific instrument 810, and / or facilitate communication between service local computing device 830 and scientific instrument 810.

[0085] In some embodiments, one or more of the elements of the scientific instrument support system 800 in FIG8 may be absent. Furthermore, in some embodiments, multiple elements of the various elements of the scientific instrument support system 800 in FIG8 may be present. For example, the scientific instrument support system 800 may include multiple user local computing devices 820 (e.g., different user local computing devices 820 associated with different users or located in different locations). In another example, the scientific instrument support system 800 may include multiple scientific instruments 810, all of which communicate with a serving local computing device 830 and / or a remote computing device 840; in such embodiments, the serving local computing device 830 may monitor multiple scientific instruments 810, and the serving local computing device 830 may cause updates, or other information may be simultaneously “broadcast” to multiple scientific instruments 810. Different scientific instruments among the scientific instruments 810 in the scientific instrument support system 800 may be close to each other (e.g., in the same room) or far from each other (e.g., on different floors of a building, in different buildings, in different cities, etc.). In some embodiments, scientific instrument 810 may be connected to an Internet of Things (IoT) stack that allows command and control of scientific instrument 810 via web-based applications, virtual or augmented reality applications, mobile applications, and / or desktop applications. Any of these applications may be accessible to a user operating a user-local computing device 820 that communicates with scientific instrument 810 via an intermediate remote computing device 840. In some embodiments, scientific instrument 810 may be sold by a manufacturer along with one or more associated user-local computing devices 820 as part of a local scientific instrument computing unit 812.

[0086] In some embodiments, the different scientific instruments included in the scientific instruments 810 of the scientific instrument support system 800 may be different types of scientific instruments 810: for example, one scientific instrument 810 may be an optical emission spectrometer. In some such embodiments, a remote computing device 840 and / or a user local computing device 820 may combine data from the different types of scientific instruments 810 included in the scientific instrument support system 800.

Claims

1. A method for acquiring a digital image of a spectrum from a time series of image frames of a spectrum imaged onto an image sensor by an optical system of a spectrometer, the method comprising: Acquire the time series of the corresponding spectral image frames recorded as the imaging spectrum moves relative to the image sensor; Each image frame is upsampled to generate a corresponding upsampled image frame; Transform the upsampled image frames to align with the recorded spectrum; as well as The transformed image frames are combined to generate the digital image.

2. The method of claim 1, wherein combining the transformed digital image frames comprises: The corresponding array values ​​for each aligned image frame are averaged to generate an array from a plurality of arrays equal in number to the number of image frames in the time series of the digital image frames.

3. The method according to any one of the preceding claims, wherein the shift in the spectrum is caused by thermal expansion and / or thermal contraction of at least one of the optical components.

4. The method according to any one of the preceding claims, wherein the movement is caused by the movement, such as vibrational movement, of an actuator coupled to one of the components.

5. The method according to any one of the preceding claims, which is dependent on claim 3, wherein the thermal expansion and / or thermal contraction caused by temperature changes are due to the operation of the spectrometer.

6. The method according to any one of the preceding claims, which is dependent on claim 3, wherein the temperature change is caused by heating or cooling of the optical system by a temperature control system.

7. The method according to any one of the preceding claims, wherein transforming the image frame to align the recorded spectrum comprises: The image frame is translated using linear translation.

8. The method according to any one of the preceding claims, wherein transforming the image frame to align the recorded spectrum comprises: Rotate the image frame.

9. The method according to any one of claims 1 to 6, wherein transforming the image frame to align the recorded spectrum comprises: The deformation field is applied to the image frame.

10. The method according to any one of the preceding claims, wherein the spectrometer is an inductively coupled plasma optical emission spectrometer (ICP-OES).

11. The method according to any one of the preceding claims, further comprising: The time series of image frames is recorded as the imaging spectrum shifts relative to the image sensor.

12. A system comprising one or more processors and one or more memories storing computer-readable instructions thereon, the computer-readable instructions being configured to cause the one or more processors to perform operations including the steps of any one of claims 1 to 11.

13. The system of claim 12, further comprising a spectrometer for acquiring the time series of images.

14. The system of claim 13, wherein the spectrometer includes an optical system, wherein the optical system includes an image sensor having a pixel array defining a sensor resolution.

15. A computer-readable medium comprising instructions, which, when executed by one or more data processing means, cause the one or more data processing means to perform operations including the steps according to any one of claims 1 to 11; or A computer program comprising instructions that, when executed by a computer, cause the computer to perform the method according to any one of claims 1 to 11.

Images