Spectral image analysis via integral constraint fitting

Abstract

To provide a system or a technology for facilitating spectral image analysis via integral constraint fitting.SOLUTION: In various embodiments, a system is able to access a spectral image of a sample captured by a scientific device, where each pixel of the spectral image corresponds to an energy spectrum. In various aspects, the system is able to fit a function to the energy spectrum on a pixel-by-pixel basis, where the function includes a plurality of additively combined terms, a first term of the plurality of terms represents a fine structure of the energy spectrum, and an integral associated with the first term is constrained to be zero. In various cases, the system is able to segment the spectral image by material based on the first term.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application was filed on March 20, 2024, and is entitled "SPECTRAL IMAGE This application claims the benefit of and priority to pending EP patent application no. 24164816.1 entitled "ANALYSIS VIA INTEGRATION-CONSTRAINED FITTING," which is incorporated herein by reference in its entirety.

[0002] The present invention relates to spectral image analysis via integral constraint fitting. [Background technology]

[0003] Various scientific devices are capable of capturing spectral images of a sample, and analysis of such spectral images can be facilitated by fitting a function to the energy spectrum represented by the spectral image. Summary of the Invention

[0004] The following presents a summary to provide a basic understanding of one or more embodiments. This summary is not intended to identify key or critical elements or to define the scope of particular embodiments or the claims. Its sole purpose is to present concepts in a simplified form as a prelude to the more detailed description that is presented later. One or more embodiments described herein describe devices, systems, computer-implemented methods, devices, or computer program products that facilitate spectral image analysis via integral constraint fitting.

[0005] In accordance with one or more embodiments, a system is provided. The system may include a non-transitory computer-readable memory capable of storing computer-executable components. The system may further include a processor operably coupled to the non-transitory computer-readable memory and capable of executing the computer-executable components stored in the non-transitory computer-readable memory. In various embodiments, the computer-executable components may include an access component capable of accessing a spectral image of the sample captured from a scientific device, wherein each pixel of the spectral image corresponds to an energy spectrum. In various aspects, the computer-executable components may include a fitting component capable of fitting a function to the energy spectrum on a pixel-by-pixel basis, the function including a plurality of additively combined terms, a first term of which represents a fine structure of the energy spectrum, and an integral associated with the first term being constrained to be zero. In various cases, the computer-executable components may include an execution component capable of segmenting the spectral image by material based on the first term.

[0006] In accordance with one or more embodiments, a computer-implemented method is provided. In various embodiments, the computer-implemented method includes accessing, by a device operatively coupled to a processor, a spectral image of a sample captured by a scientific device. Each pixel of the spectral image corresponds to an energy spectrum. The device fits a function to the energy spectrum on a pixel-by-pixel basis, the function including multiple additively combined terms, a first term of which represents fine structure in the energy spectrum, where an integral associated with the first term is constrained to be zero. The device then segments the spectral image by material based on the first term.

[0007] In accordance with one or more embodiments, a computer program product is provided for facilitating spectral image analysis via integral constraint fitting. In various embodiments, the computer program product can include a non-transitory computer-readable memory having program instructions embedded therein. In various aspects, the program instructions are executable by a processor to access a spectral image of a sample captured with an electron energy loss microscope, where each pixel of the spectral image corresponds to an energy loss spectrum. A function is fitted to the energy loss spectrum on a pixel-by-pixel basis, the function including a fine structure term, where an integral associated with the fine structure term is constrained to zero, and the spectral image is segmented based on the fine structure term rather than based on a remainder of the function. [Brief explanation of the drawings]

[0008] Various embodiments will be readily understood from the following detailed description taken in conjunction with the accompanying drawings. To facilitate this description, like reference numerals refer to like structural elements. The embodiments are illustrated by way of example, and not by way of limitation. The figures are not necessarily drawn to scale.

[0009] [Figure 1] 1 illustrates an example non-limiting block diagram of a scientific device module according to various embodiments described herein. [Figure 2] 1 illustrates an example non-limiting flow diagram of a computer-implemented method according to various embodiments described herein. [Figure 3] 1 shows a block diagram of an example non-limiting system that facilitates spectral image analysis via integral constraint fitting, according to one or more embodiments described herein. [Figure 4] FIG. 1 illustrates an example, non-limiting block diagram of a spectral image according to one or more embodiments described herein. [Figure 5]FIG. 1 shows a block diagram of an example non-limiting system that includes a fitting function that includes a background term, an atomic cross-section term, and a fine structure term to facilitate spectral image analysis via integral constraint fitting according to one or more embodiments described herein. [Figure 6] 1 illustrates an example, non-limiting block diagram of a fitness function and its constituent terms, according to one or more embodiments described herein. [Figure 7] 1 illustrates an example, non-limiting block diagram of a fitness function and its constituent terms, according to one or more embodiments described herein. [Figure 8] 1 illustrates an example, non-limiting block diagram of a fitness function and its constituent terms, according to one or more embodiments described herein. [Figure 9] 1 illustrates an example, non-limiting block diagram of a fitness function and its constituent terms, according to one or more embodiments described herein. [Figure 10] 1 illustrates an example, non-limiting block diagram of a fitness function and its constituent terms, according to one or more embodiments described herein. [Figure 11] 1 illustrates an example, non-limiting block diagram of a fitness function and its constituent terms, according to one or more embodiments described herein. [Figure 12] 1 illustrates an example, non-limiting block diagram of a fitness function and its constituent terms, according to one or more embodiments described herein. [Figure 13] 1 illustrates an example, non-limiting block diagram of a fitness function and its constituent terms, according to one or more embodiments described herein. [Figure 14] 1 shows a block diagram of an example non-limiting system that facilitates spectral image analysis via integral constraint fitting, according to one or more embodiments described herein. [Figure 15] 1 shows an example of a non-limiting block diagram illustrating how a segmentation mask can be generated in accordance with one or more embodiments described herein. [Figure 16] 10 illustrates exemplary, non-limiting simulation results according to one or more embodiments described herein. [Figure 17]10 illustrates exemplary, non-limiting simulation results according to one or more embodiments described herein. [Figure 18] 10 illustrates exemplary, non-limiting simulation results according to one or more embodiments described herein. [Figure 19] 10 illustrates exemplary, non-limiting simulation results according to one or more embodiments described herein. [Figure 20] 10 illustrates exemplary, non-limiting simulation results according to one or more embodiments described herein. [Figure 21] 10 illustrates exemplary, non-limiting simulation results according to one or more embodiments described herein. [Figure 22] 10 illustrates exemplary, non-limiting simulation results according to one or more embodiments described herein. [Figure 23] 10 illustrates exemplary, non-limiting simulation results according to one or more embodiments described herein. [Figure 24] FIG. 1 illustrates an example, non-limiting block diagram of a graphical user interface that can be used to implement some or all of the methods or techniques disclosed herein, in accordance with various embodiments described herein. [Figure 25] 1 illustrates an example, non-limiting block diagram of a computing device capable of implementing some or all of the methods or techniques disclosed herein, in accordance with various embodiments described herein. [Figure 26] 1 illustrates an example, non-limiting block diagram of a scientific device support system in which some or all of the methods or techniques disclosed herein may be implemented, in accordance with various embodiments described herein. [Figure 27] 1 illustrates a block diagram of an example non-limiting operating environment that can facilitate one or more embodiments described herein. [Figure 28] 1 illustrates an example of a network environment operable to execute various implementations described herein. DETAILED DESCRIPTION OF THE INVENTION

[0010] The following detailed description is exemplary only and is not intended to limit the embodiments and their application / uses, nor is there any intention to be bound by any express or implied information presented in the preceding Background and Summary sections or in the Detailed Description section.

[0011] One or more embodiments will now be described with reference to the drawings, wherein like reference numerals are used to refer to like elements throughout. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a more thorough understanding of one or more embodiments. It will be apparent, however, that in various instances, one or more embodiments may be practiced without these specific details.

[0012] Various operations may be described as multiple separate actions or operations, in a sequence that is most helpful in understanding the subject matter disclosed herein. However, the order of description should not be construed as implying that these operations are necessarily order dependent. In particular, these operations may be performed in a different order than presented. The operations described may be performed in a different order than in the described embodiment. Various additional operations may be performed, or the operations described may be omitted, in additional embodiments.

[0013] Although some elements may be referred to in the singular (e.g., a "processing device"), any suitable element may be represented by multiple instances of that element, and vice versa. For example, a set of operations described as being performed by a processing device may be implemented with different ones of the operations performed by different processing devices. As used herein, the phrase "based on" should be understood to mean "based at least in part on," unless otherwise specified.

[0014] A scientific device (e.g., mass spectrometer, charged particle microscope) may be any suitable computerized device (e.g., capable of capturing or generating spectroscopic images or compositional spectra) capable of acquiring or generating electronic measurements in a scientific, laboratory, research, or clinical operational setting. To facilitate the capture or generation of such electronic measurements, a scientific device may utilize a complex arrangement of operable parts (e.g., ion sources, ion lenses, heaters, coolers, infusion valves, infusion pumps, circuit cycles, sample stages, apertures, etc.), sensors (ion detectors, voltmeters, thermistors, voltage dividers, manometers), or consumables (transport fluids, calibrators, filters, etc.).

[0015] Various scientific devices (e.g., electron energy loss microscopes) can capture spectral images of a sample. A spectral image is an array of pixels that represent the sample; however, instead of each pixel representing a single intensity value (e.g., a measured Hounsfield unit value), each pixel can represent a distinct energy spectrum. That is, each pixel can have multiple measured intensities or counts distributed across defined energy bins. In other words, a scientific device that generates a spectral image of a sample can be thought of as counting, for each defined energy bin, the number of charged particles (e.g., electrons) that fall within that bin that strike the sample at each pixel's location. Analyzing such energy spectra can identify or reveal information about the sample, such as the electronic structure of chemicals in the sample.

[0016] Analysis of a spectral image can be facilitated by fitting a function to the energy spectrum represented by a pixel in the spectral image. In particular, the energy spectrum of a pixel can be modeled as an additive or multiplicative combination of background, atomic cross-sections, and fine structure. The background can be represented by one or more monotonically decreasing power functions. The atomic cross-sections can be represented by one or more probability functions that describe the probability that a given incident excitation (e.g., a charged particle beam) will be scattered in a given way when colliding with a given type of atom. The fine structure can be represented by a spline constructed with appropriate basis functions, such as polynomials or rectangular functions. Note that the background may be considered an uninteresting or uninformative component of the energy spectrum. Furthermore, the coefficients (e.g., amplitudes) of the atomic cross-sections can be set to indicate interesting or useful properties of the sample (e.g., the abundance of a chemical in the sample), but the general shape of the atomic cross-sections may be known. (For example, the properties of the charged particle beam used in a scientific device to capture a spectral image can be controllably selected, and it is understood how to use those properties to calculate probabilistic atomic cross-section functions (up to a scaling factor) for desired atoms.) Thus, in certain situations or contexts, atomic cross-sections may be considered the less interesting or least informative component of the energy spectrum. On the other hand, fine structure, being an unknown component of the energy spectrum, may be considered the more interesting or most informative component. Indeed, this fine structure may be thought of as carrying or conveying information about the electronic structure of the sample (e.g., effects from surrounding atoms or compounds). To further explain, the goal or objective of identifying a function fit to the energy spectra of a spectral image may be to discover or capture a particular form or shape of the fine structure of those energy spectra.Therefore, after fitting the energy spectra of the spectral images, the fine structure of those energy spectra (other than the background and atomic cross sections) can be utilized to perform appropriate downstream analyses of the spectral images, such as image classification and image segmentation (e.g., to identify electronic structures and compounds at different locations on the sample).

[0017] Unfortunately, as recognized by the inventors of the various embodiments described herein, existing techniques for modeling the fine structure of energy spectra suffer from various drawbacks.

[0018] First, the basis functions used to model the microstructure can accept various input parameters as arguments. One example of such an argument is the energy window (also referred to as the energy width). The inventors have recognized that existing techniques are overly sensitive to the choice of energy window. That is, the inventors have recognized that changes in the selected energy window can lead to corresponding changes in the resulting microstructure, and therefore to corresponding changes in downstream image classification or segmentation. However, such downstream changes should not occur (e.g., should lead to the same or similar downstream results). Therefore, it is believed that existing techniques are unstable with respect to the selected energy window.

[0019] Second, in addition to being overly sensitive to the energy window selection, the inventors also recognized that existing techniques generally provide insufficient accuracy for downstream analysis. In particular, the inventors recognized that when fine structures are obtained through existing techniques, downstream results, image classification, or image segmentation obtained from those fine structures often have lower accuracy than they should or should have (e.g., the determined electronic structure is often inaccurate, and many pixels are often misclassified or missegmented). Furthermore, because atomic cross sections and fine structures are simultaneously fitted to the energy spectrum, they can be considered interdependent (e.g., the fitted value of the atomic cross section depends on the fitted value of the fine structure, and vice versa). Therefore, because the fine structures in existing techniques are often inaccurate, the amplitudes of the atomic cross sections calculated by existing techniques are often inaccurate, which can lead to chemical abundances that are too high, too low, or incorrect. Therefore, existing techniques are considered to be insufficiently reliable.

[0020] As such, systems and techniques that can ameliorate one or more of these technical problems may be desirable.

[0021] Various embodiments described herein may address one or more of these technical problems. One or more embodiments described herein may include a system, computer-implemented method, device, or computer program product that facilitates spectral image analysis via integral constraint fitting. In particular, as described above, the energy spectrum of a pixel in a spectral image can be fit by a function composed of three distinct terms: background, atomic cross-section, and fine structure. That is, each pixel can be considered to have its own background, its own atomic cross-section, and its own fine structure. Also, as described above, downstream analysis (e.g., image classification, image segmentation) can be performed based on the fine structure of those pixels, rather than on their background or atomic cross-section. Currently, existing implementations of techniques result in downstream analysis that is either overly energy window sensitive (e.g., the choice of energy window makes a large difference in image classification or segmentation) or is too inaccurate (e.g., includes a large number of misclassified or mis-segmented pixels). ). The inventors have demonstrated that constraining one or more integrals associated with the fine structure of a pixel in a spectral image can lower the energy window and increase the accuracy of downstream analysis. Specifically, the inventors recognized that during the fitting process, the fine structure of a pixel image can be represented not only by a spline, but also by a spline that may be associated with integrals with respect to energy (or energy loss) that are constrained to zero. That is, the weighted integrals of the spline can be constrained to zero. The inventors have experimentally verified that applying such zero-integral constraints on the fine structure can reduce the sensitivity of the energy window and increase the accuracy of downstream analysis. That is, when such zero-integral constraints are applied, the resulting detailed structure of the spectral image can better represent or convey information about the electronic structure or compounds in the sample.

[0022] Various embodiments described herein can be considered as computerized tools (e.g., a suitable combination of computer-executable hardware or computer-executable software) through integral constraint fitting. In various aspects, such computerized tools can include an accessing component, a fitting component, or an execution component.

[0023] In various embodiments, a scientific device can be used. In various aspects, the scientific device can be a suitable computerized device capable of electronically capturing or generating a spectral image of a suitable sample (e.g., a lamellar sample). As a non-limiting example, the scientific device can be an electron energy loss microscope.

[0024] In either case, the spectral image can be a three-dimensional array, with two dimensions of such an array representing pixels that collectively display or describe the sample, and the third dimension of the array representing the pixel-by-pixel energy spectrum recorded by the scientific device. As a non-limiting example, the spectral image can be an x, y, and z array, where x, y, and z are any suitable positive integers. In such a case, the spectral image can be considered to consist of a total of xy pixels, and each of these pixels can be considered to have a respective distribution over z energy bins (e.g., the size of these z energy bins can be determined to achieve an appropriate level of energy loss granularity depending on the appropriate electron volt (eV) resolution supported by the detector of the scientific device). That is, each pixel can be considered to represent or define a unique two-dimensional location on the sample, and the scientific device can count or measure the number of incident ions belonging to each z energy bin that were emitted by the scientific device and struck the sample at each of these unique two-dimensional locations.

[0025] In various cases, it may be desirable to segment or analyze the spectral image, such as to determine which chemical components or compounds are located at each location in the sample, and in various cases, the computerized tools described herein can facilitate such segmentation or analysis.

[0026] In various embodiments, an access component of the computerized tool can electronically access the spectral image. For example, the access component can receive, extract, or otherwise obtain the spectral image from a suitable centralized or distributed data structure (e.g., a graph data structure, an association data structure, a hybrid data structure, etc.). As a non-limiting example, the access component can receive, extract, or obtain the spectral image from the scientific device itself. In either case, the access component can be considered a conduit through which other components of the computerized tool can electronically interact with (read, write, edit, copy, manipulate) the spectral image.

[0027] In various embodiments, the fitting component of the computerized tool can electronically fit a function to the energy spectrum represented or transmitted by the spectral image on a pixel-by-pixel basis. In various aspects, the function may include a background term, an atomic cross-section term, and a fine structure term. In various aspects, the fine structure term can be integrally constrained to 0, as described herein.

[0028] More specifically, consider a given pixel in a spectral image. In various cases, the given pixel can have a given energy spectrum. Continuing with the example above, if the spectral image is an xxyxz array, the given energy spectrum can be a distribution of ion counts or intensities across z energy bins. That is, the given energy spectrum can be viewed as a sequence of z tuples, each tuple being a different energy bin and corresponding count or intensity. In various embodiments, the fitting component can fit a function to the sequence of such tuples (e.g., by a suitable fitting technique such as least-squares sum (LSS)). Here, the incident energy (or energy loss) can be viewed as the independent variable of such a function, and the count or intensity can be viewed as the dependent variable of such a function. Here, in various cases, the function can be fitted to a constituent function of a background term, an atomic cross-section term, and a fine structure term.

[0029] In various cases, the background term can be any suitable number of monotonically decreasing power law functions (e.g., a single monotonically decreasing power law function or a sum of multiple monotonically decreasing power law functions) including any suitable number of first fitting coefficients. In some cases, the various first fitting coefficients can be exponents of the monotonically decreasing power law functions. In some cases, the various first fitting coefficients can be scaling factors that are each multiplied by the monotonically decreasing power law functions.

[0030] In various embodiments, the atomic cross-section term can be a probabilistic atomic cross-section term probability function (e.g., a single probabilistic atomic cross-section term probability function or a sum of multiple probabilistic atomic cross-section term probability functions) that includes any suitable number of second fitting coefficients. In various cases, the second fitting coefficients can each be a scaling factor multiplied by the probabilistic atomic cross-section term probability function.

[0031] In various cases, the fine structure terms can be splines with any suitable basis functions including any suitable number of third fitting coefficients. In various cases, the basis functions can be polynomials of any suitable order or degree (e.g., zeroth-order or constant polynomials, first-order or linear polynomials, second-order polynomials, third-order polynomials), and the third fitting coefficients can be each term of such basis functions or a scaling factor multiplied by each term of such basis functions. In such situations, polynomials adjacent to the spline can be forced to match in value and first derivative at their boundaries. In other cases, the basis functions can be any suitable non-polynomial functions, such as rectangular functions (e.g., the difference between Heaviside step functions) or triangular functions (e.g., functions based on absolute value functions), each multiplied by a separate one of the third fitting coefficients. Regardless of the basis functions selected for the spline, the spline can be defined in steps at any suitable intervals. In some cases, such intervals can be linear or even. In other cases, such intervals may be quadratic in space (eg, each successive interval is proportional to the square of the previous interval).

[0032] In some cases, atomic cross-section and fine structure terms can be convolved with the low-loss (or equivalently high-energy) portion of a given energy spectrum. More specifically, various z-energy bins in a particular energy spectral region can be considered to form the low-loss (or high-energy) region of the given energy spectrum. As a non-limiting example, the range from 0 eV loss to 200 eV loss in the bin cover or region can be considered to collectively form the low-loss (or high-energy) region of the given energy spectrum. The range from 0 eV loss to 200 eV loss can be considered to collectively form the low-loss (or high-energy) region of the given energy spectrum. Note that interesting electronic activity (e.g., K-edge occurrences in electron energy loss spectra) typically occurs in energy bands or bins beyond the 200 eV loss. Therefore, count or intensity measurements measured in the low-loss region of a given energy spectrum (e.g., measured before the 200 eV loss) can be convolved with atomic cross-section and fine structure terms. Such convolution serves to mitigate or otherwise control the effect of sample thickening on the fit function.

[0033] In either case, the fitting component can fit a function to a given energy spectrum (e.g., via LSS). Such a fit can be viewed as identifying an error metric (e.g., sum of squared errors) between the given energy spectrum and the function that is minimized or nearly minimized (e.g., below an appropriate threshold) by specific values ​​of the first, second, and third fitting coefficients. In this respect, the function can be viewed as closely matching the given energy spectrum. However, such a fit can constrain the integral associated with the fine structure term with respect to energy (or energy loss) to zero. That is, fitting a function to a given energy spectrum can be viewed as identifying specific values ​​of the first, second, and third fitting coefficients that not only minimize the error metric, but also bring the integral associated with the fine structure term as close to zero as possible (e.g., within an appropriate threshold margin of zero).

[0034] In various embodiments, the execution component of the computerized tool can electronically perform appropriate downstream analyses on the spectral image based on the fitted fine structure terms identified by the fitting component. That is, the execution component can perform such downstream analyses regardless of the fitted background terms or fitted atomic cross-section terms of the spectral image. As a non-limiting example, the execution component can calculate or generate a segmentation mask for the spectral image by applying any suitable clustering algorithm (e.g., K-means) to the fitted fine structure terms or by running any suitable trained device learning segment (e.g., a deep learning neural network) on the fitted fine structure terms. In either case, the segmentation mask can be thought of as indicating which of two or more defined classes each pixel of the spectral image belongs to (e.g., the two or more defined classes can represent two or more defined chemical compounds or oxidation states known to be present in the sample, and the segmentation mask can indicate which pixels of the spectral image are composed of one of the defined compounds or oxidation states). In various aspects, the execution component can visually display the segmentation mask on a suitable computer screen or monitor. In various aspects, the execution component can electronically transmit the segmentation mask to an appropriate computing device.

[0035] It should be noted that the inventors have experimentally verified that the segmentation mask can be considered more accurate or reliable than other methods and less sensitive to the choice of energy window due to the fact that the fine structure terms of the spectral image are integrally constrained to 0. That is, if the fitted fine structure terms of the spectral image are identified without being integrally constrained to 0, the accuracy of the segmentation mask may be lower (e.g., have a higher number of misclassified pixels) and may vary more widely depending on the choice of energy window.

[0036] The computerized tools described herein can therefore be viewed as providing improved analysis of spectral images, and can achieve such improvement by constraining the fitted fine structure terms to zero integrals.

[0037] Various embodiments described herein may use hardware or software to solve inherently highly technical problems (e.g., to facilitate spectral image analysis via integral constraint fitting) that are not abstract and cannot be performed as a series of mental acts by a human. Furthermore, some of the processes performed may be performed by specialized computers (e.g., electron energy loss microscopes) to perform defined acts related to spectral images.

[0038] For example, such defined acts include: accessing, by a device operatively coupled to the processor, a spectral image of the sample captured by a scientific device, each pixel of the spectral image corresponding to an energy spectrum; fitting, by the device, a function to the energy spectrum on a pixel-by-pixel basis, the function including a plurality of additively combined terms, a first term of the plurality of terms representing the fine structure of the energy spectrum, where an integral associated with the first term is constrained to be zero; and segmenting, by the device, the spectral image by material based on the first term. In various embodiments, the plurality of terms can further include a second term representing a monotonically decreasing background of the energy spectrum and a third term representing atomic cross-sections of the energy spectrum.

[0039] Such defined operations are inherently computerized. Indeed, scientific devices such as electron energy loss microscopes are highly sophisticated computerized devices comprised of specific computerized hardware (e.g., temperature sensors, pressure sensors, ion beam emitters, ion-forming emitters, ion-focusing lenses, mass analyzers, ion detectors, beam apertures, fluid valves, etc.). Scientific devices and the operations they perform cannot be implemented in any reasonable or practical way by the human brain or by pen and paper without a computer. Furthermore, spectral images are arrays of particular types of pixels, each with its own measured energy spectrum or distribution (rather than its own Hounsfield unit value). Spectral images cannot be generated or captured in any reasonable or practical way by the human brain or by pen and paper without a computer. Furthermore, spectral image segmentation is a unique computerized task, which involves classifying each pixel in a spectral image into one of two or more defined classes, thereby generating a segmentation mask that indicates which pixels in the spectral image belong to which class. Discussing the task of computerized image segmentation outside the context of computing simply makes no sense.

[0040] Furthermore, various embodiments described herein can integrate various teachings related to spectral image analysis via integral constraint fitting into practical applications. As explained above, when a spectral image of a sample is captured, each pixel in the spectral image has a unique energy spectrum. Also, as explained above, each of these energy spectra can be fit to a function composed of three different terms: a background term, an atomic cross-section term, and a fine structure term. The background term can be appropriately modeled by one or more monotonically decreasing power functions. Therefore, the background term has a known shape or form and can be considered a quantity of no interest. Similarly, the atomic cross-section term can be appropriately modeled by a known stochastic or probabilistic function, up to a scaling factor or amplitude, depending on which particular atom is of interest to model. Therefore, the atomic cross-section term has a known shape or form and can also be considered a quantity of no interest. However, the fine structure term can be modeled as a spline of any appropriate basis function (e.g., polynomial basis function, rectangular basis function, triangular basis function, etc.) and can be considered to capture or represent all of the remaining structural information in the energy spectrum of a spectral image after accounting for background and atomic cross-section terms. That is, the fine structure term can be considered an interesting quantity because it has an unknown shape and morphology. Furthermore, the fine structure term can be considered the most informative component of the energy spectrum of a spectral image, and such informative component can be used to determine where and how much of which chemical compounds appear in the spectral image. Unfortunately, the inventors have recognized that existing techniques can make such downstream determinations overly sensitive or unstable (e.g., highly variable depending on the energy window selection) or overly unreliable or inaccurate (e.g., prone to misclassification and missegmentation).

[0041] Various embodiments described herein help alleviate one or more of these technical problems. In particular, various embodiments described herein involve fitting a function to the energy spectrum of a spectral image, as described above, where the function is comprised of a background term, an atomic cross-section term, and a fine structure term. However, rather than performing such fitting in an unconstrained manner, various embodiments described herein can perform such fitting while simultaneously enforcing a zero-integral constraint on the fine structure term. That is, specific values ​​of any fitting coefficients included in the background term, the atomic cross-section term, and the fine structure term can be identified (e.g., set within any suitable threshold margin) not only to minimize the error index between the energy spectrum and the function, but also to constrain the integral of the fine structure term with respect to the incident energy or energy loss to zero. The inventors have performed various experiments to verify or demonstrate that such zero-integral constraints reduce the sensitivity or instability of downstream analysis (e.g., reduce variation between different energy windows) and increase the accuracy and reliability of such downstream analysis (e.g., reduce pixel misclassification or mis-segmentation). Therefore, the various embodiments described herein can be considered as superior or improved techniques for fitting or identifying the fine structure of the energy spectrum of a spectral image compared to existing techniques. For at least these reasons, the various embodiments described herein can be considered as specific and clear technical improvements in the area of ​​spectral imaging. Therefore, the various embodiments described herein certainly qualify as useful and practical applications of computers.

[0042] Furthermore, various embodiments described herein can control real-world, tangible devices based on the disclosed teachings. For example, various embodiments described herein can electronically activate, deactivate, or otherwise activate actual hardware (e.g., ion beam emitters, ion focusing lenses, carrier fluid valves / pumps) of real-world scientific devices (e.g., electron energy loss microscopes, etc.). Real-world analysis can be performed on real-world data captured by those real-world scientific devices (e.g., calculating segmentation masks for energy loss spectral images) and the results of such real-world analysis can be electronically displayed on a physical computer screen (e.g., visually displaying the calculated segmentation masks for viewing by a user or technician).

[0043] FIG. 1 illustrates an example, non-limiting block diagram of a scientific device module 102 according to various embodiments described herein.

[0044] The scientific device module 102 may be implemented by circuitry (e.g., including electrical and / or optical components) such as a programmed computing device. The logic of the scientific device module 102 may be contained in a single computing device or may be distributed across multiple computing devices that communicate with each other as needed. Examples of computing devices that may implement the scientific device module 102, alone or in combination, are described herein using Figures 25 and 27, and examples of interconnected systems or networks in which the scientific device module 102 may be implemented across one or more of the computing devices are discussed herein with reference to Figures 26 and 28.

[0045] The scientific device module 102 may include first logic 104, second logic 106, and third logic 108. As used herein, the term “logic” may include a device that performs a sequence of operations associated with the logic. For example, any of the logic elements included in the scientific device module 102 may be implemented by one or more computing devices programmed with instructions that cause one or more processing devices of the computing devices to perform a sequence of associated operations. In particular embodiments, a logic element may include one or more non-transitory computer-readable media having instructions that, when executed by one or more processing devices of the one or more computing devices, cause the one or more computing devices to perform the associated sequence of operations. As used herein, the term “module” may refer to a collection of one or more logic elements that together perform a function associated with the module. Different logic elements within a module may take the same form or different forms. For example, some logic within a module may be implemented by a programmed general-purpose processing device, while other logic within the module may be implemented by an application-specific integrated circuit (ASIC). In another example, different ones of the logic elements in a module may be associated with different sets of instructions executed by one or more processing devices. A module may omit one or more of the logic elements shown in an associated figure. For example, a module may include a subset of the logic elements shown in an associated figure if that module performs a subset of the operations described herein with reference to that module.

[0046] In various embodiments, a scientific device corresponding to the scientific device module 102 may be used. In various aspects, a scientific device may be any suitable computerized device capable of electronically measuring a scientific, clinical, or research-related property, characteristic, or attribute of an analytical sample (e.g., a known or unknown mixture, compound, or collection of substances). As a non-limiting example, a scientific device may be a mass spectrometer operatively coupled to a gas or liquid chromatograph. In such a case, the scientific device may measure or determine the ion spectrum of the analytical sample (e.g., relative ion abundance as a function of mass-to-charge ratio). As another non-limiting example, a scientific device may be a scanning electron microscope. In such a case, the scientific device may measure or determine the surface structure of the analytical sample. As yet another non-limiting example, a scientific device may be a transmission electron microscope. In such a case, the scientific device may measure or determine details of the internal structure of the analytical sample. As a more general non-limiting example, a scientific device may be any suitable type of charged particle microscope (e.g., some types of microscopes may capture images using non-electron ion beams).

[0047] In various embodiments, the initial logic 104 has access to a spectral image captured or generated by the scientific device. For various aspects, the spectral image can be any suitable array of pixels, with each pixel containing its own unique energy spectrum. As a non-limiting example, the scientific device can be an electron energy loss microscope, and the spectral image can be an electron energy loss spectroscopy (EELS) image. In that case, each pixel can contain a measured count or intensity value across a range of defined energy loss bins or bands. That is, each pixel can represent a respective two-dimensional location of the analytical sample, and each pixel can indicate the number of electrons emitted from the scientific device that belong to each defined energy loss bin or band and impinge on the analytical sample at each two-dimensional location during a scan performed by the scientific device.

[0048] In various embodiments, the second logic 106 can electronically fit a function to the energy spectrum represented or transmitted by the spectral image on a pixel-by-pixel basis. In various aspects, the function can include multiple additively combined terms. In particular, the function can be the sum of a background term, an atomic cross-section term, and a fine structure term. In various cases, the background term can be comprised of a decreasing power function multiplicatively scaled according to the respective fitting coefficients. In various cases, the atomic cross-section term can be comprised of a stochastic or probabilistic cross-section function multiplicatively scaled according to the respective fitting coefficients. In various aspects, the fine structure term can be comprised of a spline whose individual components (e.g., constant, linear, quadratic, cubic) are multiplicatively scaled according to the respective fitting coefficients, or can be comprised of a sum of step functions multiplicatively scaled according to the respective fitting functions. In some cases, the atomic cross-section term and the fine structure term can be convolved with a low-loss portion of the energy spectrum of the spectral image. In various cases, the function of the fine structure term can be constrained to zero during fitting, as described herein. In various aspects, such integral constraints can cause the final, resulting, or fitted fine structure terms to more accurately, closely, or reliably represent interesting or desirable information about the analyzed sample conveyed by the energy spectrum of the spectral image.

[0049] In various embodiments, the third logic 108 can segment the spectral image by material, element, compound, or other suitable physical or chemical property based on the final, resulting, or fitted fine structure terms (and not based on background or atomic cross-section terms). In some aspects, such segmentation can be facilitated by applying a suitable unsupervised clustering technique (e.g., K-means) to the final, resulting, or fitted fine structure terms of the pixels of the spectral image. In other aspects, such segmentation can be facilitated by running any suitable trained machine learning model (e.g., a deep learning neural network) on the final, resulting, or fitted fine structure terms of the pixels of the spectral image. In some cases, such segmentation can be achieved by analyzing the scalar output of the final, resulting, or fitted fine structure terms. In some cases, such segmentation can instead be achieved by analyzing the fitting coefficients of the final, resulting, or fitted fine structure terms. In either case, because the fine structure terms are constrained to zero, the segmentation is considered more accurate or reliable than in other cases.

[0050] Thus, the scientific device module 102 can facilitate the analysis of spectral images through integral constraint fitting.

[0051] Figure 2 illustrates an example, non-limiting flow diagram of a computer-implemented method 200 in accordance with various embodiments described herein. The operations of computer-implemented method 200 can be used in an appropriate context to perform appropriate operations (e.g., can be performed by or in combination with any of the various modules, computing devices, or graphical user interfaces described in Figures 1, 24, 25, 26, 27, and 28). Although operations are illustrated in Figure 2 once each and in a particular order, the operations can be reordered or repeated as desired (e.g., different operations performed can be performed in parallel, as appropriate).

[0052] In various embodiments, operation 202 includes a first operation of accessing, by a device operatively coupled to the processor, a spectral image of the sample captured by the scientific device, wherein pixels of the spectral image each correspond to an energy spectrum. In various cases, first logic 104 can perform or otherwise facilitate operation 202.

[0053] In various cases, operation 204 includes performing a second operation by the device to fit a function to the energy spectrum on a pixel-by-pixel basis, the function including multiple additively combined terms, a first of the multiple terms representing fine structure in the energy spectrum, and an integral associated with the first term being constrained to be zero. In various cases, second logic 106 can perform operation 204 or otherwise facilitate operation 202.

[0054] In various embodiments, operation 206 includes segmenting the spectral image by material by the device based on paragraph 1. In various cases, third logic 108 can perform operation 206 or otherwise facilitate operation 202.

[0055] Thus, the computer-implemented method 200 can facilitate spectral image analysis through integral constraint fitting.

[0056] FIG. 3 illustrates a block diagram of an example non-limiting system 306 that facilitates spectral image analysis via integral constraint fitting, in accordance with one or more embodiments described herein.

[0057] In various embodiments, a scientific device 302 can be used. From various aspects, the scientific device 302 can be used as described above. That is, the scientific device 302 can be any suitable computerized device capable of electronically measuring suitable scientifically, clinically, or research-related characteristics, attributes, or properties of a suitable analytical sample. In particular, the scientific device 302 can be an electron energy loss microscope. In such cases, the scientific device 302 can utilize its component hardware (electron source, anode, condenser lens, condenser aperture, scan coil, objective lens, objective aperture, deflector, condenser, stigmator, electron detector, x-ray detector, and actuable sample stage) to electronically generate or capture a spectral image 304 of a given analytical sample.

[0058] In various embodiments, the spectral image 304 may exhibit any suitable format, size, or dimensions. More specifically, the spectral image 304 may be an array of pixels, with each pixel containing a respective energy loss spectrum. A non-limiting example is described with respect to FIG. 4 .

[0059] FIG. 4 illustrates an example, non-limiting block diagram of a spectral image 304 according to one or more embodiments described herein.

[0060] In various embodiments, the spectral image 304 can include a plurality of pixels 402. As shown, in various cases, the plurality of pixels 402 can include s pixels, from pixel 402(1) to pixel 402(s), for any suitable positive integer s. In various cases, the plurality of pixels 402 can be arranged in any suitable layout or organization. As a non-limiting example, the plurality of pixels 402 can be arranged in a linear array (e.g., an xxy array, where xy=s, for any suitable positive integers x and y).

[0061] In various embodiments, each of the plurality of pixels 402 may include a respective energy loss spectrum. In particular, the scientific device 302 may be configured to scan a given analytical sample by irradiating the given analytical sample with an incident electron beam (or other suitable charged particle beam). Thus, a given analytical sample may be affected by the electrons (or other charged particle beam) of such incident beam. Here, when an electron (or other charged particle) is emitted by the scientific device 302, the electron (or other charged particle) may initially have a known amount of energy. Furthermore, as the electron (or other charged particle) contacts or impacts the given analytical sample, the electron (or other charged particle) may lose or dissipate some of its energy as it passes through the given analytical sample. The amount of energy lost or dissipated by the electron (or other charged particle) wherever the electron (or other charged particle) contacts, strikes, or passes through the given analytical sample may depend on the physical or chemical properties and characteristics (thickness, material composition, elemental concentrations, etc.) of the given analytical sample. In various cases, the scientific device 302 can be configured to record the quantity and energy loss of incident electrons (or other charged particles) that contact, strike, or pass through the analytical sample. In particular, there may be a total of t disjoint (e.g., non-overlapping, etc.) energy loss bins or bands, for any suitable positive integer t (e.g., a first energy loss bin or band may represent energy loss from 0 eV to just before the first eV value; a second energy loss bin or band may represent energy loss from the first eV value to just before a second eV value greater than the first eV value; a third energy loss bin or band may represent energy loss from the second eV value to just before a third eV value greater than the second eV value, etc.). In various cases, the scientific device 302 may be configured to count or measure the number of incident electrons (or other charged particles) contained in each of the t energy loss bins or bands that contact, impinge on, or pass through each physical location of a given analytical sample.

[0062] As a non-limiting example, pixel 402(1) can include an energy loss spectrum 404(1). In various embodiments, energy loss spectrum 404(1) can include a respective energy loss count for each of t energy loss bins or bands. For example, energy loss spectrum 404(1) can include energy loss counts 404(1)(1), which can be a scalar indicative of, or otherwise proportional to, the number of incident electrons (or other charged particles) belonging to the first energy loss bin or band that have contacted, struck, or passed through a given analytical sample at a particular location represented or indicated by pixel 402(1). In other cases, energy loss spectrum 404(1) can include energy loss counts 404(1)(t), which can be a scalar indicative of, or otherwise proportional to, the number of incident electrons (or other charged particles) belonging to the t energy loss bin or band that have contacted, struck, or passed through a given analytical sample at a particular location represented or indicated by pixel 402(1). Thus, energy loss spectrum 404(1) can be viewed as a sequence of bin-count tuples of incident electrons (or other charged particles) that collide, contact, or pass through the analytical sample at the location represented or indicated by pixel 402(1).

[0063] As a non-limiting example, pixel 402(s) can include an energy loss spectrum 404(s). In various embodiments, energy loss spectrum 404(s) can include a respective energy loss count for each of t energy loss bins or bands. For example, energy loss spectrum 404(s) can include energy loss counts 404(s)(1), which can be a scalar indicative of or otherwise proportional to the number of incident electrons (or other charged particles) belonging to a first energy loss bin or band that contacted, collided, or passed through a given analytical sample at a particular location represented or indicated by pixel 402(s). In other cases, the energy loss spectrum 404(s) may include energy loss counts 404(s)(t), which may be a scalar indicating or otherwise proportional to the number of incident electrons (or other charged particles) belonging to the tth energy loss bin or band that have contacted, struck, or passed through a given analytical sample at a particular location represented or indicated by pixel 402(s). Thus, the energy loss spectrum 404(s) may be viewed as a sequence of bin-count tuples of incident electrons (or other charged particles) that have contacted, struck, or passed through a given analytical sample at a location represented or indicated by pixel 402(s).

[0064] 3, the system 306 can be electronically integrated (e.g., via a suitable wired or wireless electronic connection) with the spectral image 304 or scientific device 302. In various cases, it may be desirable to segment or analyze the spectral image 304 according to material, such as to determine where and to what extent which chemical constituents or compounds are located in the analytical sample. In various cases, the system 306 can facilitate such segmentation or analysis as described herein.

[0065] In various aspects, the system 306 can include a processor 308 (e.g., a computer processing unit, microprocessor) and non-transitory computer-readable memory 310 operably, operatively, or communicatively connected or coupled to the processor 308. The non-transitory computer-readable memory 310 can store computer-executable instructions that, when executed by the processor 308, cause the processor 308 or other components of the system 306 (e.g., the access component 312, the fitting component 314, the execution component 316) to perform one or more operations. In various embodiments, the non-transitory computer-readable memory 310 can store, and the processor 308 can execute, the computer-executable components (e.g., the access component 312, the fitting component 314, the execution component 316).

[0066] In various embodiments, the system 306 can include an access component 312. In various aspects, the access component 312 can electronically access the scientific device 302. That is, the access component 312 can electronically communicate with or otherwise interact with the scientific device 302 (e.g., send electronic instructions or commands or receive electronic data). As such, the access component 312 can be considered a proxy or conduit through which other components of the system 306 can interact with, communicate with, or otherwise manipulate the spectral images 304 of the scientific device 302. In various cases, the access component 312 can electronically access the spectral images 304. That is, the access component 312 can electronically receive, extract, or otherwise obtain the spectral images 304 from an appropriate electronic information source or database (not shown). As a non-limiting example, the access component 312 can electronically receive, extract, or otherwise obtain the spectral images 304 from the scientific device 302. In either case, the access component 312 can be considered a proxy or conduit through which other components of the system 306 can interact with or otherwise manipulate the spectral image 304. These are non-limiting examples only. In other cases, the access component 312 can be omitted, and other components of the system 306 can communicate with or interact directly with the scientific device 302 or the spectral image 304.

[0067] In various embodiments, the system 306 can include an access component 314. As described herein, in various aspects, the fitting component 314 can fit a polynomial function to the energy loss spectrum of the spectral image 304, where some terms of the polynomial function can represent fine structure in the energy loss spectrum of the spectral image 304 and can be forced to zero by an integral constraint.

[0068] In various embodiments, the system 306 can include an execution component 316. In various cases, as described herein, the execution component 316 can segment or otherwise analyze the spectral image 304 based on the fine structure fitted by the fitting component, rather than based on the entire energy loss spectrum of the spectral image 304.

[0069] FIG. 5 illustrates a block diagram of an example non-limiting system that includes a fitting function that includes a background term, an atomic cross-section term, and a fine structure term to facilitate spectral image analysis via integral constraint fitting according to one or more embodiments described herein.

[0070] In various embodiments, the fitting component 314 can electronically fit a function 502 to the energy loss spectrum of the spectral image 304 on a pixel-by-pixel basis. In various aspects, the function 502 can include a background term 504, an atomic cross-section term 506, and a fine structure term 508. Various non-limiting aspects are described with respect to FIGS.

[0071] 6-13 illustrate example, non-limiting block diagrams of function 502 and its constituent terms, according to one or more embodiments described herein.

[0072] Consider first Figure 6. In various embodiments, function 502 can be thought of as a function that maps real-valued scalars to other real-valued scalars (e.g.,

number

[0073] In various embodiments, the background term 504 can include one or more power functions 602 that can utilize one or more fitting coefficients 604. Indeed, when considered without regard to the corresponding end (K-end, L-end, etc.), the energy loss spectrum is expected to decline monotonically or exponentially as the energy loss increases, and such decline can be modeled or represented by a power function with a negative exponent. As a non-limiting example, the background term 504 can be expressed as follows:

number

number

[0074] In various aspects, the atomic cross-section term 506 can include one or more material-based probability functions 606 that can utilize one or more fitting coefficients 608. In various cases, the one or more material-based probability functions 606 can be any suitable probability function that represents the probability or likelihood of various atomic events (e.g., scattering, absorption, emission) occurring when an incident electron (or other charged particle) beam collides with one or more given atoms or each atom. For example, the one or more material-based probability functions 606 can include or be based on an appropriate hydrogen wave function or an appropriate Hartree-Slater central field function. In other cases, the one or more material-based probability functions 606 can be derived from a generalized oscillator strength, a convergence angle, and an acceleration voltage. As a non-limiting example, the atomic cross-section term 506 (at least for the inner shells of a given atom) can be expressed as follows:

number

number

number

number

number

number

number

number

number

number

[0075] In various embodiments, the fine structure term 508 can be a spline made up of one or more basis functions 610 that can utilize one or more fitting coefficients 612. In various cases, the one or more basis functions 610 can be any suitable mathematical function, and one or portions of the basis functions 610 can be multiplicatively scaled by one or more fitting coefficients 612.

[0076] As a non-limiting example, the one or more basis functions 610 can take the form of basis splines (B-splines) of appropriate order defined by piecewise polynomials over various energy loss intervals. In such a case, the fine structure term 508 can be written as:

number

[0077] While various embodiments described above include the fine structure term 508 being a spline, it should be understood that these are merely non-limiting examples, and in various embodiments, the fine structure term 508 (and one or more basis functions 610) may take or be represented in any other suitable form.

[0078] As a non-limiting example, one or more basis functions 610 can be defined for each energy loss interval. In such a case, the fine structure term 508 can be written as:

number

[0079] Regardless of what particular mathematical form the basis functions 610 take or represent, the fine structure terms 508 can be thought of as capturing activity or behavior in the energy loss spectrum that is not collectively captured by the background terms 504 and the atomic cross section terms 506.

[0080] In some cases, function 502 can be an additive combination of background terms 504, atomic cross section terms 506, and fine structure terms 508. That is, it can be the case that:

number

number

[0081] In either case, the fine structure term 508, unlike the background term 504 or the atomic cross section term 506, can be considered to represent or contain the most informative or most interesting portion of a given energy loss spectrum (although elemental abundances that may be of interest can be inferred from the atomic cross section term 506).

[0082] In various embodiments, given the energy loss spectrum of any pixel in the spectral image 304, the fitting component 314 can electronically fit the function 502 to that energy loss spectrum. That is, the fitting component 314 can electronically calculate, determine, or otherwise identify values ​​for one or more fitting coefficients 604, one or more fitting coefficients 608, and one or more fitting coefficients 612 that cause the function 502 to closely match or resemble the energy loss spectrum of the given pixel. That is, the fitting component 314 can electronically calculate or approximate any values ​​for the one or more fitting coefficients 604, one or more fitting coefficients 608, and one or more fitting coefficients 612 that minimize the error between the function 502 and the energy loss spectrum therebetween (e.g., below a suitable threshold error value). In various cases, the fitting component 314 can accomplish this through implementation of any suitable curve fitting technique. As a non-limiting example, the fitting component 314 may utilize a least sum of squares (LSS) method to identify which values ​​of the one or more fitting coefficients 604, the one or more fitting coefficients 608, and the one or more fitting coefficients 612 best match or adapt the function 502 to the energy loss spectrum. In various cases, the fitting component 314 may be implemented in any suitable software package or platform (e.g., Quadprog The Python package quadratic programming can be used to facilitate such fitting.

[0083] In various embodiments, the fitting component 314 may apply a zero integral constraint to the fine structure terms 508 during such fitting. That is, the fitting component 314 may electronically calculate, determine, or otherwise identify values ​​for one or more fitting coefficients 604, one or more fitting coefficients 608, and one or more fitting coefficients 612 that not only ensure that the function 502 closely matches or resembles the energy loss spectrum of a given pixel, but also ensure that the integral associated with the fine structure terms 508 is as close to zero as possible (e.g., within an appropriate threshold of zero).

[0084] In some cases, such an integral can be applied to the fine structure term 508 itself. That is, the fitting component 314 can apply the following equation when fitting the function 502 to the energy loss spectrum of a given pixel:

number

[0085] However, in some cases, such an integral may be applied to a constant multiple of the fine structure term 508. That is, the fitting component 314 may apply the following equation when fitting the function 502 to the energy loss spectrum of a given pixel:

number

[0086] However, such integration may otherwise be applied to any suitable composite function that includes fine structure terms 508. That is, fitting component 314 may apply the following equation when fitting function 502 to the energy loss spectrum of a given pixel:

number

[0087] In some cases, E max Note that can be replaced by ∞ in any of the above integral constraints, and E0 can be replaced by 0 in any of the above integral constraints.

[0088] Bethe sum rule states that a system has momentum

number

number

number

number

number

number

number

number

number

number

number

[0089] In various cases, the above zero-integral constraint formulation may be applied to incident electrons described as plane waves. In various aspects, similar integral constraints may be derived for appropriate types and descriptions of incident electrons. As a non-limiting example, assume a convergent incident electron beam is used. In such cases, the following zero-integral constraint may be derived:

number

number

number

number

[0090] In either case, function 502 is an additive combination of background terms 504, atomic cross-section terms 506, and fine structure terms 508, and fitting component 314 can electronically fit each version or instance of function 502 to the energy spectrum of each pixel in the spectral image. As such, fitting component 314 can generate a total of different or unique fitted versions of function 502, one for each pixel in spectral image 304. Figures 7-13 show various non-limiting details regarding such fitting.

[0091] 7 represents a curve 702 as shown. In various cases, the curve 702 can be considered a plot of the energy loss spectrum measured or recorded for any given pixel of the spectral image 304. As shown, the independent variable of the curve 702 (e.g., the variable displayed on the horizontal axis) can be energy loss, and the dependent variable of the curve 702 (e.g., the variable displayed on the vertical axis) can be count or intensity. In various cases, as shown, a K-edge occurs with the curve 702 starting at or around an energy loss value of E. In various embodiments, it may be desirable to determine which particular values ​​of the one or more fitting coefficients 604, the one or more fitting coefficients 608, and the one or more fitting coefficients 612 match or fit the function 502 to the curve 702.

[0092] 8 illustrates a curve 802. In various embodiments, as described herein, the curve 802 can be viewed as a plot of the background term 504 after fitting a function 502 to the curve 702. As shown, the curve 802 is monotonically decreasing and therefore explains or accounts for a portion of the curve 702, but the curve 802 does not explain the K-end of the curve 702.

[0093] 9 next shows curve 902. In various embodiments, as described herein, curve 902 can be viewed as a plot of atomic cross-section area 506 after fitting function 502 to curve 702. As shown, curve 902 can exhibit a sharp increase in phase with the K-edge of curve 702, but curve 902 does not account for the most interesting or useful activity or behavior of the K-edge of curve 702.

[0094] Finally, Figure 10 illustrates curve 1002. In various aspects, as described herein, curve 1002 can be considered a plot of background term 508 after fitting function 502 to curve 702. As shown, curve 1002 can be considered to capture or describe the most interesting or useful activity or behavior of the K-end of curve 702. That is, curves 802 and 902 can be considered to capture the known or low-information portion of curve 702, and curve 1002 can be considered to capture the remaining portion of curve 702.

[0095] Figures 7-10 are illustrative reproductions that can clarify the various embodiments described herein, while Figures 11-13 show simulation data and actual experimental data obtained by the inventors.

[0096] 11 shows curve 1102 and curve 1104. Curve 1102 shows a computer-simulated energy loss spectrum. Curve 1104 shows the fitted, final, or resulting form of the fine structure term 508 after being fitted to curve 1102 as described herein.

[0097] In particular, Figure 12 shows curves 1202, 1204, 1206, and 1208. Curve 1202 shows a real-world energy loss spectrum recorded by a real-world electron energy loss microscope. Curve 1204 shows a simulated version of background term 504. Curve 1206 shows atomic cross-section term 506. Curve 1208 shows a simulated version of fine structure term 508.

[0098] As described herein, the fine structure term 508 can be integrally constrained to 0. In various embodiments, such an integral constraint can result in at least one portion of the fine structure term 508 being negative (e.g., less than 0) and at least another portion of the fine structure term 508 being positive (e.g., greater than 0). Without such an integral constraint, the fine structure term 508 can instead be entirely positive. This is shown in a non-limiting manner in FIG. 13.

[0099] In particular, FIG. 13 includes graph 1302 and graph 1304. Graph 1302 includes curve 1306, curve 1308, curve 1310, and curve 1312. Curve 1306 represents a real-world energy loss spectrum recorded by a real-world electron energy loss microscope of a silicon sample. That is, curve 1306 can be considered to represent the Si K-edge. Curve 1308 represents the fitted, final, or resulting form of background term 504 after being fitted to curve 1306 as described herein. Curve 1310 represents the fitted, final, or resulting form of background term 506 (for silicon atoms) after being fitted to curve 1306 as described herein. Curve 1312 represents the fitted, final, or resulting form of fine structure term 508 (e.g., a rectangular spline) after being fitted to curve 1306, although curve 1312 was generated without the zero-integral constraint described herein. As shown, all portions of curve 1312 are greater than or equal to 0. That is, no portion of curve 1312 is negative.

[0100] Here, graph 1304 includes curve 1306, curve 1314, curve 1316, and curve 1318. Curve 1314 illustrates an alternative fitted, final, or resulting form of background term 504 after being fitted to curve 1306, as described herein. Curve 1316 illustrates an alternative fitted, final, or resulting form of atomic cross-section term 506 (for silicon atoms) after being fitted to curve 1306, as described herein. Curve 1318 illustrates an alternative fitted, final, or resulting form of fine structure term 508 (e.g., a rectangular function spline) after being fitted to curve 1306, except that curve 1318 was generated using the zero-integral constraint described herein. As shown, application of such a zero-integral constraint causes the heights of curves 1314 and 1316 to differ from the heights of curves 1308 and 1310, and application of such a zero-integral constraint causes at least a portion of curve 1318 to be less than zero. This serves to demonstrate that application of the zero integral constraint, as described herein, can significantly alter the final, resulting, or adapted version of the fine structure term 508.

[0101] 7-13 are understood to be merely non-limiting examples, not necessarily drawn to scale. In some cases, without the integral constraints described herein, function 502 may be entirely or mostly positive, or entirely or mostly negative. In either case, when the integral constraints described herein are implemented, fine structure term 508 may be significantly different (indeed, much more accurate) than it would otherwise be.

[0102] In either case, the fitting component 314 can generate a total of s different or unique fitted versions of the function 502, one for each pixel in the spectral image.

[0103] FIG. 14 illustrates a block diagram of an example non-limiting system including a segmentation mask that facilitates spectral image analysis via integral constraint fitting, according to one or more embodiments described herein.

[0104] In various embodiments, the execution component 316 can electronically generate the segmentation mask 1402 based on an adapted pixel-by-pixel version of the fine structure terms 508 rather than based on the entire adapted pixel-by-pixel version of the function 502. A non-limiting embodiment is described with respect to FIG.

[0105] FIG. 15 illustrates an example, non-limiting block diagram of how the segmentation mask 1402 may be generated in accordance with one or more embodiments described herein.

[0106] As previously mentioned, in various embodiments, the fitting component 314 can generate a respective fitted version of the function 502 for each pixel of the spectral image 304. In various cases, each fitted version of the function 502 can be considered to constitute a respective fitted version of the fine structure term 508. In various cases, each fitted version of the fine structure term 508 can be a plurality of fitted fine structure terms 1502.

[0107] In particular, because spectral image 304 can include s pixels, the plurality of fitted fine structure terms 1502 can include s fitted fine structure terms, namely fitted fine structure term 1502(1) through fitted fine structure term 1502(s). In various embodiments, fitted fine structure term 1502(1) can be considered to be any version of fine structure term 508 calculated by fitting component 314 for pixel 402(1). That is, the fitting component 314 may calculate a first version of the background term 504 (e.g., an initial combination of particular values ​​for one or more fitting coefficients 604), a first version of the atomic cross-section term 506 (e.g., an initial combination of particular values ​​for one or more fitting coefficients 608), and a first version of the fine structure term 508 (e.g., an initial combination of particular values ​​for one or more fitting coefficients 612), all based on the energy loss spectrum 404 of the pixel 402(1). In various cases, each version of the fine structure term 508 may be a fitted fine structure term 1502(1). Accordingly, the fitted fine structure term 1502(1) may be considered to convey interesting or useful electronic activity present in the energy loss spectrum 404(1). Similarly, the fitted fine structure term 1502(s) may be considered to be any version of the fine structure term 508 that the fitting component 314 has calculated for the pixel 402(s). That is, the fitting component 314 may calculate the sth version of the background term 504. (e.g., the sth combination of particular values ​​of one or more fitting coefficients 604), the sth version of the atomic cross section term 506 (e.g., the sth combination of particular values ​​of one or more fitting coefficients 608), and the sth version of the fine structure term 508 (e.g., the sth combination of particular values ​​of one or more fitting coefficients 612), all in the energy loss spectrum 404(s) of pixel 402(s). The sth version of the fine structure term 508 can be calculated based on the energy loss spectrum 404(s). In various cases, the sth version of the fine structure term 508 can be the fitted fine structure term 1502(s). The fitted fine structure term 1502(s) can therefore be thought of as conveying interesting or useful electronic activity present in the energy loss spectrum 404(s).

[0108] In various embodiments, the execution component 316 can electronically generate the segmentation mask 1402 based on the plurality of fine structure terms 1502 .

[0109] As a non-limiting example, the implementation component 316 can apply a suitable unsupervised clustering technique, such as K-means in conjunction with principal component analysis (PCA), to the plurality of fine structure terms 1502. It should be understood that any data formatting or preprocessing steps can be performed before or after such clustering (e.g., normalizing the amplitude and energy axes of the plurality of fitted fine structure terms 1502 to unity, centering the plurality of fitted fine structure terms 1502 by subtracting an average count or average intensity). In either case, such clustering can cause the implementation component 316 to separate the plurality of fitted fine structure terms 1502 and the plurality of pixels 402 into two or more clusters, where the fitted fine structure terms within any given cluster can be considered to be more numerically similar to each other than to fitted fine structure terms in several different clusters. Because the fitted microstructure term for any given pixel is believed to convey interesting or useful electronic behavior of whatever portion of the analytical sample is represented by that pixel, any two pixels within the same cluster can be considered to exhibit the same or similar electronic behavior and represent portions of the analytical sample having the same or similar material composition. Conversely, any two pixels within different clusters can be considered to exhibit different electronic behavior and represent portions of the analytical sample having different material compositions. In various embodiments, each of the two or more clusters identified by the execution component 316 can be assigned a unique label (e.g., Cluster 1, Cluster 2, Cluster 3), and the segmentation mask 1402 can be a matrix indicating which cluster each pixel of the spectral image 304 is assigned to. For example, assume again that the plurality of pixels 402 are arranged in a linear array of x, y, where x, y ... In this case, the segmentation mask 1402 can be an array of xxy, where the (a, b) element of the segmentation mask can indicate the label of the cluster to which the (a, b)-th pixel of the plurality of pixels 402 is assigned for row a and column b.Thus, the segmentation mask 1402 can be thought of as a visual indication of which pixels in the spectral image 304 belong to the same or different clusters, and therefore are determined to be composed of the same or different materials or concentrations. Indeed, the fitted fine structure term for any given pixel can be thought of as indicating the electronic environment around the desired element at any location represented by that given pixel. That is, the fitted fine structure can indicate which chemical substance the desired element at the location represented by the given pixel is part of.

[0110] As another non-limiting example, rather than using unsupervised clustering, the execution component 316 can run a suitable machine learning model trained or configured to perform segmentation on the plurality of fine structure terms 1502. Upon such execution, the machine learning model can generate a segmentation mask 1402 as output.

[0111] In particular, the machine learning model may exhibit a suitable internal structure, such as a deep learning neural network structure. Indeed, in various cases, the machine learning model may include an input layer, one or more hidden layers, and an output layer. In various cases, such layers may be coupled by suitable interneuronal or inter-layer connections, such as forward connections, skip connections, or recurrent connections. Furthermore, in various cases, such layers may be suitable types of neural network layers with suitable learnable or trainable internal parameters. For example, such an input layer, one or more hidden layers, or output layer may all be a mixture layer, and its learnable or trainable parameters may be convolution kernels. As another example, any of such an input layer, one or more hidden layers, or output layer may be a dense layer, and its learnable or trainable parameters may be a weight matrix or a bias value. As yet another example, any of such an input layer, one or more hidden layers, or output layer may be a batch normalization layer, and its learnable or trainable parameters may be a shift factor or a scale factor. As yet another example, any of such input layers, one or more hidden layers, or output layers can be long short-term memory (LSTM) layers, whose learnable or trainable parameters can be input state weight matrices or hidden state weight matrices. Furthermore, in various cases, such layers can be any suitable type of neural network layer with suitable learnable or trainable internal parameters. For example, any of such input layers, one or more hidden layers, or output layers can be nonlinear layers, padding layers, pooling layers, or concatenation layers.

[0112] Regardless of its particular internal structure, the machine learning model can be trained (e.g., in a supervised manner using a ground truth segmentation mask, in an unsupervised manner without a ground truth segmentation mask, or in a reinforcement learning manner with an appropriate reward or obligation policy) to receive adapted fine structure terms as input and generate an output segmentation mask based on those adapted fine structure terms. Accordingly, the execution component 316 can execute the machine learning model on the adapted fine structure terms 1502, and such execution generates the segmentation mask 1402. More specifically, the execution component 316 can feed the adapted fine structure terms 1502 to an input layer of the machine learning model, and the adapted fine structure terms 1502 can complete a forward pass through one or more hidden layers of the machine learning model. The output layer of the machine learning model can compute the segmentation mask 1402 based on the activation maps or feature maps generated by the one or more hidden layers.

[0113] In either case, the segmentation mask 1402 can indicate which pixels in the spectral image 304 belong to which of two or more defined classes, which can be considered to represent different or unique material compositions.

[0114] In some embodiments, the execution component 316 may generate the segmentation mask 1402 based on the entire plurality of fitted fine structure terms 1502. However, in other embodiments, the execution component 316 may generate the segmentation mask 1402 based solely on the fitting coefficients of the plurality of fitted fine structure terms 1502. That is, each of the plurality of fine structure terms 1502 may be considered to have a respective value for one or more fitting coefficients 612, and the execution component 316 may perform clustering or run a machine learning model on the respective values ​​of the one or more fitting coefficients 612 (e.g., in such cases, the remaining portions of each of the plurality of fine structure terms 1502 may be ignored or discarded). The inventors have found that by clustering or otherwise analyzing the fitting coefficients of the plurality of fitted fine structure terms 1502, acceptable segmentation accuracy can be achieved in a shorter time and with fewer computational resource usage.

[0115] In various embodiments, the execution component 316 can electronically transmit the segmentation mask 1402 to a suitable computing device. In various cases, the execution component 316 can electronically render or display the segmentation mask 1402 on a suitable computer screen.

[0116] Although the implementation component 316 is primarily described herein as generating a segmentation mask 1402 for the spectral image 304, this is merely a non-limiting example for ease of explanation. In various other embodiments, the implementation component 316 can perform other suitable downstream analyses on the spectral image 304, which may be based on the plurality of fine structure terms 1502 (or fitting coefficients thereof, or a subset thereof) rather than the raw energy spectrum of the spectral image 304. As a non-limiting example, the implementation component 316 can classify the spectral image 304 based on the plurality of fitted fine structure terms 1502. As another non-limiting example, the implementation component 316 can explicitly calculate the approximate material composition of the analyzed sample shown in the spectral image 304 based on the plurality of fitted fine structure terms 1502. Various other non-limiting examples of such downstream analyses include phase analysis, mobility mapping, oxidation state mapping, or the like.

[0117] To demonstrate the various technical advantages of the various embodiments described herein, the inventors have conducted various experiments, some of the results of which are shown in Figures 16-23.

[0118] FIG. 16 shows a block diagram of sample 1600 analyzed in accordance with various embodiments described herein. Sample 1600 is composed of four regions: region 1602, region 1604, region 1606, and region 1608. Region 1602 is composed of Al4O6. Region 1604 is composed of Cu4O2. Region 1606 is composed of SiO4. Region 1608 is composed of Sr1Ti1O3. Furthermore, the thickness of these four regions is continuously varied, increasing from 0.2 to 1.5 times the mean free path in 25 steps. A spectral image of sample 1600 was simulated, and the energy loss spectrum of the spectral image was fitted with the microstructure described herein.

[0119] FIG. 17 shows a segmentation mask 1700 obtained from the adapted microstructure described herein. In particular, the segmentation mask 1700 was obtained by K-means. The different shades of color in FIG. 17 can be considered to indicate different material compositions. As shown, the segmentation mask 1700 indicates with great accuracy which portions of the sample 1600 are composed of the same material as each other. In fact, only two pixels (e.g., one in region 1602 and one in region 1606) were misclassified or missegmented.

[0120] Figure 18 shows a scatter plot 1800 generated via PCA from the data used for the K-means algorithm described above. As shown, the resulting fitted microstructures were clearly separated into four distinct dense clusters (e.g., one cluster per material composition).

[0121] Next, FIG. 19 shows a segmentation mask 1900 obtained solely from the final or resulting fitting coefficients of the adapted microstructure. As noted above, the segmentation mask 1900 was obtained by K-means, and the different shades of color in FIG. 19 can be considered to indicate different material compositions. As shown, the segmentation mask 1900 indicates, with slightly lower but acceptable accuracy, which portions of the feed 1600 are composed of the same material as each other. In fact, more pixels were misclassified or missegmented compared to FIG. 17. However, the accuracy level of the displayed segments is still sufficient.

[0122] FIG. 20 shows a scatter plot 2000 generated via PCA from the data used for the K-means algorithm described above. As shown, the final or resulting fitting coefficients of the resulting fitted microstructure exhibit a slightly reduced cluster density compared to FIG. 18, but are still clearly separated into four distinct clusters (e.g., one cluster per material composition) overall. Note that clustering only the coefficients of the fitted microstructure terms requires less time and computational resources (e.g., memory and processing power) than clustering the entire fitted microstructure term set. However, as these experimental results demonstrate, acceptable segmentation accuracy can be achieved despite the reduced consumption of time and computational resources.

[0123] Next, FIG. 21 illustrates a segmentation mask 2100 obtained from a spectral image of sample 1600 using existing techniques. Notably, such existing techniques did not include the integrally constrained microstructure fitting described herein. As noted above, segmentation mask 2100 was obtained using K-means, and the different shades of color shown in FIG. 21 can be considered to indicate different material compositions. As noted above, segmentation mask 2100 was obtained using K-means, and the different shades of color shown in FIG. 21 can be considered to indicate different material compositions. As shown, segmentation mask 2100 has very low accuracy (e.g., it reliably indicates which portions of sample 1600 are composed of the same material as one another). In fact, segmentation mask 2100 indicates that existing techniques are significantly hindered or otherwise confounded by the differences in material composition and thickness throughout sample 1600.

[0124] Figure 22 shows a scatter plot 2200 generated via PCA from the data used in the K-means algorithm described above. As can be seen, the fitted microstructures obtained with existing techniques did not accurately separate into four distinct clusters. This again highlights the inability of existing techniques to handle the compositional and geometric diversity of sample 1600.

[0125] FIG. 23 shows graphs 2302, 2304, and 2306 obtained during various additional experiments conducted by the inventors.

[0126] Specifically, while keeping the irradiation area constant, amorphous Si3N4 A 5000 energy loss spectrum was obtained at the Si K-edge of the sample. A collimated beam was used to maximize the illuminated area and minimize electron beam damage and contamination. In some cases, the 5000 energy loss spectrum was fitted using the zero-integral constraint described herein. In other cases, the 5000 energy loss spectrum was fitted instead without the zero-integral constraint described herein. In both cases, the fine structure term 508 was calculated using 125 rectangular functions (e.g., h) with ΔE=4 eV. max =125), generating a 500 eV energy window. We found that when element abundances were estimated using a combination of atomic cross-section fitting coefficients and integrally unconstrained fine structure, the resulting abundance values ​​(shown by numeral 2308 in graph 2302) were widely dispersed. In contrast, when element abundances were estimated using atomic cross-section fitting coefficients and integrally constrained fine structure, the resulting abundance values ​​(shown by numeral 2310 in graph 2302) were much denser. Thus, we found that integrally unconstrained fine structure enabled more accurate element abundance measurements. This is because function 502 has a large number of fitting coefficients (e.g., 604, 608, 612), for which specific values ​​must be determined. This large number of parameters can lead to overfitting and noise sensitivity. However, we believe that the zero-integral constraint described herein helps reduce the likelihood of such overfitting and noise sensitivity.

[0127] We conducted additional tests to investigate how the precision and accuracy of element abundances change with the energy window. In these tests, we used the same 5,000 energy loss spectra, but varied the energy window from a low of 60 eV to a high of 540 eV in 60 eV increments. Again, in some cases, the energy loss spectra were fitted using the integral constraints described herein to obtain element abundance results. In other cases, no integral constraints were used. Graph 2304 shows the average abundance of silicon as a function of energy window, both with integral constraints and without integral constraints. As shown, the average determined abundance in the unconstrained case changes significantly (e.g., a 25% decrease) as the energy window is changed. In contrast, the average determined abundance in the constrained case changes only slightly (e.g., a 6% decrease) as the energy window is changed. Graph 2306 shows the standard deviation of silicon as a function of energy window, both with integral constraints and without integral constraints. As shown, the standard deviation in the unconstrained case increases by nearly four times as the energy window is changed. In contrast, the standard deviation with the constraint increases slightly as the energy window is changed. These experimental results demonstrate that the zero-integral constraint described herein can significantly improve both the accuracy and precision of downstream analysis of energy loss spectra.

[0128] Therefore, various embodiments described herein can be considered to facilitate improved spectral image analysis through the implementation of integral constraint fitting of fine structure.

[0129] The scientific device systems, methods, or techniques disclosed herein may involve interaction with a human user (e.g., via a user local computing device 2620 described herein with reference to FIG. 26 ). These interactions may include providing information to the user (e.g., information regarding the operation of a scientific device such as scientific device 2610 of FIG. 26 , information regarding a sample being analyzed or other tests or measurements being performed by the scientific device, information obtained from a local or remote database, or other information), or providing the user with the option to enter commands (e.g., to control the operation of a scientific device such as scientific device 2610 of FIG. 26 or to control the analysis of data generated by the scientific device), queries (e.g., directed to a local or remote database), or other information. In some embodiments, these interactions may be performed via a graphical user interface (GUI) that includes a visual display on a display device (e.g., display device 2510 described herein with reference to FIG. 25) that provides output to the user and / or prompts the user to provide input (e.g., via one or more input devices such as a keyboard, mouse, trackpad, or touchscreen included in other I / O devices 2512 described herein with reference to FIG. 25). The scientific device systems, methods, or techniques disclosed herein may include any suitable GUI for interaction with a user.

[0130] 24 illustrates an exemplary graphical user interface 2400 (hereinafter, "GUI 2400") that may be used in implementing some or all of the assistance, methods, or techniques disclosed herein, according to various embodiments. In various aspects, GUI 2400 may be provided on a display device (e.g., display device 2600 described herein with reference to FIG. 26) of a computing device (e.g., computing device 2500 described herein with reference to FIG. 25) of a scientific device display (e.g., scientific device system 2510 described herein with reference to FIG. 25), and a user or technician may interact with GUI 2400 using any suitable input device (e.g., any of the input devices included in other I / O devices 2512 described herein with reference to FIG. 25) and input technology (e.g., cursor movement, motion capture, facial recognition, gesture detection, voice recognition, button activation, etc.).

[0131] GUI 2400 may include a data display area 2402, a data analysis area 2404, a scientific device control area 2406, and a settings area 2408. The particular number and arrangement of areas shown in Figure 24 is merely exemplary, and any number and arrangement of areas containing any desired features may be included in GUI 2400.

[0132] The data display area 2402 is capable of displaying data generated by a scientific device (eg, scientific device 2610 described herein with reference to FIG. 26).

[0133] The data analysis area 2404 may display the results of a data analysis (e.g., the results of analyzing the data shown in the data display area 2402 or other data). In some embodiments, the data display area 2402 and the data analysis area 2404 may be combined in the GUI 2400 (e.g., to include data output from a scientific device and an analysis of some of the data in a common graph or area).

[0134] The scientific device control area 2406 may include options that allow a user or technician to control a scientific device (e.g., scientific device 2610 described herein with reference to FIG. 26). For example, the scientific device control area 2406 may include configurable parameters that govern the operation of such a scientific device (e.g., a configurable parameter that controls the voltage or current of a scientific device, a configurable parameter that controls the internal temperature of a scientific device, or a configurable parameter that controls the flow rate of a fluid in a scientific device).

[0135] Settings area 2408 may include options that enable a user or technician to control the features and functionality of GUI 2400 (or other GUIs) or perform common computing operations related to data display area 2402 and data analysis area 2404 (e.g., saving data on a storage device, such as storage device 2504 described herein with reference to FIG. 25 , sending data to another user, labeling data, etc.).

[0136] As noted above, the scientific device module 102 may be implemented by one or more computing devices. Figure 25 is a block diagram of a computing device 2500 that may implement some or all of the scientific device methods disclosed herein, according to various embodiments. In some embodiments, the scientific device module 102 may be implemented by a single computing device 2500 or by multiple computing devices 2500. Furthermore, as discussed below, the computing device 2500 (or multiple instances) that implements the scientific device module 102 may be part of one or more of the scientific device 2610, user local computing device 2620, service local computing device 2630, or remote computing device 2640 of Figure 26.

[0137] While computing device 2500 is illustrated as having several components, any one or more of these components may be omitted or duplicated as appropriate for the application and setting. In some embodiments, some or all of the components included in computing device 2500 may be mounted on one or more motherboards and enclosed in a housing (e.g., comprising plastic, metal, or other material). In some embodiments, these components may be fabricated on a single system-on-chip (SoC) (e.g., an SoC may include one or more processing devices 2502 and one or more storage devices 2504). Additionally, in various embodiments, computing device 2500 may not include one or more of the components illustrated in FIG. 25, but may include interface circuitry (not shown) for coupling to one or more components using any suitable interface (e.g., a universal serial bus (USB) interface, a high-definition multimedia interface (HDMI) interface, a controller area network (CAN) interface, a serial peripheral interface (SPI) interface, an Ethernet interface, a wireless interface, or any other suitable interface). For example, computing device 2500 may omit display device 2510, but may include display device interface circuitry (eg, connectors and driver circuitry) to allow display device 2510 to be coupled.

[0138] Computing device 2500 may include a processing device 2502 (e.g., one or more processing devices). As used herein, the term "processing device" may refer to any device or portion of a device that processes electronic data from registers or memory and converts the electronic data into other electronic data, which may be stored in registers or memory. Processing device 2502 may include one or more digital signal processors (DSPs), application specific integrated circuits (ASICs), central processing units (CPUs), graphics processing units (GPUs), cryptographic processors (dedicated processors that execute cryptographic algorithms in hardware), server processors, or any other suitable processing devices.

[0139] The computing device 2500 also includes a storage device 2504 (e.g., one or more storage devices). The storage device 2504 may include one or more memory devices, such as random access memory (RAM) (e.g., static RAM (SRAM) devices, magnetic RAM (MRAM) devices, dynamic RAM (DRAM) devices, resistive RAM (RRAM) devices, or conductive bridge RAM (CBRAM) devices), hard drive-based memory devices, solid state memory devices, network drives, cloud drives, or any combination of memory devices. In some embodiments, the storage device 2504 may include memory that shares a die with the processing device 2502. In such embodiments, the memory may be used as cache memory and may include, for example, embedded dynamic random access memory (eDRAM) or spin transfer torque magnetic random access memory (STT-MRAM). In some embodiments, storage device 2504 may include a non-transitory computer-readable medium having instructions that, when executed by one or more processing devices (e.g., processing device 2502), cause computing device 2500 to perform any suitable of the methods or portions of those methods disclosed herein.

[0140] The computing device 2500 may include an interface device 2506 (e.g., one or more interface devices 2506). The interface device 2506 may include one or more communication chips, connectors, or other hardware and software to manage communications between the computing device 2500 and other computing devices. For example, the interface device 2506 may include circuitry to manage wireless communications for transferring data to and from the computing device 2500. The term "wireless" and its derivatives may be used to describe circuits, devices, systems, methods, techniques, communication channels, etc. that may communicate data through the use of modulated electromagnetic radiation over a non-solid medium. This term does not imply that the associated devices do not include any wiring, although in some embodiments they may not. The circuitry included in interface device 2506 for managing wireless communications may implement any of several wireless standards or protocols, including, but not limited to, Wi-Fi (IEEE 802.11 family), Institute of Electrical and Electronics Engineers (IEEE) standards including the IEEE 802.16 standard (e.g., the IEEE 802.16-2005 Amendment), the Long Term Evolution (LTE) project with any amendments, updates, and / or revisions (e.g., the Advanced LTE project, the Ultra Mobile Broadband (UMB) project (also referred to as "3GPP®2"), etc.). In some embodiments, the circuitry included in interface device 2506 for managing wireless communications may operate in accordance with a Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Universal Mobile Telecommunications System (UMTS), High Speed ​​Packet Access (HSPA), Evolved HSPA (E-HSPA), or LTE network.In some embodiments, the circuitry included in the interface device 2506 for managing wireless communications may operate in accordance with GSM Evolution High-Speed ​​Data (EDGE), GSM EDGE Radio Access Network (GERAN), Universal Terrestrial Radio Access Network (UTRAN), or Evolved UTRAM (E-UTRAN). In some embodiments, the circuitry included in the interface device 2506 for managing wireless communications may operate in accordance with Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Digital Enhanced Cordless Communications (DECT), Evolution Data Optimized (EV-DO), and derivatives thereof, as well as any other wireless protocols designated as 3G, 4G, 5G, and beyond. In some embodiments, the interface device 2506 may include one or more antennas (e.g., one or more antenna arrays) for receiving and / or transmitting wireless communications.

[0141] In some embodiments, interface device 2506 may include circuitry for managing wired communications, such as electrical, optical, or any other suitable communications protocol. For example, interface device 2506 may include circuitry to support communications according to Ethernet technology. In some embodiments, interface device 2506 may support both wireless and wired communications, or may support multiple wired or wireless communications protocols. For example, a first set of circuits in interface device 2506 may be dedicated to short-range wireless communications, such as Wi-Fi or Bluetooth, and a second set of circuits in interface device 2506 may be dedicated to long-range wireless communications, such as Global Positioning System (GPS), EDGE, GPRS, CDMA, WiMAX, LTE, EV-DO, etc. In some embodiments, a first set of circuits in interface device 2506 may be dedicated to wireless communications, and a second set of circuits in interface device 2506 may be dedicated to wired communications.

[0142] Computing device 2500 may include battery / power circuitry 2508. Battery / power circuitry 2508 may include one or more energy storage devices (e.g., batteries or capacitors) or circuitry for coupling components of computing device 2500 to an energy source separate from computing device 2500 (e.g., AC line power).

[0143] Computing device 2500 may include a display device 2510 (e.g., multiple display devices). Display device 2510 may include any visual indicator, such as a heads-up display, a computer monitor, a projector, a touchscreen display, a liquid crystal display (LCD), a light-emitting diode display, or a flat panel display.

[0144] Computing device 2500 may include other input / output (I / O) devices 2512. The other I / O devices 2512 may include, for example, one or more audio output devices (e.g., speakers, headsets, earphones, alarms, etc.), one or more audio input devices (e.g., microphones or microphone arrays), a location device (e.g., a GPS device that communicates with a satellite-based system to receive the location of computing device 2500), an audio codec, a video codec, a printer, sensors (e.g., thermocouples or other temperature sensors, humidity sensors, pressure sensors, vibration sensors, accelerometers, gyroscopes, etc.), an image capture device such as a camera, a keyboard, a cursor control device (e.g., a mouse, stylus, trackball, or touchpad), a barcode reader, a quick response (QR) code reader, or a radio frequency identification (RFID) reader.

[0145] Computing device 2500 may have any suitable form factor for the application and configuration, such as a handheld or mobile computing device (e.g., a cell phone, a smartphone, a mobile Internet device, a tablet computer, a laptop computer, a netbook computer, an ultrabook computer, a personal digital assistant (PDA), an ultra-mobile personal computer, etc.), a desktop or server computing device, or other network computing component.

[0146] One or more computing devices implementing any of the scientific device assistance modules or methods disclosed herein may be part of a scientific device assistance system. Figure 26 is a block diagram of an example scientific device assistance system 2600 capable of performing some or all of the scientific device assistance methods disclosed herein, according to various embodiments. The scientific device assistance modules, methods, and techniques disclosed herein (e.g., scientific device assistance module 102, computer-implemented method 200, system 306) may be implemented by one or more of the scientific devices 2610, user local computing device 2620, service local computing device 2630, or remote computing device 2640 of the scientific device assistance system 2600.

[0147] Any of the scientific device 2610, the user local computing device 2620, the service local computing device 2630, or the remote computing device 2640 can take the form of any suitable embodiment of the computing device 2500, and any of the scientific device 2610, the user local computing device 2620, the service local computing device 2630, or the remote computing device 2640 can take the form of any suitable embodiment of the computing device 2500.

[0148] Each of the scientific device 2610, the user local computing device 2620, the service local computing device 2630, or the remote computing device 2640 may include a processing device 2602, a storage device 2604, and an interface device 2606. The processing device 2602 may take any suitable form, including any form of the processing device 2502, and the processing devices 2602 included in different ones of the scientific device 2610, the user local computing device 2620, the service local computing device 2630, or the remote computing device 2640 may take the same form or different forms. The storage device 2604 may take any suitable form, including any form of the interface device 2504, and the interface devices 2604 included in different ones of the scientific device 2610, the user local computing device 2620, the service local computing device 2630, or the remote computing device 2640 may take the same form or different forms. The interface device 2606 may take any suitable form, including any form of interface device 2506, and the interface devices 2606 included in different ones of the scientific device 2610, the user local computing device 2620, the service local computing device 2630, or the remote computing device 2640 may take the same or different forms.

[0149] The scientific devices 2610, the user local computing device 2620, the service local computing device 2630, and the remote computing device 2640 may communicate with other elements of the scientific device system 2600 via communication paths 2608. The communication paths 2608 may be communicatively coupled to the interface devices 2606 of the various elements of the scientific device support system 2600, as shown, and may be wired or wireless communication paths (e.g., according to any of the communication techniques described herein with reference to the interface device 2606). While the particular scientific device support system 2600 shown in FIG. 26 includes communication paths between each pair of the scientific devices 2610, the user local computing device 2620, the service local computing device 2630, and the remote computing device 2640, this “fully connected” implementation is merely exemplary, and in various embodiments, various ones of the communication paths 2608 may not be present. For example, in some embodiments, the service local computing device 2630 may not have a direct communication path 2608 between its interface device 2606 and the interface device 2606 of the scientific device 2610, but instead may communicate with the scientific device 2610 via the communication path 2608 between the service local computing device 2630 and the user local computing device 2620, and the communication path 2608 between the user local computing device 2620 and the scientific device 2610.

[0150] Scientific devices 2610 may include any suitable scientific devices, such as scientific devices 302 .

[0151] The user local computing device 2620 may be a computing device that is local to the user of the scientific device 2610 (e.g., according to any of the embodiments of the computing device 2500 described herein). In some embodiments, the user local computing device 2620 may also be local to the scientific device 2610, but need not be. For example, a user local computing device 2620 in a user's home or office may be remote from, but in communication with, the scientific device 2610 such that the user may use the user local computing device 2620 to control or access data from the scientific device 2610. In some embodiments, the user local computing device 2620 may be a laptop, smartphone, or tablet device.

[0152] The servicing local computing device 2630 may be a computing device that is local to the entity that provides services to the scientific device 2610 (e.g., according to any of the embodiments of computing device 2500 described herein). For example, the servicing local computing device 2630 may be local to the manufacturer of the scientific device 2610 or to a third-party service company. In some embodiments, the servicing local computing device 2630 may communicate with the scientific device 2610, the user local computing device 2620, or the remote computing device 2640 (e.g., via a direct communication path 2608 or via multiple “indirect” communication paths 2608, as previously discussed) to receive data regarding the operation of the scientific device 2610, the user local computing device 2620, or the remote computing device 2640 (e.g., results of self-diagnostic tests of the scientific device 2610, calibration coefficients used by the scientific device 2610, measurements of sensors associated with the scientific device 2610, etc.). In some embodiments, the service local computing device 2630 may communicate with the scientific devices 2610, the user local computing devices 2620, and / or the remote computing devices 2640 (e.g., via a direct communication path 2608 or multiple "indirect" communication paths 2608, as previously discussed) to transmit data to the scientific devices 2610, the user local computing devices 2620, or the remote computing devices 2640 (e.g., to update programmed instructions such as firmware in the scientific devices 2610, to initiate the performance of a test or calibration sequence in the scientific devices 2610, to update programmed instructions such as software in the user local computing devices 2620 or the remote computing devices 2640, etc.).A user of the scientific device 2610 may communicate with the service local computing device 2630 and utilize the scientific device 2610 or the user local computing device 2620 to report problems with the scientific device 2610 or the user local computing device 2620, to request a technician visit to improve the operation of the scientific device 2610, to order consumables or replacement parts associated with the scientific device 2610, or for other purposes.

[0153] The remote computing device 2640 may be a computing device (e.g., according to any of the embodiments of computing device 2500 described herein) that is located remotely from the scientific device 2610 or the user local computing device 2620. In some embodiments, the remote computing device 2640 may be included in a data center or other large-scale server environment. In some embodiments, the remote computing device 2640 may include network-attached storage (e.g., as part of the storage device 2604). The remote computing device 2640 may store data generated by the scientific device 2610, perform analysis of the data generated by the scientific device 2610 (e.g., according to programmed instructions), facilitate communications between the user local computing device 2620 and the scientific device 2610, and facilitate communications between the service local computing device 2630 and the scientific device 2610.

[0154] In some embodiments, one or more of the elements of the scientific device support system 2600 illustrated in Figure 26 may not be present. Furthermore, in some embodiments, multiples of various of the elements of the scientific device support system 2600 of Figure 26 may be present. For example, the scientific device support system 2600 may include multiple user local computing devices 2620 (e.g., different user local computing devices 2620 associated with different users or in different locations). In another example, the scientific device support system 2600 may include multiple scientific devices 2610 in communication with a service local computing device 2630 and / or a remote computing device 2640. In such an embodiment, the service local computing device 2630 may monitor these multiple CPMs 2610, and the service local computing device 2630 may "broadcast" updates or other information to the multiple scientific devices 2610 simultaneously. The different scientific devices 2610 in the scientific device system 2600 may be located near each other (e.g., in the same room) or far from each other (e.g., different floors of a building, different buildings, different cities, etc.). In some embodiments, the scientific device 2610 may be connected to an Internet of Things (IoT) stack that allows for command and control of the scientific device 2610 through web-based applications, virtual or augmented reality applications, mobile applications, or desktop applications. Any of these applications may be accessed by a user operating a user local computing device 2620 that is in communication with the scientific device 2610 through an intervening remote computing device 2640. In some embodiments, the scientific device 2610 may be sold by a manufacturer along with one or more associated user local computing devices 2620 as part of a scientific device computing unit 2612.

[0155] In some embodiments, different ones of the scientific devices 2610 included in the scientific device support system 2600 may be different types of scientific devices 2610. For example, one scientific device 2610 may be a mass spectrometer, while another scientific device 2610 may be a chromatograph or an autosampler. In some such embodiments, the remote computing device 2640 or the user local computing device 2620 may combine data from the different types of scientific devices 2610 included in the scientific device system 2600.

[0156] In various cases, machine learning algorithms or models may be implemented in any suitable manner to facilitate appropriate aspects described herein. To facilitate some of the above machine learning aspects of various embodiments, consider the following discussion of artificial intelligence (AI). Various embodiments described herein may employ artificial functions to facilitate automation of one or more functions or functionalities. Components may employ various AI-based schemes to implement various embodiments / examples disclosed herein. To provide or support many of the decisions described herein (e.g., measuring, ascertaining, inferring, calculating, predicting, predicting, deriving, foreseeing, detecting, calculating), components described herein may examine all or a subset of the data to which they are granted access and may infer or determine the state of a system or environment from a series of observations obtained through events or data. Decisions may be used to identify specific situations or actions, e.g., to generate probability distributions between states. Decisions may be probabilistic, i.e., calculating a probability distribution over states of interest based on a consideration of data and events. Decisions may also refer to techniques used to compose higher-level events from a series of events or data.

[0157] Such decisions may generate new events and actions from a series of observed events or stored event data, regardless of whether those events are closely related in time and whether the events and data are derived from one or more event and data sources. The components disclosed herein relate to a variety of classification (explicitly trained (e.g., via training data) and implicitly trained (e.g., via behavioral observation, preferences, historical information, receipt of external information, etc.)) schemes or systems (e.g., support vector systems, neural networks, expert systems, Bayesian belief networks, fuzzy logic, data fusion engines, etc.) that perform automated or determined actions related to the claimed subject matter. Thus, classification schemes or systems can be used to automatically learn and perform many functions, evaluations, or decisions.

[0158] The classifier takes an input attribute vector, z = (z1, z2, z3, z4, z n ) can be mapped to a confidence that the input belongs to a class, such as f(z) = confidence(class). Such classification can use probabilistic or statistical analysis (e.g., considerations of analytical utility and cost) to determine the action to be taken automatically. A support vector machine (SVM) can be one example of a classifier that can be used. SVMs operate by finding a hypersurface in the space of possible inputs, which attempts to separate triggering criteria from non-triggering events. Intuitively, this results in correct classification of training data that is close but not identical to the training data. Other direct and indirect model classification approaches can be employed, including, for example, Naive Bayes, Bayesian networks, decision trees, neural networks, fuzzy logic models, or probabilistic classification models that offer various independence patterns. As used herein, classification also includes statistical regression, which is used to develop priority models.

[0159] To provide additional context for the various embodiments described herein, Figure 27 and the following discussion are intended to provide a brief, general description of a suitable computing environment 2700 in which various embodiments of the embodiments described herein may be implemented. While the embodiments have been described in the general context of computer-executable instructions that may be executed on one or more computers, those skilled in the art will recognize that the embodiments may also be implemented in combination with other program modules or as a combination of hardware and software.

[0160] Generally, program modules include routines, programs, components, data structures, etc. that perform particular tasks or implement particular abstract data types. Furthermore, those skilled in the art will appreciate that the methods of the present invention may be used in a variety of applications, including single-processor or multi-processor computer systems, minicomputers, mainframe computers, Internet of Things (IoT) devices, distributed computing systems, as well as personal computers, portable computing devices, microprocessor-based electronic devices, programmable consumer electronic devices, and the like, each operatively connected to one or more associated devices.

[0161] The illustrated embodiments of the present disclosure may also be practiced in distributed computing environments where certain tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.

[0162] A computing device typically includes a variety of media, which may include computer-readable storage media, machine-readable storage media, or communication media, and these two terms are used interchangeably herein as follows: A computer-readable storage medium or machine-readable storage medium may be any available storage medium that can be accessed by a computer, and includes both volatile and nonvolatile media, removable and non-removable media. By way of example and not limitation, a computer-readable storage medium or machine-readable storage medium may be implemented in connection with any method or technology for storing information, such as computer-readable or machine-readable instructions, program modules, structured data, or unstructured data.

[0163] A computer-readable storage medium may include, but is not limited to, random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD), Blu-ray disc (BD) or other optical disc storage device, magnetic cassette, magnetic tape, magnetic disc storage device or other magnetic storage device, solid-state drive or other solid-state storage device, or other tangible or non-transitory medium that can be used to store the desired information. In this regard, the terms "tangible" or "non-transitory" as applied to storage, memory, or computer-readable medium herein exclude only the propagating transitory signal itself as a modifier, and do not waive the right to all standard storage, memory, or computer-readable medium that do not merely propagate the transitory signal itself.

[0164] The computer-readable storage medium can be accessed by one or more local or remote computing devices, for example, via an access request, query, or other data retrieval protocol, and various operations can be performed on the information stored thereon.

[0165] Communication media typically embodies computer-readable instructions, data structures, program modules, or other structured or unstructured data in a data signal, such as a modulated data signal, such as a carrier wave or other transport mechanism, and includes any information delivery or transmission media. The term "modulated data signal" means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media.

[0166] 27, an exemplary environment 2700 for implementing various embodiments of the aspects described herein includes a computer 2702 that includes a computer 2702, a processing unit 2704, a system memory 2706, and a system bus 2708. The system bus 2708 couples system components including, but not limited to, the system memory 2706 to the processing unit 2704. The processing unit 2704 can be any of a variety of commercially available processors. Dual microprocessors and other multi-processor architectures can also be used as the processing unit 2704.

[0167] The system bus 2708 may be any of several types of bus structures that may be further interconnected to a memory bus (with or without a memory controller), a peripheral bus, and a local bus using any of a variety of commercially available bus architectures. The system memory 2706 includes ROM 2710 and RAM 2712. The basic input / output system (BIOS) may be stored in non-volatile memory such as ROM, erasable programmable read-only memory (EPROM), or EEPROM, and contains the basic routines that help to transfer information between elements within the computer 2702, such as during start-up. The RAM 2712 may also include high-speed RAM, such as static RAM for caching data.

[0168] Computer 2702 further includes an internal hard disk (HDD) 2714 (e.g., EIDE, SATA), one or more external storage devices 2716 (e.g., a magnetic floppy disk drive (FDD) 2716, a memory stick or flash drive reader, a memory card reader, etc.), and a drive 2720 capable of reading from or writing to a disk 2722, such as a CD-ROM disk, DVD, BD, etc., e.g., a solid-state drive, optical disk drive, etc. Alternatively, if a solid-state drive is included, disk 2722 is not included unless it is separate. While internal HDD 2714 is shown as being located within computer 2702, internal HDD 2714 can also be configured for external use in a suitable chassis (not shown). Additionally, although not shown in environment 2700, a solid-state drive (SSD) can be used in addition to or in place of HDD 2714. HDD 2714, external storage device 2716, and drive 2720 can be connected to system bus 2708 by HDD interface 2724, external storage interface 2726, and drive interface 2728, respectively. Interface 2724 for external drive implementations can include at least one or both of Universal Serial Bus (USB) and Institute of Electrical and Electronics Engineers (IEEE) 1394 interface technologies. Other external drive connection technologies are within the contemplation of the embodiments described herein.

[0169] The drives and their associated computer-readable storage media provide non-volatile storage of data, data structures, computer-executable instructions, etc. In the case of computer 2702, the drives and storage media accommodate the storage of any data in a suitable digital format. While the above description of computer-readable storage media refers to each type of storage device, those skilled in the art will recognize that other types of computer-readable storage media, whether now existing or later developed, can be used in the exemplary operating environment, and further, any such storage media can include computer-executable instructions for performing the methods described herein.

[0170] The drives and RAM 2712 can store a number of program modules, including an operating system 2730, one or more application programs 2732, other program modules 2734, and program data 2736. All or portions of the operating system, applications, modules, or data can also be cached in RAM 2712. The systems and methods described herein can be implemented using various commercially available operating systems or combinations of operating systems.

[0171] Computer 2702 may optionally include emulation technology. For example, a hypervisor (not shown) or other intermediary may emulate the hardware environment of operating system 2730, where the emulated hardware may optionally differ from the hardware shown in FIG. 27 . In such an embodiment, operating system 2730 may comprise one of multiple virtual machines (VMs) hosted on computer 2702. Additionally, operating system 2730 may provide a runtime environment, such as the Java Runtime Environment or the .NET Framework, for application 2732. A runtime environment is a consistent execution environment that allows application 2732 to run on any operating system that includes the runtime environment. Similarly, operating system 2730 may support containers, where application 2732 may be in the form of a container, which is a lightweight, standalone, executable package of software that includes, for example, code, runtime, system tools, system libraries, and the application's configuration.

[0172] Additionally, computer 2702 can be enabled with a security module such as a Trusted Processing Module (TPM), where, for example, a boot component hashes the boot component in time and waits for the result to match a secure value before loading the next boot component.

[0173] This process can occur at any layer in the code execution stack of computer 2702, for example, applied at the application execution level or the operating system (OS) kernel level, thereby enabling security at any level of code execution. A user can input instructions and information into computer 2702 via one or more wired / wireless input devices, such as a keyboard 2738, a touchscreen 2740, or a pointing device such as a mouse 2742. Other input devices (not shown) can include a microphone, an infrared (IR) remote control, a radio frequency (RF) remote control, or other remote control, a joystick, a virtual reality controller or headset, a gamepad, a stylus pen, an image input device such as a camera, a gesture sensor input device, a visual motion sensor input device, an emotion or face detection device, a biometric input device such as a fingerprint or iris scanner, etc. These and other input devices are often connected to the processing unit 2704 via an input device interface 2744, which may be coupled to the system bus 2708, but may also be connected by other interfaces, such as a parallel port, an IEEE 1394 serial port, a game port, a USB port, an IR interface, a BLUETOOTH interface, etc.

[0174] A monitor 2746 or other type of display device can also be connected to the system bus 2708 via an interface, such as a video adapter 2748. In addition to the monitor 2746, a computer typically includes other peripheral output devices (not shown), such as speakers, printers, etc.

[0175] The computer 2702 can operate in a networked environment using logical connections via wired or wireless communications to one or more remote computers, such as a remote computer 2750. The remote computer 2750 can be a workstation, a server computer, a router, a personal computer, a portable computer, a microprocessor-based entertainment device, a peer device, or other common network node, and typically includes many or all of the elements described relative to the computer 2702, although for simplicity, only a memory / storage device 2752 is shown. The logical connections shown include wired and wireless connections to a local area network (LAN) 2754 or a wide area network (e.g., a wide area network (WAN) 2756). Such LAN and WAN networking environments are commonplace in offices and businesses and facilitate enterprise-wide computer networks, such as intranets, all of which can connect to a global communications network, e.g., the Internet.

[0176] When used in a LAN networking environment, the computer 2702 can be connected to the local network 2754 through a wired or wireless communication network interface or adapter 2758. The adapter 2758 can facilitate wired or wireless communication to the LAN 2754, and the LAN can also include a wireless access point (AP) disposed thereon for communicating with the adapter 2758 in a wireless mode.

[0177] When used in a WAN networking environment, the computer 2702 may include a modem 2760 or may be connected to a communications server on the WAN 2756 via other means for establishing communications over the WAN 2756, such as the Internet. The modem 2760 may be internal or external and a wired or wireless device and may be connected to the system bus 2708 via the input device interface 2744. In a networked environment, program modules depicted relative to the computer 2702, or portions thereof, may be stored in the remote memory / storage device 2752. It will be appreciated that the network connections shown are exemplary and other means of establishing a communications link between the computers may be used.

[0178] When used in either a LAN or WAN networking environment, computer 2702 can access a cloud storage system or other network-based storage system, such as a networked virtual machine, that provides one or more aspects of information storage or processing in addition to, or in place of, the above-mentioned external storage device 2716. Generally, the connection between computer 2702 and the cloud storage system can be established via LAN 2754 or WAN 2756, for example, by adapter 2758 or modem 2760, respectively. Upon connecting computer 2702 to an associated cloud storage system, external storage interface 2726, with the aid of adapter 2758 or modem 2760, can manage storage devices provided by the cloud storage system in the same way as other types of external storage devices. For example, external storage interface 2726 can be configured to provide access to cloud storage sources as if those sources were physically connected to computer 2702.

[0179] The computer 2702 may be operable to communicate with any wireless device or entity operatively arranged for wireless communication, such as a printer, a scanner, a desktop or portable computer, a portable data assistant, a communications satellite, any device or location associated with a radio-detectable tag (e.g., a kiosk, a newsstand, a store shelf, etc.), and a telephone. This may include Wireless Fidelity (Wi-Fi) and Bluetooth wireless technologies. Thus, communication may be in a predefined structure similar to an existing network, or simply ad-hoc communication between at least two devices.

[0180] FIG. 28 is a schematic block diagram of a sample computing environment 2800 with which the disclosed subject matter can interact. The sample computing environment 2800 includes one or more client(s) 2810. The client(s) 2810 can be hardware or software (e.g., threads, processes, computing devices). The sample computing environment 2800 also includes one or more server(s) 2830. The server(s) 2830 can also be hardware or software (e.g., threads, processes, computing devices). The server(s) 2830 can store threads for performing transformations, for example, employing one or more embodiments described herein. One possible communication between the client(s) 2810 and the server(s) 2830 can be in the form of a data packet adapted to be transmitted between two or more computer processes. The sample computing environment 2800 includes a communication framework 2850 that can be employed to facilitate communications between the client(s) 2810 and the server(s) 2830. The client(s) 2810 are operably connected to one or more client data store(s) 2820 that can be employed to store information local to the client(s) 2810. Similarly, the server(s) 2830 are operably connected to one or more server data store(s) 2840 that can be employed to store information local to the servers 2830 .

[0181] Various embodiments may be systems, methods, devices, or computer program products at any possible level of technical detail. A computer program product may include a computer-readable storage medium (or media) having computer-readable program instructions recorded thereon for causing a processor to execute aspects of the various embodiments. A computer-readable storage medium may be a tangible device capable of holding and storing instructions for use by an instruction-execution device. A computer-readable storage medium may be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. A non-exhaustive list of more specific examples of computer-readable storage media also includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as a punched card or ridge-in-groove structure having instructions recorded thereon, or any suitable combination of the above. As used herein, computer-readable storage media should not be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., light pulses passing through a fiber optic cable), or electrical signals transmitted over wires.

[0182] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, or edge servers. A network adapter card or network interface within each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium within each computing / processing device. The computer-readable program instructions for carrying out the operations of various embodiments are either assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, integrated circuit configuration data, or source code or object code created using a combination of one or more programming languages, such as object-oriented programming languages ​​such as Smalltalk and C++, and procedural programming languages ​​such as the "C" programming language. The computer readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet Service Provider).In some embodiments, to perform various aspects, an electronic cycle, such as a programmable logic cycle, a field programmable gate array (FPGA), or a programmable logic array (PLA), can execute computer readable program instructions by utilizing state information of the computer readable program instructions to individualize the electronic cycle.

[0183] Various aspects are described herein with reference to flowchart or block diagrams of methods, devices (systems), and computer program products according to various embodiments. It will be understood that each block of the flowchart or block diagrams, and combinations of blocks in the flowchart or block diagrams, can be implemented by computer-readable program instructions. These computer-readable program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, or other programmable data processing device to produce a machine such that the instructions, executed by the processor of the computer or other programmable data processing device, create means for implementing the functions / acts specified in one or more blocks of the flowchart or block diagram. These computer-readable program instructions can also be stored on a computer-readable storage medium that causes a computer, programmable data processing device, or other device to function in a particular manner, and a computer-readable storage medium having instructions stored therein constitutes a product containing instructions that implement aspects of the functions / acts specified in one or more blocks of the flowchart or block diagram. These computer-readable program instructions can also be loaded into a computer, other programmable data processing device, or other device to generate a computer-implemented process by a series of operations performed on the computer, other programmable device, or other device. Therefore, instructions executed on a computer, other programmable device, or other device perform the functions / acts specified in one or more blocks of the flowchart or block diagram.

[0184] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, or computer program products according to various embodiments. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, and may include one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the blocks may occur in the order noted in the figures. For example, two blocks shown in succession may be executed substantially simultaneously, or the blocks may sometimes be executed in the reverse order, depending on the functionality involved. It should also be noted that each block of a block or flowchart diagram, and combinations of blocks in a block or flowchart diagram, may be implemented by a dedicated hardware-based system that performs the specified functions or operations or executes a combination of dedicated hardware and computer instructions.

[0185] While the subject matter of the present invention has been described above in the general context of computer-executable instructions for a computer program product executed on one or more computers, those skilled in the art will recognize that the present disclosure can be or may be implemented in combination with other program modules. Generally, program modules include routines, programs, components, data structures, etc. that perform particular tasks or implement particular abstract data types. Moreover, those skilled in the art will appreciate that various aspects can be implemented with other computer system configurations, including single-processor or multiprocessor computer systems, minicomputing devices, mainframe computers, and other computer systems, including computers, portable computing devices (e.g., PDAs, cellular phones), microprocessor-based or programmable consumer or industrial electronic devices, etc. The illustrated aspects can also be implemented in distributed computing environments where tasks are performed by remote processing devices linked through a communications network. However, not all aspects of the present disclosure may be practiced on a stand-alone computer. In a distributed computing environment, program modules can be located in both local and remote memory storage devices.

[0186] In this application, terms such as "component," "system," "platform," and "interface" may refer to or include computer-related entities or entities associated with an operating device having one or more specific functionalities. The entities disclosed herein may be hardware, a combination of hardware and software, software, or software in execution. For example, a component may be, but is not limited to, a process running on a processor, a processor, an object, an executable file, a thread of execution, a program, or a computer. By way of example, both an application running on a server and the server may be a component. One or more components may reside within a process or thread of execution, and a component may be localized on one computer or distributed across two or more computers. In another example, each component may execute from various computer-readable media having various data structures stored thereon. Components may communicate via local or remote processes according to signals containing one or more data packets (e.g., data from one component interacting with another system via signals over a local system, a distributed system, or a network such as the Internet). As another example, a component may be a device having specific functionality provided by mechanical parts operated by electrical or electronic circuits operated by software or firmware applications executed by a processor. In such cases, the processor may be internal or external to the device and may execute at least a portion of the software or firmware application. As yet another example, a component may be a device that provides a particular functionality without mechanical parts but through electronic components, which may include a processor or other means for executing software or firmware that at least partially provides the functionality of the electronic component.In one aspect, the component may emulate an electronic component via a virtual machine, for example, in a cloud computing system.

[0187] Furthermore, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or." That is, unless otherwise specified or clear from the context, "X uses A or B" means all natural inclusive permutations. That is, if X uses A, if X uses B, or if X uses both A and B, then "X uses A or B" is satisfied in each of the foregoing cases. As used herein, the term "and / or" is intended to have the same meaning as "or." Furthermore, the articles "a" and "an" used in this specification and the accompanying drawings should generally be interpreted to mean "one or more" unless otherwise specified or clear from the context as indicating the singular form. As used herein, the terms "example" or "exemplary" are utilized to mean serving as an example, instance, or illustration. For the avoidance of doubt, the subject matter described herein is not limited by such examples. Furthermore, any aspect or design described herein as "example" or "exemplary" is not necessarily to be construed as preferred or advantageous over other aspects or designs, and is not meant to exclude equivalent exemplary structures and techniques known to those skilled in the art.

[0188] The disclosure herein describes non-limiting examples. For ease of description and explanation, various parts of the disclosure herein use the terms "each," "each," or "all" when describing various examples. The use of the terms "each," "each," or "all" is not limiting. That is, when the disclosure herein presents a description that applies to "each," "each," or "all" of a particular object or component, it should be understood that this is a non-limiting example, and it should be further understood that in various other examples, such a description may apply to less than "each," "each," or "all" of that particular object or component.

[0189] As used herein, the term "processor" can refer to virtually any computing processing unit or device, including, but not limited to, a single-core processor, a single processor with software multithreading execution capabilities, a multi-core processor, a multi-core processor with software multithreading execution capabilities, a multi-core processor with hardware multithreading technology, and a parallel platform. Furthermore, a processor can refer to an integrated circuit, an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a field-programmable gate array (FPGA), a programmable logic controller (PLC), a complex programmable logic device (CPLD), discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. Furthermore, a processor can utilize nanoscale architectures, such as, but not limited to, molecular and quantum dot-based transistors, switches, or gates, to optimize space usage or improve the performance of a user device. A processor can be implemented as a combination of computing processing units. As used herein, terms such as "store," "memory," "data store," "data storage," "database," and virtually any other information storage component related to the operation and functionality of a component are used to refer to a "memory component," an entity embodied in a memory, or a component that includes a memory. It should be understood that memory or memory components described herein may be either volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. By way of example and not limitation, non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable ROM (EEPROM), flash memory, or non-volatile random access memory (RAM) (e.g., ferroelectric RAM (FERAM)).Volatile memory may include, for example, RAM, which may act as external cache memory. By way of example, and not limitation, RAM may come in many forms, including synchronous RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), SyncLink DRAM (SLDRAM), direct Rambus RAM (DRRAM), direct Rambus dynamic RAM (DRDRAM), or Rsmbus dynamic RAM (RDRAM). Additionally, the memory components of the systems or computer-implemented methods described herein are intended to comprise, without being limited to, these and any other suitable types of memory.

[0190] The foregoing are merely examples of systems and computer-implemented methods. Of course, for purposes of describing this disclosure, it is not possible to describe every conceivable combination of components or computer-implemented methods, although many more combinations and permutations of the present disclosure are possible. Furthermore, to the extent that terms such as "comprising," "having," "possessing," and the like are used in the detailed description, claims, appendices, and drawings, such terms are intended to be inclusive in the same manner as the term "comprising" is interpreted when used as a transitional term in the claims.

[0191] The description of various embodiments is presented for purposes of illustration and is not intended to be exhaustive or limited to the embodiments disclosed herein. It will be apparent that many modifications and variations are possible without departing from the scope and spirit of the described embodiments. The terms disclosed herein have been selected to best explain the principles of the embodiments, practical applications or technical improvements over commercially available technologies, or to enable those skilled in the art to understand the embodiments disclosed herein.

[0192] Various non-limiting aspects are illustrated in the following examples.

[0193] Example 1: A system can include the following components: a processor executing computer-executable components stored in a non-transitory computer-readable memory, the computer-executable components including: an accessing component capable of accessing a spectral image of a sample captured from a scientific device, where each pixel of the spectral image corresponds to an energy spectrum; a fitting component capable of fitting a function to the energy spectrum on a pixel-by-pixel basis, the function including a plurality of additively combined terms, where a first term of the plurality of terms represents a fine structure of the energy spectrum, and an integral associated with the first term is constrained to be zero; and an executing component capable of segmenting the spectral image by material based on the first term.

[0194] Example 2: The system of any of the previous examples can be implemented, and the multiple terms can further include a second term representing a monotonically decreasing background of the energy spectrum and a third term representing an atomic cross section of the energy spectrum.

[0195] Example 3: The system of any of the previous examples can be implemented, and the first and third terms can be convolved with the low-loss portion of the energy spectrum.

[0196] Example 4: The system of any of the previous examples can be implemented, where the first term can be a spline whose basis functions are quadratic polynomials computed in quadratic space.

[0197] Example 5: The system of any of the previous examples can be implemented, and the first term can include a spline whose basis function is a rectangular function or a trigonometric function.

[0198] Example 6: The system of any of the previous examples can be implemented, wherein the execution component can segment the spectral image by applying K-means to the first term of each pixel of the spectral image or by running a trained machine learning model.

[0199] Example 7: The system of any of the previous examples can be implemented, wherein the execution component can segment the spectral image by applying K-means to fitting coefficients of the first term for each pixel of the spectral image or by running a trained machine learning model.

[0200] Example 8: The system of any of the previous examples can be implemented and the scientific device can be an electron energy loss microscope. In various embodiments, any one or more combinations of Examples 1-8 can be implemented.

[0201] Embodiment 9: A computer-implemented method includes accessing, by a device operatively coupled to a processor, a spectral image of a sample captured by a scientific device, where each pixel of the spectral image corresponds to an energy spectrum. The device fits a function to the energy spectrum on a pixel-by-pixel basis, the function including a plurality of additively combined terms, where a first term of the plurality of terms represents fine structure in the energy spectrum, where an integral associated with the first term is constrained to be zero. The device then segments the spectral image by material based on the first term.

[0202] Example 10: The computer-implemented method of any of the previous examples can be performed, and the plurality of terms can further include a second term representing a monotonically decreasing background of the energy spectrum and a third term representing an atomic cross section of the energy spectrum.

[0203] Example 11: The computer-implemented method of any of the previous examples can be performed, and the first and second terms can be convolved with a low-loss portion of the energy spectrum.

[0204] Example 12: The computer-implemented method of any of the previous examples can be performed, and the first term can include a spline whose basis functions are quadratic polynomials computed in quadratic space.

[0205] Example 13: The computer-implemented method of any of the previous examples can be performed, and the first term can include a spline whose basis function is a rectangular function or a trigonometric function.

[0206] Example 14: The computer-implemented method of any of the previous examples can be performed, and segmenting the spectral image can be performed based on: applying, by the device, a K-means algorithm to the first term of each pixel of the spectral image; or running, by the device, a trained machine learning model on the first term of each pixel of the spectral image.

[0207] Example 15: The computer-implemented method of any of the previous examples can be performed, and segmentation of the spectral image can be performed based on: the device applying K-means to the fitting coefficients of the first term for each pixel of the spectral image; or the device running a trained machine learning model on the fitting coefficients of the first term for each pixel of the spectral image.

[0208] Example 16: The computer-implemented method of any of the previous examples can be performed where the scientific device is an electron energy loss microscope.

[0209] In various embodiments, any one or more combinations of Examples 9-16 can be implemented.

[0210] Example 17: A computer program product for facilitating spectral image analysis via integral constraint fitting can include a non-transitory computer-readable memory having program instructions embedded therein. In various aspects, the program instructions are executable by a processor to cause the processor to: access a spectral image of a sample captured with an electron energy loss microscope, where each pixel of the spectral image corresponds to an energy loss spectrum; fit a function to the energy loss spectrum on a pixel-by-pixel basis, where the function includes a fine structure term, where an integral associated with the fine structure term is constrained to be zero; and segment the spectral image based on the fine structure term rather than based on a remainder of the function.

[0211] Example 18: The computer program product of the preceding examples can be implemented, with the remaining functionality being to construct atomic cross-section terms that are additive combinations of monotonically decreasing background terms and fine structure terms.

[0212] Example 19: The computer program product of any of the previous examples can be implemented such that fine structure terms and atomic cross section terms, rather than monotonically decreasing background terms, can be convolved with the low-loss portion of the energy loss spectrum.

[0213] Example 20: The computer program product of any of the previous examples can be implemented, and the fine structure terms can include splines with basis functions.

[0214] In various embodiments, any one or more combinations of Examples 17-20 can be implemented.

[0215] In various embodiments, any one or more combinations of Examples 1-20 can be implemented.

Claims

1. 1. A system comprising: A processor executing computer-executable components stored in non-transitory computer-readable memory, the computer-executable components comprising: an access component for accessing a spectral image of a sample captured by a scientific instrument, wherein each pixel of the spectral image corresponds to an energy spectrum; a fitting component that fits a function to the energy spectrum on a pixel-by-pixel basis, the function including a plurality of additively combined terms, a first term of the plurality of terms representing a fine structure of the energy spectrum, and an integral associated with the first term being constrained to zero; an execution component that segments the spectral image by material based on the first term.

2. 2. The system of claim 1, wherein the plurality of terms further comprises a second term representing a monotonically decaying background of the energy spectrum and a third term representing an atomic cross section of the energy spectrum.

3. The system of claim 2 , wherein the first term and the third term are convolved with a low-loss portion of the energy spectrum.

4. The system of claim 1 , wherein the first term comprises a spline whose basis functions are quadratic polynomials calculated over quadratically spaced intervals.

5. The system of claim 1 , wherein the first term comprises a spline whose basis functions are rectangular or triangular functions.

6. 10. The system of claim 1, wherein the execution component segments the spectral image by applying K-means clustering to the first term for each pixel of the spectral image or by running a trained machine learning model on the first term for each pixel of the spectral image.

7. 10. The system of claim 1, wherein the execution component segments the spectral image by applying K-means clustering to the fitting coefficients of the first term for each pixel of the spectral image or by running a trained machine learning model on the fitting coefficients of the first term for each pixel of the spectral image.

8. The system of claim 1 , wherein the scientific instrument is an electron energy loss microscope.

9. 1. A computer-implemented method comprising: a device operatively coupled to a processor accessing a spectral image of the sample captured by a scientific instrument, wherein pixels of the spectral image each correspond to an energy spectrum; the device fitting a function to the energy spectrum on a pixel-by-pixel basis, the function comprising a plurality of additively combined terms, a first term of the plurality of terms representing fine structure of the energy spectrum, and an integral associated with the first term being constrained to zero; and wherein the device segments the spectral image by material based on the first term.

10. 10. The computer-implemented method of claim 9, wherein the plurality of terms further comprises a second term representing a monotonically decaying background of the energy spectrum and a third term representing an atomic cross section of the energy spectrum.

11. 11. The computer-implemented method of claim 10, wherein the first term and the third term are convolved with a low-loss portion of the energy spectrum.

12. 10. The computer-implemented method of claim 9, wherein the first term comprises a spline whose basis functions are quadratic polynomials calculated over quadratically spaced intervals.

13. 10. The computer-implemented method of claim 9, wherein the first term comprises a spline whose basis functions are rectangular or triangular functions.

14. The step of segmenting the spectral image comprises: the device applying K-means clustering to the first term for each pixel of the spectral image; or 10. The computer-implemented method of claim 9, wherein the device is based on running a machine learning model trained on the first term for each pixel of the spectral image.

15. The step of segmenting the spectral image comprises: the device applying K-means clustering to the fitting coefficients of the first term for each pixel of the spectral image; or 10. The computer-implemented method of claim 9, wherein the device is based on running a trained machine learning model on fitting coefficients of the first term for each pixel of the spectral image.

16. 10. The computer-implemented method of claim 9, wherein the scientific instrument is an electron energy loss microscope.

17. 1. A computer program product for facilitating spectral image analysis via integrally constrained fitting, the computer program product comprising: a non-transitory computer-readable memory having program instructions embodied therein, the program instructions being executable by a processor, the computer program product comprising: accessing a spectral image of the sample captured by an electron energy loss microscope, wherein each pixel of the spectral image corresponds to an energy loss spectrum; fitting a function to the energy loss spectrum on a pixel-by-pixel basis, the function including a fine structure term, wherein an integral associated with the fine structure term is constrained to zero; and segmenting the spectral image based on the fine structure terms and not based on the remainder of the function.

18. 18. The computer program product of claim 17, wherein the remaining portion of the function comprises a monotonically decaying background term and an atomic cross section term that are additively combined with the fine structure term.

19. 20. The computer program product of claim 18, wherein the fine structure term and the atomic cross section term are convolved with a low-loss portion of the energy loss spectrum, but the monotonically decaying background term is not convolved.

20. 18. The computer program product of claim 17, wherein the fine structure terms comprise splines whose basis functions are quadratic polynomials, rectangular functions, or triangular functions calculated over quadratically spaced intervals.

Images