Machine Vision for Analytical Data-Based Characterization

Abstract

Machine vision techniques can be used to predict the properties of products produced by chemical processes. The predictions can be based on analytical characterization of the chemical process or the products produced by the chemical process using detectors that generate sequence data. The sequence data can be converted into images and input into an artificial neural network (ANN) trained to predict the product properties based on the images. The predictions of the product properties can be received from the ANN and used to adjust the chemical process or to determine whether to reject the product.

Description

[Technical Field]

[0001] This disclosure relates to machine vision for characterization based on analytical data. Such techniques can be particularly useful for predicting product properties in order to adjust the chemical processes used to produce the product or to determine whether to reject the product. [Background technology]

[0002] An artificial neural network (ANN) is a network that can process information by modeling a network of neurons, such as neurons in the human brain, to process information (e.g., stimuli) being sensed in a particular environment. Similar to the human brain, a neural network typically includes multiple neuron topologies (e.g., which may be referred to as artificial neurons). ANN operations refer to operations that use artificial neurons to process inputs to perform a given task. ANN operations may include implementing various machine learning algorithms to process the inputs. Exemplary tasks that can be processed by implementing ANN operations include machine vision, speech recognition, machine translation, social network filtering, and / or medical diagnosis.

[0003] Chromatography, spectroscopy, and many other analytical characterization methods can generate series data, such as time series or paired xy series data types. Separation can be useful for material characterization. For example, size exclusion chromatography, such as gel permeation chromatography (GPC), combined with careful calibration using molecular weight standards or a molecular weight-sensitive detector such as laser light scattering, can provide a quantitative molecular weight distribution of a polymer sample. The molecular weight distribution can predict many physical properties of a polymer material. Tailoring the molecular weight distribution is beneficial in polymer manufacturing. For example, improved GPC data analysis can improve process control or structure elucidation. Summary of the Invention

[0004] The present disclosure is directed to using improvements in machine vision technology to predict properties of products produced by a chemical process. The predictions can be based on analytical characterization of the chemical process or the products produced by the chemical process using a detector that generates sequence data. The sequence data can be converted into images and input into an artificial neural network (ANN) trained to predict product properties based on the images. A prediction of the product properties can be received from the ANN and used to adjust the chemical process or to determine whether to reject the product.

[0005] As a specific example, the effectiveness of machine vision models for applications in process chemometrics and analytical chemistry is described herein. Images of GPC data collected from chemical products can be used for classification problems (e.g., good vs. bad chemical products) and / or to predict product properties. The present disclosure provides improved model performance compared to the use of summary statistics from GPC data (e.g., number average molecular weight and weight average molecular weight).

[0006] The above summary of the present disclosure is not intended to describe each disclosed embodiment or to describe every implementation of the present disclosure. More particularly, the present specification exemplifies exemplary embodiments. In several places throughout the application, guidance is provided through lists of examples, which examples can be used in various combinations. In each instance, the recited list serves only as a representative group and should not be interpreted as an exclusive list. [Brief explanation of the drawings]

[0007] [Figure 1A] One exemplary approach is shown where the detector responses are plotted in separate panels, with the scale adjusted for each detector.

[0008] [Figure 1B] 1 shows one exemplary approach to overlaying detector responses from three detectors onto a single plot.

[0009] [Figure 1C] Data from Figure 1A are shown after Gramian angular summation field (GASF) transformation.

[0010] [Figure 1D] Data from Figure 1B after GASF transformation are shown.

[0011] [Figure 2] An example of GASF transformation of data is shown below.

[0012] [Figure 3] 1 shows a schematic diagram of a network for use in accordance with at least one embodiment of the present disclosure.

[0013] [Figure 4A] A set of 100 GPC runs spanning several years is shown without aligning to the solvent peak.

[0014] [Figure 4B] A set of 100 GPC runs spanning several years is shown, aligned to the solvent peak.

[0015] [Figure 5A] 1 shows histograms of weight average molecular weights of chemical batches, including some batches known to be bad, according to several previous approaches.

[0016] [Figure 5B] 1 shows histograms of number average molecular weights for chemical batches, including some batches known to be bad, according to several previous approaches.

[0017] [Figure 6] Principal component analysis clustering of chromatograms indicated by chemical product quality.

[0018] [Figure 7] FIG. 1 shows a schematic of a machine vision workflow for chromatography.

[0019] [Figure 8] 1 illustrates a comparison of predicted and actual chemical product properties at weight percent levels using an overlay image of chromatograms trained with a machine learning architecture, in accordance with at least one embodiment of the present disclosure.

[0020] [Figure 9] 1 illustrates an example of a system for machine vision for characterization based on analytical data.

[0021] [Figure 10] 1 illustrates an example machine upon which a set of instructions may be executed to cause the machine to perform various methodologies discussed herein. DETAILED DESCRIPTION OF THE INVENTION

[0022] Deep learning is a type of machine learning that is being enabled by increasing computational power, the availability of data, and improvements in software tools. Deep learning applies ANNs to accomplish tasks previously thought impossible for computers to perform. The "deep" in deep learning refers to the use of multiple layers in the ANN. These layers successively extract higher-level features from the raw input. In machine vision, examples of low-level features in an input image include edges or color. Higher-level features learned in deeper layers in the network could be objects such as faces or handwritten digits.

[0023] Open-source, well-trained networks have been built on databases containing millions of images. These networks can work with new data through transfer learning, meaning that while millions of data points were needed to build the initial network, less data is needed to adapt the network to new uses. According to at least one embodiment of the present disclosure, pre-trained deep learning networks, such as two-dimensional image input networks, can be used on paired xy data created by analytical characterization methods. Converting GPC chromatograms to images can enable classification by ANNs with predictive accuracy of over 96%. Converting analytical data (such as GPC data) to images can be done, for example, by array images of xy paired data into line plots. Another example is the GASF conversion of analytical data (e.g., detector responses or y values) into a two-dimensional matrix, which is then colored by the value of each matrix entry. As used herein, an image can refer to any visual or optical representation (e.g., a visual representation of data). An image can also refer to the data defining the image (e.g., when the image is stored on a tangible, machine-readable medium). For example, an image can refer to a visual representation displayed on a computer screen or an electronic file containing data that defines the image displayed on the screen. Converting data to an image means that the data is converted from a non-image format into an image format suitable for use with an ANN trained to predict product properties based on the image.

[0024] Embodiments of the present disclosure can be extended to combine chromatographic spectral processes with other data, requiring significantly less subject matter expertise. This presents a much faster and more easily leveraged method for using all available data. There is a range of data sources that can be applied to various embodiments of the present disclosure, including chemical and physical characterization techniques. While GPC is described with respect to various examples herein, the embodiments are not so limited. Other chemical and physical characterization techniques can be used. Examples of such techniques include size exclusion chromatography (e.g., GPC), liquid chromatography, gas chromatography, thermal gradient chromatography, calorimetry, rheology, optical spectroscopy, mass spectrometry, viscometry, particle size measurement, or nuclear magnetic resonance spectroscopy. This list is not exhaustive. Rather, embodiments of the present disclosure can be applied to any measurement method consistent with the analytical and / or sequence data described herein.

[0025] As used herein, the singular forms "a," "an," and "the" include both singular and plural referents unless the context clearly dictates otherwise. Furthermore, the word "may" is used throughout this application in its permissive (i.e., potential, can) sense rather than its obligatory (i.e., must) sense. The term "comprises" and its derivatives mean "including, but not limited to." The term "coupled," unless otherwise specified, means directly or indirectly connected, and can include wireless connections.

[0026] As will be understood, elements shown in the various embodiments herein may be added, interchanged, and / or eliminated to provide additional embodiments of the present disclosure. Additionally, it will be understood that the proportions and relative scales of the elements provided in the figures are intended to illustrate certain embodiments of the present invention and should not be construed in a limiting sense.

[0027] A GPC data structure is an array of xy data. The x-axis is either time (usually minutes) or volume (usually milliliters, or "mL"). The y-axis is the detector response, which may consist of multiple detectors. The change in detector response as a function of time (or volume) provides the information needed to determine the molecular weight distribution of a given sample. This data structure is a type of time series because the data is ordered by elution time. One example of time series data analysis is time series forecasting, which uses historical data for a set of variables over time to predict future values ​​of those variables over a set period of time. Predicting future values ​​at later retention volumes may not be useful for GPC. At least one embodiment of the present disclosure includes time series classification or time series regression. Time series classification involves classifying GPC data into a set of predefined categories, such as lot quality discrimination (e.g., good and bad material), which can be used to determine whether to reject a product produced by a chemical process. Time series regression performs the same underlying task, but the predicted output is a continuous variable, such as a prediction of viscosity or melt index.

[0028] Univariate time series X=[x1,x2,...,x T ] is an ordered set of real values. The length of X is equal to the number of real values ​​T. A multivariate time series is a set of real values ​​X=[X1,X2,...,X M ] and X i ∈R T The dataset D = (X1,Y1),(X2,Y2),...,(X N ,Y N ) is (X i , Y i ) and X i can be either a univariate or a multivariate time series. An example of this type of data is GPC, where X is the retention volume and Y is the detector response.

[0029] Traditionally, various statistics are used to reduce these high-dimensional data to a manageable size. The traditional approach to analyzing GPC data is to obtain the molecular weight distribution (e.g., number average molecular weight M n , weight average molecular weight M w , degree of dispersion

number

[0030] At least one embodiment of the present disclosure includes a new approach for analyzing chromatographic data that uses images as input instead of summary statistics or digitized time-intensity sequences. Leveraging the success of machine vision applications, ANNs, such as deep neural networks, can be trained on images of GPC data for both classification and regression tasks. Compared to traditional GPC data analysis, this requires significantly more computational resources and larger datasets for successful implementation.

[0031] A wide array of silicone materials exist with complex polymer structures. Chromatography, primarily GPC, can be used to characterize the quality of these materials. Silicone materials can be used as raw materials to make other products. However, raw material lots (e.g., M) that do not exhibit anomalous properties via GPC summary statistics are often unsuitable for use. n , M w ), or other lot-acceptance requirements (e.g., silanols), have nevertheless caused problems downstream.

[0032] The problem of adequately characterizing the composition and performance of advanced materials is prevalent across many applications in silicones. A quality control gap exists between a product-by-process approach and obtaining quantitative property metrics that can identify a lot of material. Analytical characterization experts can enable process improvements by better understanding the target material and its properties.

[0033] As an example, we use GPC data collected for in-process analysis of silicone materials over several years. ANNs can be used to predict the quality of silicone polymer raw materials, as determined by known manufacturing upsets, and to predict the properties of the final product, i.e., vinyl and silanol percentages. Previous approaches, which involved reducing GPC data to summary statistics, have not been able to successfully model the quality classification of the polymer or downstream products.

[0034] Various approaches can be used to generate images of a sample. Figure 2A shows one exemplary approach, plotting detector responses in separate panels, with the scale adjusted for each detector. This is referred to as a faceted plot. Figure 1B shows one exemplary approach, overlaying detector responses from three detectors onto a single plot. Figure 1C shows the data from Figure 1A after GASF transformation. Figure 1D shows the data from Figure 1B after GASF transformation. The GASF transformation is an alternative encoding of the data.

[0035] Figure 2 shows an example of a GASF transformation of data. The GASF transformation can involve three steps of data augmentation. First, a time series of n observations, X = {x1, x2, ..., x n} is given, rescales X so that all values ​​are in the interval [0, 1].

number

number

number

number

number

[0036] In the above equation, I is a unit row vector. Polar coordinates preserve absolute time relationships, whereas Cartesian coordinates do not. In polar coordinates, the angle cosine is a value (e.g., detector response) and the radius is a time step (e.g., retention volume). One advantage of polar coordinates is the preservation of absolute time relationships. The Gramian transformation reduces the sparsity of the image fed to the network compared to a Cartesian time series plot. In this case, sparsity refers to the proportion of white space in the chromatogram image. In the signal overlay image (Figure 1B), over 93% of the pixels are white. The Gramian transformation reduces the number of white pixels to nearly zero. Imputation can be improved by this method compared to raw time series data.

[0037] FIG. 3 shows a schematic diagram of a network used in accordance with at least one embodiment of the present disclosure. The network has 11 layers overall, with three convolutional blocks. Each block contains three convolutional layers, for a total of nine convolutional layers. However, embodiments are not limited to any particular number of layers or convolutional blocks. After each convolution, there may be a batch normalization and activation step. The batch normalization step may normalize the layer output to have a mean close to zero and a standard deviation close to one. Normalization methods may be used as an alternative to dropout to limit overfitting. The activation step may use a rectified linear unit as the activation function. The penultimate layer may perform a global average pooling operation. The final layer is the prediction step. Shortcuts shown in the network schematic refer to residual network connections. Residual network bypass convolutional blocks have been shown to significantly improve training times for deep networks by solving the vanishing gradient problem during network optimization. For example, these shortcuts allow network optimization and error reduction to pass through many layers of a deep neural network.

[0038] Hyperparameters for the model can be tuned to improve model accuracy for a given embodiment of the present disclosure. As a non-limiting example, CNN layer filters can be 32, 64, and 64 for layers in code blocks 1, 2, and 3, respectively. Kernel sizes of 8x8, 3x3, and 1x1 can be used for layers 1, 2, and 3, respectively, within a given convolutional block. Data analysis can be performed using available tools.

[0039] Figure 4A shows a set of 100 GPC runs spanning several years without aligning to the solvent peak. The peaks appearing at retention volumes between 17.5 and 18.5 mL correspond to known monomer species with consistent sizes. Because the monomers are structurally similar across all samples, corresponding peaks in GPC should overlap across all experiments analyzed using the same method. In Figure 4A, the peaks are not well aligned, indicating that the GPC results have some drift over months or years. Figure 4B shows a set of 100 GPC runs spanning several years aligned to the solvent peak. By aligning to this solvent peak, the overall experimental alignment observed with the monomer peak is much improved. The aligned GPC results were used for subsequent analysis. While the data presented in Figures 4A-4B show the results of aligned chromatograms, embodiments are not limited to such alignments for generating accurate machine learning models. The accuracy of classification models may be indistinguishable between aligned and unaligned data in terms of model accuracy.

[0040] Previous efforts at analyzing summary statistics have failed to identify GPC features that can distinguish between good 503 and bad 501 chemicals in these batches. Figure 5A shows histograms from several previous approaches for the weight average molecular weight of chemical batches, including some batches known to perform poorly. The dashed lines in Figures 5A-5B represent the means for that category. Good 503 and bad 501 appear to have nearly identical distributions. The overlap is labeled 505. In Figure 5A, the dashed line represents the overlap 505 between the means of the good 503 and bad 501 chemical batches. In Figure 5B, separate dashed lines are shown for the means of the good 503 and bad 501 categories. Controlling these properties when making chemicals does not mean that the entire product will have a similar tightness in the distribution of number average molecular weight values.

[0041] FIG. 5B shows a histogram from several previous approaches for the number average molecular weight of a chemical batch, including some batches known to be defective 501. The distribution of chemical number average molecular weights shows a larger difference between good 503 and bad 501 samples than the weight average molecular weight. The distribution of defective 501 samples has a higher number average molecular weight on average, meaning that despite consistent weight average molecular weights, the product distribution still drifts from batch to batch. While the number average molecular weight shows more difference between good 503 and bad 501 materials, the large overlap between the distributions prevents number average molecular weight from being an accurate discriminator of batch quality.

[0042] Unsupervised learning can be applied to the assembled data to distinguish differences in GPC data across various chemical lots. The unsupervised learning task is to model the underlying structure of the data without explicit labels (Y data) for each sample. Such methods can identify previously unknown patterns or features within the data. In this example, the unsupervised learning task identifies patterns in the chromatograms and then uses those patterns to separate clusters of samples. These clusters should represent good and bad chemical lots, but labels are not included in the analysis. One example of a method for unsupervised learning is principal component analysis (PCA), a dimensionality reduction technique that highlights data differences. Figure 6 shows PCA clustering of chromatograms indicated by chemical quality. In Figure 6, different chemical qualities are indicated by solid and empty circles due to limitations in the printed patent publication. However, real chromatograms typically make such distinctions by color. Figure 6 shows that the PCA-based visualization can show the difference between average good and bad lots, but does not show a significant enough separation to distinguish between good and bad for unseen GPC data (to predict the quality of new lots).

[0043] Image classification using ANNs, such as deep neural networks, is a successful application of machine learning. According to the present disclosure, generated images of chromatographic data can be used as input images to perform chemical product classification tasks. Figure 7 shows a schematic diagram of a machine vision workflow for chromatography. A network architecture for lot classification can train an ANN ("deep neural network") by vanishing gradients, for example. Network optimization can be performed using backpropagation and gradient descent to minimize a defined loss function. Small gradients, which tend to vanish when propagated through an ANN, can cause saturation or degradation of model performance. Shortcut connections can be integrated to skip layers in the network, enabling gradient propagation through many layers (e.g., more than 100 layers).

[0044] Each layer of the ANN is represented in the simplified image of FIG. 7 as a column of nodes. The nodes, which may correspond to artificial neurons, may receive various inputs. Interconnection regions may connect nodes between different layers, as indicated by the lines connecting the nodes in FIG. 7. Nodes may receive inputs from other nodes through the interconnection regions. In at least one embodiment, the interconnection regions may connect each node in a first layer with each node in a second layer, although the embodiment is not so limited. The ANN may be configured with a training process in which the various connections in the interconnection regions are assigned weight values ​​or updated with new weight values ​​used in operations or calculations at the nodes. The training process may vary depending on the particular application or use of the ANN. For example, the ANN may be trained for image recognition as described herein or another processing or computational task.

[0045] The ANN may include an output layer, represented by the last column of nodes on the right side of the image. This last column of nodes may be referred to as an output node. Each of the output nodes may be coupled to receive inputs from nodes in the previous layer of nodes (to the left). The process of receiving a usable output at the output layer of output nodes as a result of inputs provided to nodes in the first layer (the leftmost layer shown in FIG. 7) may be referred to as inference or forward propagation. That is, input signals representing some real-world phenomenon or application may be provided to the trained ANN, and results may be output through inference that occurs as a result of computations enabled by the various nodes and interconnections. In the case of an ANN trained for image recognition, the input may be a signal representing a chromatogram, and the output may be a signal representing the quality of a chemical product indicated by the chromatogram.

[0046] Tests were performed, and the results of chemical classification were summarized as either good or bad according to the input image type as follows. In each case, the images were cropped and normalized. Cropping refers to narrowing the range of retention volumes to include only the region of the chromatogram considered to be relevant domain experts. Normalization was performed by scaling the plot to include values ​​between 0 and 1. When separate GPC curves were input, each quadrant contained a single GPC chromatogram. Data preprocessing is not required for modeling tasks, but in some cases, discarding unhelpful data can improve model performance. The accuracy of the test setup was 98.9%. When overlaid GPC curves were input, the GPC curves were overlaid on each image. The accuracy of the test setup was 99.2%. When separate GASF transformations were input, each quadrant contained a single GASF-transformed GPC signal. The accuracy of the test setup was 99.2%. When a single GASF-transformed image was input, the image was the GASF transform of a linear extension of the GPC signal. The accuracy for the test setup was 98.7%. Performance for each image category was excellent, with an accuracy of 99.0 ± 0.2%. Comparing across image input types, there does not appear to be any significant performance difference for any of the inputs. Even state-of-the-art analytical characterization methods applied to this classification task have not published a clear standalone method for assessing chemical products.

[0047] The network architecture and hyperparameters for regression can be the same as for classification models. To convert from a classification model to regression, the last layer of the neural network can be changed from a single-node sigmoid activation to a layer with two output nodes without any activation. For example, with reference to FIG. 7, the output is a product characteristic rather than lot quality. Specifically, for a chemical process that makes silicone materials, one of the nodes corresponds to vinyl content and the other corresponds to silanol content. Multiple output nodes can predict their variables simultaneously, or different models can be used for each prediction (output node).

[0048] 8 shows a comparison of predicted and actual chemical property values ​​using an overlay image of a chromatogram trained with a machine learning architecture, according to at least one embodiment of the present disclosure. The prediction plot demonstrates that quantitative prediction from GPC image data alone can be used to predict chemical property values ​​with a relative error of 0.7%. This demonstrates the practical application of machine learning methods for chemical products produced by chemical processes.

[0049] FIG. 9 illustrates an example of a system for machine vision for analytical data-based characterization. The system can include a detector 920 configured to analytically characterize a product 922 produced by a chemical process 924. Examples of detectors 920 include a concentration-sensitive detector, a molecular weight-sensitive detector, a composition-sensitive detector, or a combination thereof. Examples of concentration-sensitive detectors include ultraviolet absorbers, differential refractometers or refractive index detectors, infrared absorbers, and density detectors. Examples of molecular weight-sensitive detectors include low-angle light scattering detectors and multi-angle light scattering detectors. Examples of analytical characterization methods include size exclusion chromatography, liquid chromatography, gas chromatography, thermal gradient chromatography, calorimetry, rheology, optical spectroscopy, mass spectrometry, viscometry, particle size measurement, and nuclear magnetic resonance spectroscopy. The detector 920 can be configured to generate series data 926 from the analytical characterization. For example, the series data 926 can be multivariate data. The series data 926 may be multivariate, for example, in embodiments including an instrument with multiple detectors (e.g., GPC with refractive index and light scattering) or multiple instruments each with at least one detector (e.g., for multiple separate characterizations of the same product). The product 922 may be a polymeric material produced by a chemical process 924.

[0050] The system may include an ANN 930 trained with multiple images of transformed series data from a prior product produced by chemical process 924 to predict properties 932 of product 922 based on images 928 transformed from series data 926. ANN 930 may be pre-trained to identify features within the images and further trained via transfer learning with multiple images of transformed series data from a prior product produced by chemical process 924, such that the features ANN 930 is now trained to identify are properties 932 of product 922. Examples of properties 932 of product 922 include molecular weight, density, quality, performance, and identity. In at least one embodiment, ANN 930 may be a two-dimensional image input network. While shown as separate from controller 900, ANN 930 may be implemented by controller 900 in at least one embodiment. ANN 930 is described in more detail above.

[0051] The system may include a controller 900 coupled to the detector 920 and coupled to an ANN 930. While not specifically shown, the controller 900 may include processor and memory resources that store instructions executable by the controller 900 to perform the functions described herein. An example of a controller 900 is described in more detail with respect to FIG. 10. The controller 900 may be configured to convert the series data 926 into images 928 and input the images 928 to the ANN 930. In at least one embodiment, the controller 920 may be configured to convert the series data 926 into images 928 without preprocessing the series data 926. As described herein, conversion of the time series data 926 (such as GPC data) into images 928 may be performed, for example, by array images into two-dimensional line plots of x-y paired data or by GASF conversion, among other conversion methods. The controller 900 may be configured to receive predictions of properties 932 of the product 922 from the ANN 930.

[0052] The controller 900 can be configured to provide an output 934 based on the prediction of the characteristic 932 (e.g., if the characteristic 932 does not meet a predefined specification for the characteristic 932). One example of the output 934 is an adjustment to the chemical process 924. Thus, in at least one embodiment, the controller 900 can be configured to control the chemical process 924 or be coupled to other control circuitry that controls the chemical process 924. In one such example, the controller 900 can cause one or more parameters of the chemical process 924 to be adjusted so that the characteristics of a chemical product subsequently produced by the chemical process 924 are more likely to be within the predefined specifications. As another example, the output 934 from the controller 900 can be used to adjust the chemical process 924 through human intervention (e.g., when a human adjusts one or more parameters of the chemical process 924 so that the characteristics of a chemical product subsequently produced by the chemical process 924 are more likely to be within the predefined specifications). The output 934 can be a control signal for the chemical process 924, data indicative of the acceptability of the product 922, or an indicator, such as a light or sound, indicative of the acceptability of the product 922. As another example, output 934 may be a rejection of product 922. For example, controller 900 may provide an indication to an operator that product 922 should be rejected, or controller 900 may automatically flag product 922 for rejection. In at least one embodiment, controller 900 may be configured to both adjust chemical process 924 and reject product 922 based on product characteristics 932.

[0053] 10 illustrates an example machine 1000 upon which a set of instructions may be executed to cause the machine 1000 to perform various methodologies discussed herein. In various embodiments, the machine 1000 may be similar to the controller 900 described with respect to FIG. 9. In alternative embodiments, the machine 1000 may be connected (e.g., networked) to other machines in a LAN, an intranet, an extranet, and / or the Internet. The machine 1000 may operate as a peer machine in a peer-to-peer (or distributed) network environment, or as a server or client machine in a cloud computing infrastructure or environment, or as a server or client machine in a client-server network environment.

[0054] Machine 1000 may be a personal computer (PC), tablet PC, set-top box (STB), personal digital assistant (PDA), mobile phone, web appliance, server, network router, switch, or bridge, or any machine capable of executing a set of instructions (sequentially or otherwise) that specify actions to be taken by the machine. Moreover, although a single machine 1000 is illustrated, the term "machine" should also be taken to include any collection of machines that individually or collectively execute a set (or sets) of instructions to perform any one or more of the methodologies discussed herein.

[0055] The exemplary machine 1000 includes a processing device 1002, a main memory 1004 (e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM) such as synchronous DRAM (SDRAM) or Rambus DRAM (RDRAM)), a static memory 1006 (e.g., flash memory, static random access memory (SRAM), etc.), and a data storage system 1008 that communicate with each other via a bus 1010.

[0056] The processing device 1002 represents one or more general-purpose processing devices, such as a microprocessor, a central processing unit (CPU), or the like. More specifically, the processing device may be a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, or a processor implementing other sets of instructions or a combination of sets of instructions. The processing device 1002 may also be one or more special-purpose processing devices, such as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), a network processor, or the like. The processing device 1002 is configured to execute instructions 1018 to perform the operations and steps discussed herein. The machine 1000 may further include a network interface device 1012 for communicating over a network 1014.

[0057] The data storage system 1008 may include a machine-readable storage medium 1016 (also known as a computer-readable medium) that stores one or more sets of instructions 1018 or software that embody any one or more of the methodologies or functions described herein. The instructions 1018 may also reside, completely or at least partially, within the main memory 1004 and / or within the processing device 1002 during execution thereof by the machine 1000, with the main memory 1004 and the processing device 1002 also constituting machine-readable media.

[0058] In one embodiment, instructions 1018 include instructions for implementing functionality corresponding to an ANN described herein. While machine-readable storage medium 1016 is shown in one exemplary embodiment to be a single medium, the term "machine-readable storage medium" should be interpreted to include a single medium or multiple media that store one or more sets of instructions. The term "machine-readable storage medium" should also be interpreted to include any medium that can store or encode a set of instructions for execution by a machine, causing the machine to perform any one or more of the methodologies of the present disclosure. Accordingly, the term "machine-readable storage medium" should be interpreted to include, but is not limited to, solid-state memory, optical media, and magnetic media.

[0059] Although specific embodiments have been described above, these embodiments are not intended to limit the scope of the present disclosure, even if only a single embodiment is described with respect to a particular feature. The example features provided in this disclosure are intended to be illustrative rather than limiting, unless otherwise stated. The above description is intended to cover such alternatives, modifications, and equivalents as would be apparent to one skilled in the art having the benefit of this disclosure.

[0060] The scope of the present disclosure includes any feature or combination of features (either explicit or implicit) disclosed herein, or any generalization thereof, whether or not it mitigates any or all of the problems addressed herein. While various advantages of the present disclosure have been described herein, embodiments may provide some, all, or none of such advantages, or may provide other advantages.

[0061] In the foregoing detailed description, some features are grouped together in a single embodiment for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the disclosed embodiments of the present disclosure require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive subject matter lies in less than all features of a single disclosed embodiment. Thus, the following claims are incorporated into the Detailed Description herein, with each claim standing on its own as a separate embodiment. [Example 1] 1. A method comprising: analytically characterizing a chemical process or a product produced by said chemical process using an analyzer, thereby generating sequence data; converting the series of data into an image; inputting the image into an artificial neural network (ANN) trained to predict a property of the product based on the image; receiving the prediction of the characteristic of the product from the ANN; and adjusting the chemical process or rejecting the product based on the prediction of the property of the product. [Example 2] 2. The method of example 1, further comprising adjusting the chemical process and rejecting the product based on the prediction of the property of the product. [Example 3] the ANN is pre-trained to identify features in any image; 2. The method of example 1, further comprising training the ANN via transfer learning using a plurality of images of transformed sequence data from a prior product produced by the chemical process such that the features comprise the characteristic of the product. [Example 4] 4. The method of any one of Examples 1 to 3, wherein receiving the prediction of the property of the product comprises receiving the prediction of one of a group of properties comprising molecular weight, density, quality, performance, and identity. [Example 5] 5. The method of any one of Examples 1 to 4, wherein analytically characterizing the product comprises one of the group of analytical characterizations comprising liquid chromatography, gas chromatography, thermal gradient chromatography, size exclusion chromatography, calorimetry, rheology, optical spectroscopy, mass spectrometry, viscometry, particle size measurement, and nuclear magnetic resonance spectroscopy. [Example 6] 6. The method of any one of Examples 1 to 5, wherein converting the series data into the image comprises converting the series data into a two-dimensional line plot. [Example 7] 7. The method of any one of Examples 1 to 6, wherein converting the sequence data to the image comprises converting the sequence data to a Gramian angle sum field. [Example 8] 8. The method of any one of Examples 1 to 7, wherein converting the series of data into the image comprises converting the series of data without preprocessing the series of data. [Example 9] 9. The method of any one of examples 1-8, wherein inputting the image to the ANN comprises inputting the image to a two-dimensional image input network. [Example 10] 1. A system comprising: A detector comprising: analytically characterizing a product produced by the chemical process; generating sequence data from the analytical characterization; and an artificial neural network (ANN) trained with a plurality of images of transformed sequence data from a prior product produced by the chemical process to predict a property of the product based on the images transformed from the sequence data; a controller coupled to the detector and the ANN, the controller comprising: converting the series data into the image; inputting the image into the ANN; receiving the prediction of the characteristic of the product from the ANN; and a controller configured to adjust the chemical process or reject the product. [Example 11] 11. The system of claim 10, wherein the system comprises a plurality of detectors, and the sequence data comprises multivariate data corresponding to the plurality of detectors. [Example 12] 12. The system of any one of claims 10 to 11, wherein the detector comprises one of a group of detectors comprising a concentration-sensitive detector, a molecular weight-sensitive detector, a composition-sensitive detector, and combinations thereof. [Example 13] 13. The system of any one of Examples 10-12, wherein the controller is configured to adjust the chemical process and reject the product. [Example 14] 14. The system of any one of Examples 10 to 13, wherein the controller is configured to convert the series data into the image without preprocessing the series data. [Example 15] 15. The system of any one of Examples 10 to 14, wherein the ANN is a two-dimensional image input network.

Claims

1. A method performed by a system for machine vision, comprising: a detector analytically characterizing a chemical process or a product produced by said chemical process using said detector, thereby generating series data, wherein analytically characterizing said product comprises one of a group of analytical characterizations comprising liquid chromatography, gas chromatography, thermal gradient chromatography, size exclusion chromatography, calorimetry, rheology, optical spectroscopy, mass spectrometry, viscometry, particle size measurement, and nuclear magnetic resonance spectroscopy; a controller converting the series data into an image, wherein converting the series data into an image comprises converting the series data into a two-dimensional line plot or a Gramian angle sum field; the controller inputting the image into an artificial neural network (ANN) trained to predict a characteristic of the product based on the image; the controller receiving the prediction of the characteristic of the product from the ANN; the controller adjusting the chemical process or rejecting the product based on the prediction of the property of the product.

2. The method of claim 1, further comprising the controller adjusting the chemical process and rejecting the product based on the prediction of the characteristics of the product.

3. the ANN is pre-trained to identify features in any image; 10. The method of claim 1, further comprising training the ANN via transfer learning with a plurality of images of transformed sequence data from a prior product produced by the chemical process such that the features comprise the characteristic of the product.

4. 4. The method of claim 1, wherein receiving the prediction of the property of the product comprises receiving the prediction of one of a group of properties comprising molecular weight, density, quality, performance, and identification.

5. A method according to any one of claims 1 to 4, wherein converting the series data into the image includes converting the series data without normalizing the series data as a preprocessing step.

6. A method according to any one of claims 1 to 5, wherein inputting the image into the ANN includes inputting the image into a two-dimensional image input network.

7. 1. A system comprising: A detector comprising: analytically characterizing a product produced by a chemical process, wherein analytically characterizing the product comprises one of a group of analytical characterizations comprising liquid chromatography, gas chromatography, thermal gradient chromatography, size exclusion chromatography, calorimetry, rheology, optical spectroscopy, mass spectrometry, viscometry, particle size measurement, and nuclear magnetic resonance spectroscopy; generating sequence data from the analytical characterization; and an artificial neural network (ANN) trained with a plurality of images of transformed sequence data from a prior product produced by the chemical process to predict a property of the product based on the images transformed from the sequence data; a controller coupled to the detector and the ANN, the controller comprising: converting the series data to the image, wherein converting the series data to the image comprises converting the series data to a two-dimensional line plot or a Gramian angle sum field; inputting the image into the ANN; receiving the prediction of the characteristic of the product from the ANN; and a controller configured to adjust the chemical process or reject the product.

8. the system comprises a plurality of detectors, and the sequence data comprises multivariate data corresponding to the plurality of detectors; or The system of claim 7 , wherein the detector comprises one of a group of detectors including a concentration-sensitive detector, a molecular weight-sensitive detector, a composition-sensitive detector, and combinations thereof.

9. the controller is configured to adjust the chemical process and reject the product; or 9. The system of claim 7 or 8, wherein the controller is configured to convert the series data into the image without normalizing the series data as a pre-processing step.

10. The system according to any one of claims 7 to 9, wherein the ANN is a two-dimensional image input network.

Images