Formula graphs for machine learning of chemical products

Abstract

The chemical formulation of a chemical product can be represented by a digital formulation graph for use in a machine learning model. The digital formulation graph can be input into a graph-based algorithm, such as a graph neural network, to generate a feature vector that is a denser description of the chemical product than the digital formulation graph. The feature vector can be input into a supervised machine learning model to predict the values ​​of one or more attributes of the chemical product produced by the formulation without having to actually go through the production process. The feature vector can be input into an unsupervised machine learning model that is trained to compare chemical products based on their feature vectors. The unsupervised machine learning model can recommend alternative chemical products based on the comparison.

Description

[Technical Field]

[0001] The present disclosure relates to machine learning of chemical products using formulation graphs. Such techniques can be useful for predicting attributes of chemical products without the need to actually manufacture the chemicals. Such techniques can be useful for understanding available chemical products and recommending alternative chemical products that can be substituted in downstream applications or downstream formulations. [Background technology]

[0002] An artificial neural network (ANN) is a network that can process information by modeling a network of neurons, such as neurons in the human brain, to process information (e.g., stimuli) being sensed in a particular environment. Similar to the human brain, a neural network typically includes multiple neuronal topologies, which may be referred to as artificial neurons. ANN operations refer to operations that process inputs using artificial neurons to perform a given task. ANN operations may include implementing various machine learning algorithms to process the inputs. Exemplary tasks that can be processed by implementing ANN operations include machine vision, speech recognition, machine translation, social network filtering, and medical diagnosis, among others.

[0003] Machine learning and statistical analysis can be used to aid in the design of chemical products. Some empirical modeling methods use training data that include independent variables that describe the chemical system of interest. Examples of such variables include descriptors ("X variables") and desired attributes of the chemical product to be produced ("Y variables"). Various algorithms can detect and encapsulate patterns between the X and Y variables. Modeling tools can be developed to allow users to test hypotheses about the predicted outcome of a new set of input variables or to optimize inputs to meet the desired Y variable specifications of a chemical product. Summary of the Invention

[0004] The present disclosure is directed to a new method for representing chemical formulations used in machine learning models that seeks to address the existing shortcomings of some previous approaches for representing chemical formulations. For example, a formulation can be represented as a digital formulation graph structure (e.g., a tree) for input into graph-based algorithms such as graph neural networks (GNNs). The digital formulation graph represents or includes the manufacturing history of the ingredients used in a product. A "first-level" formulation of a chemical product includes only the direct ingredients that are mixed together in the final step of producing the chemical product. The first-level formulation is also referred to as the product recipe. However, these ingredients may also be mixtures of other ingredients (e.g., via blending, chemical reactions, etc.). The digital formulation graph can include representations of each ingredient working backward toward the leaf nodes until a logical conclusion, such as the purchase of a source monomer or raw material, is reached. The use of such digital formulation graphs in machine learning algorithms can enable more accurate prediction of the attributes of chemical products produced according to the formulation. Using the digital formulation graph as a descriptor can enable much wider and faster scaling of machine learning capabilities across a chemical producer's product line. Instead of each project taking months to collect descriptors, the data is available almost instantly from the formulation database.

[0005] As a specific example, the effectiveness of a machine learning model using a digital formula graph of a chemical product formula in predicting attributes of a chemical product is described herein. The machine learning model can be used to predict one or more attributes of a chemical product resulting from a formula without the need to actually produce the chemical product, which can be time-consuming and / or expensive, especially when it is desirable to test different formulas. The present disclosure provides improved model performance compared to the use of either a statistical model with formula data or a statistical model with formula data and ingredient descriptors. Ingredient descriptors can be difficult, costly, and / or time-consuming to obtain.

[0006] The machine learning model can be used to compare chemical products based on their feature vectors and performance characteristics. There is no need to produce the chemical products, which can save time and / or money, especially when it is desirable to test different formulations. The machine learning model can be used to recommend alternative chemical products for a particular chemical product based on the comparison. For example, a particular chemical product may be temporarily unavailable or too expensive for a desired formulation or application, and an alternative chemical product may be desired to replace the particular chemical product without sacrificing the performance characteristics provided by the particular chemical product.

[0007] The above summary of the present disclosure is not intended to describe each disclosed embodiment or to describe every implementation of the present disclosure. More particularly, the present specification exemplifies exemplary embodiments. In several places throughout the application, guidance is provided through lists of examples, which examples can be used in various combinations. In each instance, the recited list serves only as a representative group and should not be interpreted as an exclusive list. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is an example of a chemical product formulation graph. [Figure 2] FIG. 1 is a block diagram illustrating an example of training a graph neural network. [Figure 3] 1 is a chart showing examples of chemical product attribute prediction results using various approaches. [Figure 4] 1 illustrates an example machine upon which a set of instructions may be executed to cause the machine to perform various methodologies discussed herein. DETAILED DESCRIPTION OF THE INVENTION

[0009] Developing chemical formulations (blends of chemical components that achieve the target bulk properties of the overall mixture) is a unique skill. The selection and preferences of formulation components can be formed by intuition and / or experience, which are not easily recorded or easily found. Overall, a chemical formulation can be described as a matrix, where each column is a component that appears in at least one chemical formulation and each row is a chemical product (e.g., a formulation sample) where the weight fraction of each component is numerically represented. The terms weight fraction and weight percent are used interchangeably. While hundreds of components may be used in a training dataset, only a few dozen components are used within a single formulation. As an example, for every row, there may be 190 zeros and 10 non-zero values. Furthermore, because each component is typically used in only a few formulations, all columns are also mostly zeros, with only a few non-zero values. Therefore, the data is very sparse, and the variables are highly unbalanced and non-normal. The sparsity of the data makes modeling difficult. However, formulation data can be useful for finding new formulations to solve chemical customer problems.

[0010] Chemical descriptors may be referred to herein as "X variables" or "X data." Desired attributes of produced chemical products may be referred to herein as "Y variables" or "Y data." The nature of the X data significantly impacts which modeling techniques will be successful in predicting Y from X. For example, X data consisting of a small number (e.g., less than 100) of normally distributed continuous numeric values ​​with no missing values ​​has a very wide range of suitable algorithms for selection with a high success rate. This type of data is referred to as having dense information content. The less dense the information content of the data, the more likely a given algorithm is to miss important patterns in the data (underfitting) and / or attribute patterns to random variations in the model inputs (overfitting). Underfitting and overfitting can result in unusable or inaccurate models, respectively. The introduction of categorical variables can cause complications. While some algorithms can directly handle multilevel categorical variables, most use one-hot encoding or create dummy variables, which can introduce sparsity in the dataset in the form of many zeros. One-hot encoding uses groups of bits containing a single bit with a value of 1, with the remaining bits having a value of 0. These variables are not normally distributed and can be highly unbalanced, posing significant challenges to modeling. Another form of data sparsity can be introduced by missing data. Few algorithms can handle missing data directly. Most algorithms require filling in the data or removing missing rows or columns.

[0011] Several well-developed use cases for data-driven modeling in chemical design center around small-molecule discovery (e.g., using the properties of individual molecules to predict their performance in some system). "Molecular properties" can mean many things, and researchers have explored representing molecules via their physical properties (e.g., boiling point, measured solubility, etc.), their electronic properties (e.g., density functional theory (DFT) descriptors), and their structure (e.g., simplified molecular-input-line-entry-system (SMILES) or graphical representations of chemical structures). Most of this data tends to be relatively dense, continuous variables. In contrast, as discussed above, formalisms have very different data structures.

[0012] To address the problem of very sparse formulation matrices, some previous approaches add more subject matter expertise to the system in the form of descriptors for individual components. If descriptors can be obtained for enough components, formulation descriptors can be calculated for mixtures of components. These can be model-based insights or something as simple as the weighted average viscosity of a fluid blend. Tables of formulation descriptors can be used to add information density to data tables as well as to infuse the model with chemical know-how from subject matter experts, which can result in improved modeling success.

[0013] However, the effort to obtain chemical descriptor data to overcome the sparsity of the formulation data is costly and time-consuming. Furthermore, the first-level formulation data is a sparse matrix with low information density, making it a poor fit for many modeling algorithms. Sparse matrices can be unwieldy and highly susceptible to noise. In some previous examples, it can take months to collect sufficient descriptor information for a complex formulation. Mining component descriptors from lab notebooks or vendor data sheets and / or experimentally measuring individual component properties is resource-intensive. Some such modeling exercises can involve over 80% of the total time spent collecting descriptor data and less than 20% of the total time spent actually modeling. Often, even after the work is completed as practically as possible, the component descriptor set still suffers from significant missing data. Some classes of components may be poorly described, while others have little or no data. Depending on the prevalence of certain components, the missing data can propagate through the formulation calculations, generating large amounts of missing data in the formulation descriptor table. Modeling is more efficient and accurate when the descriptors are complete enough to capture the performance and loading of the ingredients, so that the formulation matrix can be excluded from the training data. However, such an approach can generate a large amount of missing data in the formulation descriptor table.

[0014] Furthermore, including formulation data can be particularly useful for polymeric components, as their descriptors are often insufficient to capture structural distribution, microstructural details, and product-by-product information. While a wide range of descriptors can be generated for small molecules, this is not the case for large molecules such as polymers. Many final product formulations are blends of polymeric components. While bulk properties of polymers, such as molecular weight (Mw), polydispersity (PD), or subject matter expertise properties, such as percent hydroxide (%OH), may be available, these may be insufficient to capture the level of precision required to determine the performance of components in a mixture. In these cases, formulation data remains valuable, as the presence or absence of a component in a formulation still represents information beyond its bulk descriptor.

[0015] At least one embodiment described herein addresses these and other deficiencies by introducing a new way of representing chemical formulations for use in machine learning models. Using relatively inexpensive formulation data, a digital formulation graph can be used to make predictions of chemical product attributes, overcoming sparsity and better training models on sparse data. Describing chemical products through their formulation graphs leverages the availability of formulation data versus ingredient descriptor data. The formulation data is essentially free, since the organization creating the formulation inherently knows which ingredients are used in the formulation. Graph-formatted formulation data can add significant information density to sparse matrices. A graph database can be maintained containing a list of ingredients used in any formulation available to a chemical producer. Including ingredients in the formulation graph input to a machine learning model inherently adds some descriptive information by indicating which ingredients make up the chemical product. A material formulation tree represented as a digital formulation graph can be used as a graph structure for input to graph-based algorithms such as GNNs.

[0016] An ANN can perform machine learning tasks by forming probability weight associations between inputs and outputs. The probability weight associations can be provided by multiple nodes that make up the ANN. The nodes, along with weights, biases, embeddings, and / or activation functions, can be used to generate the ANN's output based on the ANN's inputs. The nodes of an ANN can be grouped to form layers of the ANN. Deep learning is a type of machine learning that has been enabled by improvements in computational power, data availability, and software tools. Deep learning can apply ANNs to accomplish tasks previously thought impossible for computers to perform. The "deep" in deep learning refers to the use of multiple layers in the ANN. These layers extract successively higher-order features from raw inputs. A GNN is an ANN for processing data that can be represented as a digital compound graph. A GNN can use pairwise message passing so that graph nodes iteratively update their representation by exchanging information with neighboring nodes connected by edges.

[0017] Some previous approaches to representing materials using graphs have been done in terms of the final chemical product produced. In other words, the chemical product is produced (either physically or in silico) and then the chemical structure is modeled as a graph. Some such approaches may represent a single small molecule as a graph. Some such approaches use graphs to represent solid materials such as periodic crystals, surfaces, and / or alloys. Some such approaches use graphs to represent polymer distributions as examples of molecular graphs. Such approaches apply a static representation of the final product, rather than the product's composition according to its manufacturing history.

[0018] There are several previous approaches that use graphs and / or trees to describe components in products such as assembled devices (physical products, not chemical products). Some previous approaches describe using reaction graphs to model synthetic routes. However, the scope of such approaches does not include non-reactive compounding materials, nor does it include the utility of the graph as a description of the final product composition, rather than the synthetic chemical route to achieve the product.

[0019] As used herein, the singular forms "a," "an," and "the" include both singular and plural referents unless the context clearly dictates otherwise. Furthermore, the word "may" is used throughout this application in its permissive (i.e., potential, can) sense rather than its obligatory (i.e., must) sense. The term "comprises" and its derivatives mean "including, but not limited to." The term "coupled," unless otherwise specified, means directly or indirectly connected, and can include wireless connections.

[0020] As will be understood, elements shown in the various embodiments herein may be added, interchanged, and / or eliminated to provide additional embodiments of the present disclosure. Additionally, it will be understood that the proportions and relative scales of the elements provided in the figures are intended to illustrate certain embodiments of the present invention and should not be construed in a limiting sense.

[0021] FIG. 1 is an example of a compounding graph 100 for a chemical product (e.g., a compounded rubber product). Compounds may include combinations of ingredients to produce a chemical product. These combinations may include, for example, mixtures, compounds, or other combinations of ingredients formed by chemical reaction, physical reaction, or no reaction (e.g., simple combination). The ingredients that go into a compound may include monomers and / or other ingredients, such as polymers, bases, masterbatches, fillers, dyes, chemicals, and / or manufacturing ingredients. The chemicals and each ingredient in the chemical compound may be represented by a respective node (e.g., node 104-1) in the compounding graph 100. The root node 106 represents the chemical. The remaining nodes each represent a respective ingredient in the formulation. The compounding graph 100 includes edges between each parent node and each child node, indicating that the ingredient represented by the child node was used, in whole or in combination with other ingredients, to form the ingredient represented by the parent node. For example, edge 102 between node 104-1 and node 104-2 indicates that the polymer represented by node 104-2 is a component in the base represented by node 104-1.

[0022] The blend graph 100 has a tree structure with layers. As shown in FIG. 1, successive layers are represented by adjacent columns of nodes. The columns of nodes are not related to the columns of the blend matrix, as described further herein. Different columns of the blend graph 100 are separated by edges between nodes. For example, edge 102 between node 104-1 and node 104-2 indicates the difference between columns 108-1 and 108-2. A given layer (column) contains child nodes that represent the components represented by the parent node in the column immediately to the left of the given column or components that make up the chemical product. The components in column 108-1 represent the first level blend, including the components included in the final combination to form the chemical product.

[0023] The blend graph 100 can be created based on information available to a chemical producer. For example, if a chemical producer uses a manufactured component in a chemical formulation and the producer does not know the composition of the manufactured component, the manufactured component can be included in the graph without including any of the components (e.g., monomers) that led to its production. Node 104-M is an example of a manufactured component with no further chemical composition information included in the blend graph 100. This also makes node 104-M a leaf node of the blend graph 100. Another example of a leaf node is node 104-L, which represents a monomer.

[0024] The blend graph 100 can be structurally represented solely from a list of edges, where each edge is a directional connection between two nodes. For example, edge 102 forms a directional connection from node 104-1 to node 104-2. When stored or represented digitally, the blend graph 100 can be referred to as a digital blend graph 100. Each blend can have its own digital blend graph, with each graph representing a unique chemical product produced by a different blend. Collectively, the digital blend graphs form a dataset describing multiple chemical product formulations.

[0025] A list of embedding vectors may be provided for the blend graph 100, with one embedding vector provided for each node in the blend graph 100. The list of embedding vectors may also be referred to as an embedding matrix. Each embedding vector may be composed of numeric continuous data having a length “N.” In some embodiments, the embedding vector may include an individual index for each unique component represented by a node in any of the blend graphs in the dataset. In some embodiments, the embedding vector may include an index for each of some, but not all, of the unique components represented by a node in any of the blend graphs in the dataset (e.g., components known or suspected not to contribute to chemical product attributes may be omitted from the embedding vector). The value of an index for a particular embedding vector for a particular node may be based on the weight fraction of the components represented by child nodes having edges with the particular node. An index corresponding to a component not represented by a child node having an edge with the particular node may have a value of zero (0). Since each blend graph contains only a subset of all possible unique nodes in the dataset, the size of the embedding matrix for each blend graph is [(number of nodes, N)], and the index of each embedding vector in the embedding matrix maps to a node in the blend graph.

[0026] With reference to FIG. 1 , the embedding vector for node 104-1 may include an index value corresponding to 50% weight percent of the polymer indicated by node 104-2, an index value corresponding to 0.5% weight percent of the polymer indicated by node 104-3, an index value corresponding to 1.5% weight percent of the polymer indicated by node 104-4, an index value corresponding to 2.2% weight percent of the chemical indicated by node 104-5, an index value corresponding to 20% weight percent of the filler indicated by node 104-6, and an index value corresponding to 3.1% weight percent of the polymer indicated by node 104-7. The “index value corresponding to weight percent” means that the digital value stored at that index corresponds to the stated weight percent, even if the digital value is a different (e.g., normalized) number. The remaining index values ​​in the embedding vector for node 104-1 may be initialized as zero, random numbers, or other values. Some additional index values ​​associated with other nodes are shown in FIG. 1 along various edges, although not all index values ​​are shown to avoid obscuring other details shown in FIG. 1 .

[0027] In some embodiments, the embedding vector can include additional indices that can be used to add additional information (other data) about each node, such as processing conditions, chemical descriptors of the components, etc. The chemical descriptors can be obtained, for example, by analytical characterization or computation. In such embodiments, the embedding vector can include two or more indices for each unique component represented by a node in any formulation graph in the dataset, where a first index is based on weight fraction and a second index is based on other data.

[0028] The formulation graph 100 (e.g., as represented by a list of edges) and an embedding vector (embedding matrix) associated with the formulation graph 100 can be input to a trained GNN to generate a feature vector for the formulation graph 100 based on the embedding vector and the architecture of the GNN. Training of the GNN is described in more detail below. The feature vector generated by the GNN is a denser representation of the formulation than the formulation graph 100. The feature vector can be input to a supervised machine learning model trained to predict attribute values ​​of chemical products based on the feature vector. The supervised machine learning model can be a classification or regression layer of the GNN itself, or a separate machine learning model, such as a linear regression model, a logistic regression model, a partial least squares regression model, a random forest model, a support vector machine model, a regularized gradient boosting model, etc. The attribute values ​​of the chemical products can be received from the supervised machine learning model without the need to actually create the chemical products and measure the attribute values.

[0029] The feature vectors can be input into an unsupervised machine learning model that is trained to compare chemical products based on their feature vectors and performance characteristics. Unsupervised learning tasks involve modeling the underlying structure of data without explicit labels (Y data) for each sample. Such methods can identify previously unknown patterns or features within the data. In this example, the unsupervised learning task can provide recommendations for alternative chemical products to a particular chemical product. Some examples of unsupervised learning methods include principal component analysis (PCA), a dimensionality reduction technique that highlights data variance; clustering algorithms such as k-means; similarity analyses such as k-nearest neighbors; association rule mining; and anomaly detection.

[0030] FIG. 2 is a block diagram illustrating an example of training a GNN. The GNN can be trained using a training dataset 220. The dataset 220 can include digital formula graphs of different chemical formulations corresponding to different chemical products that have already been created. Because the chemical products have already been created, known attribute values ​​of the chemical products are available for training the GNN and / or a machine learning model that predicts the attribute values. A respective embedding vector can be provided for each node in each graph, as described herein. Each embedding vector describes the respective node. The embedding vector can have a length at least as long as the amount of unique nodes (unique components) that appear in any digital formula graph in the dataset 220. Each embedding vector for each node can be initialized to a random value, a zero value, or based on other criteria. For example, the embedding vector can be initialized based on the weight fraction of the components represented by the child nodes that have respective edges with the respective node. A list of embedding vectors (e.g., embedding matrices) for the nodes in the graph can be provided for each digital formula graph.

[0031] The GNN can be trained using inputs of batches 222-1, 222-2, 222-(N-1), 222-N of digital blend graphs from dataset 220 and can output respective predicted feature vectors and / or attributes for each digital blend graph (for each chemical product). The GNN can be trained to learn to optimize the embedding vectors of each node and to adjust the weights and biases of the neural network to minimize the error of the GNN's predictions relative to known true values. The GNN's predictions can include feature vectors for each digital blend graph. The GNN's predictions can also include attribute values ​​where the GNN itself predicts them relative to another machine learning model. The GNN can use a process known as message passing to distinguish between the digital blend graphs and find meaningful vectorized representations of each digital blend graph, referred to herein as feature vectors. Examples of message passing functions include simple summation, weighted averaging, and graph attention networks, among others.

[0032] The embedding vectors can be passed through convolutional layers to abstract the characteristics of the digital blended graph. The embedding vector of each node can be aggregated over the embedding vectors of neighboring nodes in each digital blended graph to represent the unique characteristics of the digital blended graph. The aggregation process is part of the generation of feature vectors by the GNN. The aggregation process can be repeated iteratively (e.g., over a quantity "k" of layers) according to the architecture of the neural network. Each layer "k" of the network architecture can include a convolutional layer 224 and an activation function 226. Examples of convolutional layers 224 include graph convolutional networks (GCNs) and graph attention networks. The convolutional layers 224 can employ message passing so that graph nodes iteratively update their representations by exchanging information with neighboring nodes.

[0033] In at least one embodiment, the convolutional layer 224 may be a GCN and may operate according to the following equation:

[0034]

number

[0035] Nodes should not all have the same importance. Therefore, a weighting factor (attention score) can be assigned to each connection between nodes. The connections between nodes are ij where i denotes the node to be updated and j denotes the neighboring node. The embedding vector is represented by h. W is the shared weight matrix. W att is an adjustable weight matrix. For example, if node 1 is connected to each of node 2, node 3, and node 4:

[0036]

number

[0037] In at least one embodiment, the convolutional layer 224 may be a graph attention network convolutional layer and may operate according to the following equation:

[0038]

number

[0039] In at least one embodiment, the activation function 226 may operate according to the following equation: Max(0,x i ) In the formula, x i is the embedding from the convolutional layer.

[0040] The outputs of the k convolutional layers may proceed to a pooling layer 228. The average pooling layer 228 may generalize the outputs of the convolutional layers 224 and the activation functions 226. The average pooling layer 228 may also be referred to as a readout layer, which provides a fixed-length representation of the digital blended graph. The fixed-length representation is referred to herein as a feature vector and will be described in more detail below. An example function of the pooling layer 228 is a global average function:

[0041]

number

[0042] The message passing function, the size of each layer, the number of layers, and the pooling layer are neural network hyperparameters that can be used to adjust the output of the GNN. The output of the pooling layer 228 of the GNN is a vector with a length determined by the pooling layer 228. The output of the pooling layer 228 is a representation of the chemical products generated by the formulation graph input to the GNN. The representation is referred to herein as a feature vector. For each graph input to the GNN, a respective feature vector is generated. The length of the feature vector can be shorter than the length of the embedding vector, and the feature vector can represent the chemical products as a denser representation than the original embedding matrix.

[0043] The feature vectors can be used in supervised machine learning of chemical product attributes. An example of a supervised machine learning model can be included in the GNN itself, as one or more classification or regression layers at the end of the network structure (e.g., a single output linear layer 230). During training, this layer can nonlinearly aggregate the set of feature vectors generated for each digital recipe graph in the dataset 220 to make predictions of chemical product attribute values. Layer 230 can be used to model graph features for chemical product attributes during training to further improve the training of the GNN using iterative weight and bias adjustment. In other words, during training, the chemical product attribute values ​​associated with each graph in the dataset 220 are known, and therefore the GNN can be trained by comparing the output of layer 230 to those known values.

[0044] After training, the single output linear layer 230 can be used as a form of supervised machine learning to predict the attribute value of a chemical product produced by a particular digital recipe graph input to the (already trained) GNN. In the trained model, a single feature vector can be input to layer 230 to generate a single output that is a prediction of the attribute value of the chemical product. Alternatively, rather than being input to layer 230, the feature vector can be input to a different supervised machine learning model to predict the value of the attribute of the chemical product. Examples of such supervised machine learning models include linear regression, logistic regression, partial least squares regression, random forests, support vector machines, and regularized gradient boosting frameworks such as eXtreme Gradient Boosting (XGBoost).

[0045] In some instances, it may be desirable to predict more than one attribute value per digital blend graph. For example, it may be desirable to predict density and tensile strength values ​​for a chemical product. In some embodiments, a GNN may be trained to optimize the feature vector output from the average pooling layer 228 for a single digital blend graph, such that the machine learning model can generate multiple attribute values ​​from a single feature vector. In some embodiments, a GNN may be trained to optimize different feature vectors output from the average pooling layer 228 for a single digital blend graph (e.g., via multiple runs of the GNN with different weights, biases, etc.), each feature vector specific to a particular attribute.

[0046] The digital formulation graph can be used in transfer learning. After a GNN is trained using the digital formulation graph to generate feature vectors useful for predicting attribute values, a transfer function can be determined. The transfer function can predict corresponding attribute values ​​for different attributes (or other information that may be useful for the chemical product formulation) based on the learning that occurred during the training of the GNN and the differences between the original attributes and the different attributes. The transfer function can be used in transfer learning to train a different GNN or to retrain an existing GNN to generate different feature vectors for a different purpose (e.g., predicting different attributes). Such transfer learning training can include adjusting the embedding vectors and weights and biases of the GNN to predict different attributes of the chemical product. Following transfer learning, predictions of different attributes of the chemical product can be received from the GNN based on the input of the chemical product's digital formulation graph.

[0047] A request for a substitute chemical product for a particular chemical product used in a downstream formulation or application can be received (e.g., via a user interface). The feature vector can be used in unsupervised machine learning of substitute chemical products for the particular chemical product. Although not specifically shown, in some embodiments, a single output linear layer 230 can represent an unsupervised machine learning model. The unsupervised machine learning model can be used to compare chemical products based on their feature vectors and performance characteristics. The feature vector of each compared chemical product can be input to the unsupervised machine learning model to generate a recommendation that one of the chemical products is a substitute chemical product for the particular chemical product. For example, the nearest neighbor in feature space can be identified by minimizing the distance between the feature vectors of the target product and the substitute product. The feature vector can be stored in a database in association with the chemical product represented by the feature vector. In some embodiments, the performance characteristics of the chemical product can also be stored in the database. The performance characteristics can have a value that indicates the relative quality of the chemical product for a given formulation or application.

[0048] In some embodiments, the unsupervised machine learning model can be configured to recommend alternative chemical products having performance characteristics that are minimally different from the value of the performance characteristic of the particular chemical product relative to other chemical products in the database. In some embodiments, the unsupervised machine learning model can be configured to recommend alternative chemical products having performance characteristics that are within a predetermined difference from the value of the performance characteristic of the particular chemical product.

[0049] For example, a target product may be Product A, which has a Shore A hardness value of 78.1 and an elongation of 135.1%. It may be desirable to identify alternative products with a Shore A hardness of 75-80 and an elongation of 130-150%, but elongation data for different products may not be available. In the absence of further information, researchers must randomly or empirically select which materials to test for elongation from products with Shore A hardness values ​​of 75-80 (e.g., 37 different products in one exemplary database). Because many products have elongations exceeding 200% or 500%, such a process may take some time to find suitable alternatives within a given performance range. Alternatively, according to at least one embodiment of the present disclosure, a GNN can be trained on the hardness data, and product feature vectors can be obtained. A Euclidean distance matrix between pairs of feature vectors can be generated, and the closest product can be selected as the alternative product. Upon measurement, this most similar product ("Product Z") is found to have an elongation of 140.6% and a hardness of 77.2. Product Z shares no common ingredients with Product A and has not been considered in some previous approaches. However, according to one or more embodiments of the present disclosure, a GNN feature vector is able to identify it as a suitable replacement on the first try.

[0050] Figure 3 is a chart 331 showing example results of chemical product attribute prediction using various approaches. In this particular example, a GNN was trained on a dataset containing 892 digital formulation graphs for 892 different chemical formulations corresponding to different chemical products. Specifically, the chemical products were different rubber materials. The dataset contained 1,667 unique components (nodes) across the 892 formulations. Therefore, the embedding vector for each node was 1 × 1,667 numeric continuous data. Because each graph contained a subset of the 1,667 unique nodes, the starting size of the embedding matrix for each graph was (number of nodes in the graph × 1,667), and the index of each embedding vector in the matrix for a given graph was mapped to a node in the edge list. The embedding vector was initialized with the weight fraction of the child material of the corresponding node. The dataset contained 41,560 nodes and 65,184 edges. The feature vector output from the pooling layer for each graph had a length of 64.

[0051] The GNN was trained to simultaneously learn an improved embedding vector for each node and adjust the neural network weights and biases to minimize the mean squared error of prediction relative to the true performance value. The GNN used seven GATv2 Conv layers, each containing 64 nodes. The output of each layer was (64 times the number of nodes in the graph). As described above with respect to Figure 2, the layers were linked by a ReLU activation function and output to a global average pooling layer.

[0052] Different approaches for predicting chemical attribute values ​​were tested and compared. The predicted chemical attribute values ​​were density 332-1, elongation 332-2, hardness 332-3, plasticity 332-4, and tensile strength 332-5, which are plotted in FIG. 3. The y-axis ("R-squared") of FIG. 3 is a statistical measure of the accuracy of the prediction (e.g., the quality of the fit of the regression model), with a value of 1 being ideal. Results using the first-level recipe matrix in a traditional partial least squares (PLS) regression model for each attribute are shown at 334. For each attribute, the PLS method on recipe data alone showed the weakest results. Results using the addition of descriptors (e.g., expensive X variables as described herein) with a PLS regression model for each attribute are shown at 336. Results using a GNN with a complete recipe tree (e.g., as shown in FIG. 1) for each attribute, according to at least one embodiment, are shown at 338. For each attribute, the results of using a GNN with a first level blend (e.g., as shown by column 108-1 in FIG. 1, instead of a full tree) are shown at 340.

[0053] Adding descriptors to the PLS method improved the results, but it was only better than the GNN method in predicting density 332-1. The GNN method showed excellent results for elongation 332-2, hardness 332-3, plasticity 332-4, and tensile strength 332-5, even without the use of descriptors.

[0054] 4 is an example machine 450 upon which a set of instructions 468 may be executed to cause the machine 450 to perform various methodologies discussed herein. The machine 450 may be connected (e.g., networked) to other machines in a LAN, an intranet, an extranet, and / or the Internet. The machine 450 may operate as a peer machine in a peer-to-peer (or distributed) network environment, or as a server or client machine in a cloud computing infrastructure or environment, or as a server or client machine in a client-server network environment.

[0055] Machine 450 may be a personal computer (PC), tablet PC, set-top box (STB), personal digital assistant (PDA), mobile phone, web appliance, server, network router, switch, or bridge, or any machine capable of executing a set of instructions (sequentially or otherwise) that specify actions to be taken by the machine. Additionally, although a single machine 450 is illustrated, the term "machine" should also be interpreted to include any collection of machines that individually or collectively execute a set (or sets) of instructions to perform any one or more of the methodologies discussed herein.

[0056] The exemplary machine 450 includes a processing device 452, a main memory 454 (e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM), such as synchronous DRAM (SDRAM), or Rambus DRAM (RDRAM)), a static memory 456 (e.g., flash memory, static random access memory (SRAM), etc.), and a data storage system 458 that communicate with each other via a bus 460.

[0057] Processing device 452 represents one or more general-purpose processing devices, such as a microprocessor, central processing unit (CPU), etc. More specifically, processing device may be a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, or a processor implementing other sets of instructions or a combination of sets of instructions. Processing device 452 may also be one or more special-purpose processing devices, such as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), a network processor, etc. Processing device 452 is configured to execute instructions 468 to perform the operations and steps discussed herein. Machine 450 may further include a network interface device 462 for communicating over network 464.

[0058] Data storage system 458 may include a machine-readable storage medium 466 (also known as a computer-readable medium) that stores one or more sets of instructions 468 or software that embody any one or more of the methodologies or functions described herein. Instructions 468 may also reside, completely or at least partially, within main memory 454 and / or within processing device 452 during execution thereof by machine 450, with main memory 454 and processing device 452 also constituting machine-readable storage media.

[0059] In one embodiment, instructions 468 include instructions for implementing functionality corresponding to the GNN and / or unsupervised machine learning models described herein. While machine-readable storage medium 466 is shown to be a single medium in one exemplary embodiment, the term "machine-readable storage medium" should be interpreted to include a single medium or multiple media that store one or more sets of instructions. The term "machine-readable storage medium" should also be interpreted to include any medium that can store or encode a set of instructions for execution by a machine, causing the machine to perform any one or more of the methodologies of the present disclosure. Accordingly, the term "machine-readable storage medium" should be interpreted to include, but is not limited to, solid-state memory, optical media, and magnetic media.

[0060] Although specific embodiments have been described above, these embodiments are not intended to limit the scope of the present disclosure, even if only a single embodiment is described with respect to a particular feature. The example features provided in this disclosure are intended to be illustrative rather than limiting, unless otherwise stated. The above description is intended to cover such alternatives, modifications, and equivalents as would be apparent to one skilled in the art having the benefit of this disclosure.

[0061] The scope of the present disclosure includes any feature or combination of features (either explicit or implicit) disclosed herein, or any generalization thereof, whether or not it alleviates any or all of the problems addressed herein. Various advantages of the present disclosure have been described herein, but embodiments may provide some, all, or none of such advantages, or may provide other advantages.

[0062] In the Detailed Description, some features are grouped together in a single embodiment for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the disclosed embodiments of the present disclosure require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive subject matter lies in less than all features of a single disclosed embodiment. Accordingly, the following claims are incorporated into the Detailed Description herein, with each claim standing on its own as a separate embodiment.

Claims

1. 1. A method comprising: creating a digital formulation graph comprising a plurality of nodes and a plurality of edges based on a chemical formulation; The root node represents the chemical product, each of the remainder of the plurality of nodes represents a respective component of the formulation; each of the plurality of edges being between a respective parent node and a respective child node; providing a respective embedding vector for each respective node; inputting the digital blended graph and the embedding vector into a graph neural network (GNN) trained to generate a feature vector of the digital blended graph based on the embedding vector and the architecture of the GNN; inputting the feature vector into a supervised machine learning model trained to predict an attribute value of the chemical product based on the feature vector; receiving a prediction of the attribute value of the chemical product from the supervised machine learning model.

2. 2. The method of claim 1, further comprising generating the feature vectors with the GNN by passing each embedding vector through multiple convolutional layers, activation functions, and pooling layers of the GNN.

3. 3. The method of claim 2, wherein generating the feature vectors by the GNN comprises aggregating the respective embedding vector of each respective node over the embedding vectors of nodes that have edges with the respective node.

4. 10. The method of claim 1, further comprising training the GNN with a plurality of digital recipe graphs, each of the plurality of digital recipe graphs representing a respective recipe for a respective chemical product having known respective attribute values.

5. each embedding vector includes a plurality of indices, each index corresponding to a respective unique component represented by a node in any of the plurality of digital blending graphs; 5. The method of claim 4, wherein the method includes initializing, for each respective node, the plurality of indices for the respective embedding vector based on a weight fraction of components represented by child nodes having edges with the respective node.

6. the plurality of indexes further comprising a respective first index and a respective second index corresponding to each unique component; the first respective index is based on the weight fraction; The method of claim 5 , wherein the second respective index is based on other data associated with the respective unique component.

7. The method of claim 6 , wherein the other data includes processing conditions.

8. The method of claim 6 , wherein the other data includes chemical descriptors of the components.

9. The method of claim 1 , wherein inputting the digital blended graph into the GNN comprises inputting the list of edges.

10. The method of claim 1 , further comprising receiving predictions of a plurality of attributes of the chemical product from the supervised machine learning model.

11. The method of claim 10 , wherein the plurality of attributes includes at least two of a group of attributes including density, elongation, tensile strength, and hardness.

12. 2. The method of claim 1, wherein inputting the feature vector into the supervised machine learning model comprises inputting the feature vector into a classification or regression layer of the GNN.

13. training the GNN to predict different attributes of the chemical products by transfer learning; adjusting the embedding vectors and weights and biases of the GNN to predict the different attributes of the chemical products; The method of claim 12 , further comprising receiving the prediction of the property of the chemical product from the GNN.

14. inputting the feature vector into the supervised machine learning model; linear regression models, logistic regression model, partial least squares regression model, Random forest model, Support Vector Machine model, and The method of claim 1 , comprising inputting the feature vector to one of a group of supervised machine learning models including a regularized gradient boosting model.

15. creating a digital formulation graph comprising a plurality of nodes and a plurality of edges based on a chemical formulation; The root node represents the chemical product, each of the remainder of the plurality of nodes represents a respective component of the formulation; each of the plurality of edges being between a respective parent node and a respective child node; creating a respective embedding vector for each respective node; operating a trained graph neural network (GNN) on the blended graph and the embedding vector to generate a feature vector of the digital blended graph based on the embedding vector and the architecture of the GNN; and operating a supervised machine learning model to predict an attribute value of the chemical product based on the feature vector.

16. 1. A method comprising: receiving a request for a replacement chemical product for a particular chemical product, the particular chemical product having performance characteristics; creating a digital formulation graph comprising a plurality of nodes and a plurality of edges based on the particular chemical formulation; a root node representing the particular chemical product; each of the remainder of the plurality of nodes represents a respective component of the formulation; each of the plurality of edges being between a respective parent node and a respective child node; providing a respective embedding vector for each respective node; inputting the digital blended graph and the embedding vector into a graph neural network (GNN) trained to generate a feature vector of the digital blended graph based on the embedding vector and the architecture of the GNN; inputting the feature vector into an unsupervised machine learning model trained to compare chemical products based on their feature vectors; receiving a recommendation from the unsupervised machine learning model of one of a plurality of chemical products to be the alternative chemical product.

17. accessing a database storing a respective feature vector for each of said plurality of chemical products; 17. The method of claim 16, further comprising inputting each of the feature vectors into the unsupervised machine learning model.

18. 18. The method of claim 17, wherein the value of the performance characteristic of the alternative chemical product is minimally different from the value of the performance characteristic of the particular chemical product for the plurality of chemical products in the database.

19. 20. The method of claim 17, wherein the value of the performance characteristic of the substitute chemical product is within a predetermined difference from the value of the performance characteristic of the particular chemical product.

20. 16. The method of claim 15, wherein the performance characteristic is a performance characteristic of the particular chemical product in a downstream application.

21. 17. The method of claim 16, wherein the performance characteristic is a performance characteristic of the particular chemical in a downstream formulation.

22. generating the feature vectors using the GNN by passing each embedding vector through a plurality of convolutional layers, an activation function, and a pooling layer of the GNN; 17. The method of claim 16, wherein generating the feature vectors by the GNN comprises aggregating the respective embedding vector of each respective node over the embedding vectors of nodes that have edges with the respective node.

23. 17. The method of claim 16, wherein the method includes initializing the respective embedding vectors based on weight fractions of components represented by child nodes that have edges with the respective node.

24. 17. The method of claim 16, wherein inputting the digital blended graph into the GNN comprises inputting the list of edges.

25. 17. The method of claim 16, wherein the unsupervised machine learning model is trained to compare chemical products based on the feature vectors and performance characteristics of the chemical products.

26. receiving a request for a replacement chemical product for a particular chemical product, the particular chemical product having performance characteristics; creating a digital formulation graph comprising a plurality of nodes and a plurality of edges based on the particular chemical formulation; a root node representing the particular chemical product; each of the remainder of the plurality of nodes represents a respective component of the formulation; each of the plurality of edges being between a respective parent node and a respective child node; creating a respective embedding vector for each respective node; operating a trained graph neural network (GNN) on the blended graph and the embedding vector to generate a feature vector of the digital blended graph based on the embedding vector and the architecture of the GNN; a non-transitory machine-readable medium storing executable instructions to operate an unsupervised machine learning model to recommend one of a plurality of chemical products as the substitute chemical product based on the feature vector of the digital recipe graph; A non-transitory machine-readable medium, wherein the unsupervised machine learning is trained to compare chemical products based on their feature vectors.

27. creating a digital formulation graph comprising a plurality of nodes and a plurality of edges based on the particular chemical formulation; a root node representing the particular chemical product; each of the remainder of the plurality of nodes represents a respective component of the formulation; each of the plurality of edges being between a respective parent node and a respective child node; creating a respective embedding vector for each respective node; operating a trained graph neural network (GNN) on the blended graph and the embedding vector to generate a feature vector of the digital blended graph based on the embedding vector and the architecture of the GNN; and operating a machine learning model to predict an attribute value of the chemical product or recommend one of a plurality of chemical products as an alternative chemical product based on the feature vector.

Images