System and method for predicting and optimizing chemical mixtures

Abstract

The disclosed invention is an end-to-end differentiable mixture physics model that combines physics-based models and GDL models, where the GDL models are designed to transform the molecular species, compositions and environment conditions to physical coefficients for use in the physics-based models.

Description

Attorney Docket: 8350.2022-254WO APPLICATION FILED UNDER THE PATENT COOPERATION TREATY AT THE UNITED STATES RECEIVING OFFICE for System and Method for Predicting and Optimizing Chemical Mixtures Applicant Carnegie Mellon University Inventors Vankatasubramanian Viswanathan Shang Zhu Bharath Ramsundar Prepared By: KDWFIRM.COM Dennis M. Carleton, Principal KDW Firm PLLC 2601 Weston Pkwy. Suite 103 Cary, NC 27513 919-396-5643Attorney Docket: 8350.2022-254WO System and Method for Predicting and Optimizing Chemical MixturesRelated Applications

[0001] This application claims the benefit of U.S. Provisional Patent Application No. 63 / 525,925, filed July 10, 2023, the contents of which are incorporated herein in their entirety. Government Interest

[0002] The invention was created with support of the U.S. Government under contract DE- AR0001211 granted by the U.S. Department of Energy. The U.S. Government has certain rights in the invention. Background

[0003] Chemical mixtures are widely used in chemical processes and devices such as energy storage and conversion, chemical reactions and catalysis, and environmental engineering. Often, the mixture chemistry and compositions are carefully designed to achieve higher device performances. For example, battery electrolytes, as mixtures of salts and solvents, have been optimized to facilitate ion transport, prevent electron transfer, and stabilize electrode-electrolyte interfaces for an energy-dense and durable battery system.

[0004] The design and optimization of chemical mixtures, and, in particular, electrolyte mixtures, remain challenging due to the complexity of mixture chemistry andAttorney Docket: 8350.2022-254WO compositions, as well as the high experimentation cost. Physics-based modeling offers a solution by probing the underlying mixture physics and rationalizing the design principles for high-performing mixtures. Among physics-based mixture modeling techniques, molecular simulation is a powerful tool to study the interactions and dynamic evolution inside a complex mixture system, but it can be limited to time and length scales due to its high computational costs. Alternatively, chemical physicists proposed empirical function relationships to describe mixture physics. For example, Redlich-Kister (R-K) polynomials were designed for modeling mixture thermodynamics, and the Arrhenius equation was proposed to describe the temperature dependence of chemical reactions and other dynamic behaviors.

[0005] Although existing physics-based modelling may provide decent model accuracy and indicate intrinsic physical behaviors such as reaction energy barriers, these empirical relationships are lacking in predictive power when new chemical species are provided. Summary of the Invention

[0006] Disclosed herein is a machine learning-based model which leverages the geometric deep learning (GDL) method for the modeling and optimization of chemical mixtures wherein, using GDL, necessary geometric priors are applied as constraints on the model space to improve model efficiency. Note that, although the invention is explained herein in terms of modeling and optimization of battery electrolyte mixtures, the invention is generally applicable to all types of chemical mixtures.Attorney Docket: 8350.2022-254WO

[0007] The disclosed invention is an end-to-end differentiable mixture physics model that combines a mixture physics basis and GDL models, where the GDL models are designed to transform the molecular species, compositions and environment conditions, to physical coefficients. Lattice-based mixture theory and a Vogel Fulcher Tammann (VFT) model are selected for mixture thermodynamic laws and transport laws, respectively. Graph neural networks (GNNs) are used to encode molecular species and compositions and predict the physical coefficients inside the selected mixture physics basis, with the component-wise permutation invariance preserved. Brief Description of the Drawings

[0008] FIG.1 is a schematic view of the architecture of the model of the present invention.

[0009] FIG.2 is a schematic representation of the graph deep learning portion of the model showing fully connected neural networks used for per-substance and pairwise interaction embeddings.

[0010] FIGS.3(a-d) show proof of concept by comparing differentiable battery electrolyte optimization using the model of the present invention and optimization by robotic experimentation. Detailed Description

[0011] The disclosed build aTaking the input of chemical graphs, compositions, and environment conditionAttorney Docket: 8350.2022-254WO vector ^ ^^, ^^, ^^^, the model processes with two components: geometric deep learning ^^ఏ^ ^^, ^^, ^^^ and physics laws, ^^^∙, ^^, ^^^. The model outputs mixture property ^^^ൌ ^^^ ^^ఏ, ^^, ^^^ in an end-to-end differentiable framework.

[0012] Physics Models are used for the thermodynamics of mixing and ion transport. The selection of physics models, ^^^∙, ^^, ^^^, depends on the mixture properties of interest. For explanation purposes herein, the mixing thermodynamics of binary non- electrolyte mixtures and ion transport of multicomponent electrolyte mixtures are used as examples. As would be realized, these can be further generalized to other forms. In onecharacteristics to the ion transport characteristics of the chemical mixture.

[0013] To describe the thermodynamics of mixing of non-electrolyte mixtures, a polynomial expansion can be used for representing the excess function of mixing ∆ ^^^(i.e., the difference between mixing thermodynamic quantity ^^^and the linear combination of each component ^^^^^^^^^, where ^^^is the property of the species ^^). It has been successfully applied in differentiable thermodynamic modeling, with the Redlich- (R-K) polynomial being a popular choice: ^^ ^^ ൌ ^^ ^ ^^ ^^ ^^ே ^ ^^ ^ழ^ ^ ^ ^ୀ^ ^^ோ^,^^ ^ ^^^ െ ^^^^ ^(1)where:Attorney Docket: 8350.2022-254WO ^^^and ^^^are mole fractions of species ^^ and ^^; ^^ோ^^,^^is the R-K polynomial coefficients between the two species and with order number ^^.

[0014] This preserves the permutation invariance of chemical species ^^ and ^^, when the odd orders of polynomials follow the parity rule of permutation. The mixture thermodynamic property ^^^can be further obtained by: ^^ ൌ ^^ ^ ^^ ^^ ^^ே ^^^ ^ ^ ^ ^^ ^ழ^ ^ ^ ^ୀ^ ோ^,^^ ^^ െ ^^^^ ^ ^ ^^^ ^^^ ^^(2)

[0015] This preserves permutation invariance over mixture components and can be applied to a wide range of mixture thermodynamic properties.

[0016] For the ion transport properties, the ionic conductivities of battery electrolytes are the focus characteristic. A higher ionic conductivity will reduce the ion transfer resistance between electrodes and lessen the formation of electrolyte concentration polarization, therefore enabling fast-charging battery applications. Here, the Vogel– Fulcher–Tammann (VFT) model is selected to capture the temperature dependence as: ^ ^^ି మ^ ൌ ^^^^^்ି^య(3)where:^^ is the temperature; and^^^^^ is a set of physicalAttorney Docket: 8350.2022-254WO

[0017] Geometric deep learning is used to learn mixture representations. Thermodynamic and ion transport laws conventionally rely on empirically fitting experimental data to obtain physical coefficients, ^ ^^ோ^, ^^^^. However, the function relationship between mixture input ^ ^^, ^^, ^^^ and physical coefficients ^ ^^ோ^, ^^^^ remains unknown. The GDL component is therefore introduced to replace the physical coefficients with learnable GDL functions, ^ ^^ோ^, ^^^^ ൌ ^^ఏ^ ^^, ^^, ^^^. In this way, the mixture physics model now becomes predictive and fully differentiable from chemical structures to properties. As shown in first step in the GDL component is a graphconvolution transformation 106 over each component graph ^^^to obtain the graph- level feature vector ^^′^, for component ^^. In the second step, ^^′^is attached with compositions and environment conditions and processed by the model 108 shown in FIG.2 to mixture component vector ^^^ൌ ^connected neural networks (FCNN), 202, 204, to learn the per-substance and pairwise-interaction ^ ^^^^ ^ ^^ி^ேே ଶ^ଶ^ ^^^^^ 208 and ^ ^^^^^ ൌ ^ ^^ி^ேே ଶ^ସ^ ^^^^^ 210,respectively, where ^^ி^ேே ଶ^ଶ^∙^and ^^ி^ேே ଶ^ଶ^∙^ represent the neural networktransformations. Depending on the mixture physics laws, ^ ^^^^ 208 and ^ ^^^^^ 210 are combined in a certain form to produce the physical coefficients. The physical coefficients ^ ^^ோ^, ^^^^ are then applied as inputs to physics laws 110. For the VFT model in battery electrolyte mixtures given by Eq. (3), the mixture feature vector ^^ 212 is created via a pooling operator ⊕, ^^ ൌ ^ ^^^^ ⊕ ^ ^^^^^ ൌ ^ ^^^^^^⋅ ^^^, ^^^^^^^^^^⋅ ^^^^^, which provides a permutation-invariant map from ^ ^^^^ and ^ ^^^^^ to ^^ byAttorney Docket: 8350.2022-254WO concatenating the weighted sums of substance and pair embeddings. The physical ^ ^^^^, is a function of ^^ via another FCNN, 206, ^ ^^^^ ൌ^^^ி^ேே ଶ^^^^^^^. When no physics laws are applied, FCNN 206 simply outputs themixture property.

[0018] A special case exists in the mixing law of thermodynamics and R-K polynomial-based model in Eq. (2), due to the intrinsic per-substance dependence of ^^^and pairwise interaction dependence of ^ ^^ோ^^,^^^. They can be modelled directly from FCNN 202 and FCNN 204 without additional pooling operations or FCNN 206. The implementation is as follows: ^^^^^ൌ^^^^ଶ^^^^^^^^ห ^^ோ^^,^^ห^ ൌ ^ ^^^ଶோ^൫ห ^^^, ^^^ห൯ ^ ^^^ଶோ^൫ห ^^^, ^^^ห൯⁄ 2 ^ where: ^^^ଶ^^∙^ and ^^^ଶோ^^∙^ are two separate FCNNs to learn the pure component property given substance embeddings and RK coefficients given pairs of substance embeddings, respectively.

[0019] An alternative implementation is to replace concatenated substance embeddings ^ ^^^, ^^^൧ with ^^^^. The mean pooling of permuted sequences is introduced via dividing the ^^^ଶோ^^∙^summation by 2 such that ห ^^ோ^^,^^ห ൌ ห ^^ோ^^,^^ห, where the absolute value symbol is added. Since the RK coefficients follow permutation parity, ^^ோ^^,^^ൌAttorney Docket: 8350.2022-254WO ^^ோ^^,^^∙ ^െ1^^. ^^ோ^^,^^ൌ ห ^^ோ^^,^^ห when chemical ^^’s molecular weight is smaller than chemical ^^’s. No additional pooling operations are required.

[0020] Note that the geometric deep learning concept refers to the approaches that incorporate appropriate geometric priors (i.e., information on the structure space and symmetry properties of input signals) and that in the GDL component, permutation invariance is preserved over components with the mixture pooling operator (⊕) in the VFT model and the intrinsic permutation invariance introduced in the R-K model. The preserved permutation invariance improves the model robustness by outputting identical mixture properties when mixture component sequences are permuted. This can be confirmed by reversing the co-solvent

[0021] For the disclosed model, the atom features considered include one hot encoding of atom type, number of heavy neighbors, formal charges, hybridization type, chirality, and number of implicit hydrogens, and numerical information on ring structures, aromaticity, atomic mass, VdW radius, and covalent radius, giving a 97-dimension feature vector. Note that no bond features are incorporated in the model, but these features could be added. The graph convolution cell is made up of 3 graph convolutional layers, each of which is followed by a ReLU and dropout layers (dropout rate, p=0.25). The whole graph convolution cell ends up with a global mean pooling layer and provides graph-level embeddings for component molecules ^^′^. These graph-level embeddings are then concatenated with compositions and environment conditions.Attorney Docket: 8350.2022-254WO

[0022] In one embodiment, FCNN 202 has dimensions of ^ ^^, ^^, ^^, ^^^, FCNN 204 has dimensions of [2 ^^, 2 ^^, 2 ^^, ^^^ and FCNN 206 has dimensions of ^2 ^^, 4 ^^, 2 ^^, 3^, where ^^ ൌ 256 ^ 1 ^ ^^ ^^ ^^ ^^ and ^^ ^^ ^^ ^^ is the dimension of the environment conditions. With the VFT model selected, FCNN 206 outputs the three physical coefficients in the VFT model. In terms of the thermodynamics of mixing, FCNN 202 and FCNN 204 are reused to obtain the component-wise physical parameters ^ ^^^^ and pair-wise physical parameters ^ ^^ோ^^,^^^. Instead of creating a FCNN 204 for each order ^^, it was designed as a multitask neural network, which shares parameters before the output layers.

[0023] During training, the learning rate is set as 0.001 with a weight decay rate of 10ିସin PyTorch Adam optimizer. An L1 loss function is used for loss backpropagation. Early stopping criteria was applied to select the epoch with the lowest validation error to avoid overfitting. The ionic conductivity labels went through a logarithm transformation before computing the loss values to distinguish mixture properties that span multiple orders of magnitudes. All models may be implemented with PyTorch and PyTorch Geometric. For each mixture property, the full dataset was randomly split into training, validation, and testing sets, in the ratio of 8: 1: 1. The cross-validation results were reported based on an ensemble of 5 models with randomly varying splits.

[0024] Fast charging of Li-ion batteries is impacted by electrolyte ionic conductivities, and electrolyte optimization can be challenging for battery design due to highAttorney Docket: 8350.2022-254WO experimentation costs. As a proof of concept of the trained model disclosed herein, its capability to evaluate ionic conductivities and design electrolyte mixtures for high-performing Li-ion batteries was tested. Three types of electrolyte solutions were selected as test cases and evaluate their ionic conductivities at 30◦C and varying co-solvent compositions. They are LiPF6 salt in solvent mixtures of (i) cyclic carbonates, including ethylene carbonates (EC), propylene carbonates (PC) and fluorinated ethylene carbonates (FEC), (ii) linear carbonates, including ethyl methyl carbonates (EMC), diethyl carbonates (DEC) and dimethyl carbonates (DMC) and (iii) cyclic and linear carbonates, including EC, PC, and DMC. We first show the ionic conductivity landscape of (i) is shown in FIG.3(a) by varying co-solvent compositions with fixed lithium mole fractions of 0.08, where a moderate ionic conductivity peak up to 8 (mS / cm) was observed in the EC-enriched region. Note that the anions and cations are treated separately when computing the mole fraction. FIG.3(b) provides the conductivity landscape of electrolyte mixture (ii), where the highest κ values are observed in the DMC-enriched region. Here, the lithium mole fraction was fixed at 0.12 due to the low dielectric constants of linear carbonates and thus the low dissociation degree of lithium salts. According to the conductivity map of the electrolyte mixture (iii) shown in FIG.3(c), adding linear carbonate molecules into cyclic carbonate solvents can significantly increase the mixture ionic conductivities, where the maximum ionic conductivity is 14.39 (mS / cm) when PC:DMC:EC mole ratio is close to 0:0.70:0.30 with a fixed lithium-ion mole fraction of 0.08. It is worth noting that the training data is produced with a temperature lower than or equal toAttorney Docket: 8350.2022-254WO 20◦C (maximum is 12.6 (mS / cm)), but its generalization capability is tested at 30◦C. This result is verified with the output of our data generator, AEM, which provides the highest conductivity of 14.2 (mS / cm) at 0.082 to 0.085 lithium mole fraction with the given PC:DMC:EC ratio. This agrees well with the differentiable modeling result and indicates good generalization of the disclosed model.

[0025] The gradient information is readily accessible by differentiating the trained model. To illustrate that, the gradient vectors are shown as arrows in the ionic conductivity landscapes in FIGS.3(a-c). In FIG.3(a) and FIG.3(c), large gradients are observed at pure EC solvent area, indicating that adding a small number of co-solvents can significantly improve the ionic conductivity. This can be explained by EC’s being solid-like at room temperature. Another interesting observation is that pure DMC solvent area in FIG.3(c) displays a much higher gradient than that in FIG.3(b), with which the conclusion can be made that adding a small quantity of high-polarity cyclic carbonate solvents (EC, PC) could enable higher ionic conductivities. The gradient field map is generated instantaneously by auto-differentiation leveraging training data generated by AEM in the trained model, which saves expensive experimental costs by reducing the number of optimization steps. Based on the gradient information provided by the model, a gradient-ascent algorithm is implemented by initializing a batch of four starting points in the mixture space and increasing the objectives iteratively following the gradient directions. From FIGS.3(a-c), the optimization algorithm robustly identified local maximum spots. This differentiable optimization framework further guides the robotic experimentation performed byAttorney Docket: 8350.2022-254WO the hardware setup (Clio). The batch of four optimization trajectories in FIG.3(c) were extracted and the ionic conductivities evaluated at each step by both the model and Clio were compared, as shown in FIG.3(d). The model (labelled as DiffMix in FIG.3(d)) and Clio results show a good agreement between simulation and experimentation, and at least an 18.8% increase from the initial ionic conductivities is shown. Note that the temperature of Clio is managed at around 27◦C, which is slightly varied from the model temperature of 30◦C. This may account for deviations between experimental and simulation data. Across FIGS.3(a-d), the focus was on varying solvent composition space while fixing lithium mole fractions. The fixed lithium mole fractions are selected for demonstration purposes and are close to the optimal lithium concentrations in the training data generated at lower temperatures (20◦C). An improved ionic conductivity was realized by fixing solvent compositions and optimizing salt concentrations to prove the generalizability of the framework. These results elucidate the capability of differentiable modeling of battery electrolytes, with which we could efficiently explore the chemical space of multi-component electrolyte mixtures.

[0026] Disclosed herein is a GDL-based differentiable model for chemical mixtures, that combines the advantages of physics-based models and geometric deep learning. The disclosed model expands the modeling and optimization techniques of battery electrolyte mixtures by unifying physics laws and geometric deep-learning in a differentiable framework. The model preserves the component-wise permutation invariance and enables more accurate and robust predictions than prior art models,

Claims

Attorney Docket: 8350.2022-254WO and, in particular, GNN-only models. At high temperatures, the model also provides ionic conductivity predictions showing superior accuracy over prior art models, due to the incorporation of a temperature-dependent VFT model. [0027] As would be realized by those of skill in the art, other physics models may be used in combination with the GDL-based portion, depending on et specific chemical mixtures being modelled. Further, specific descriptions of FCNNs are meant as examples only and are not to be considered limitations on the invention. Other configurations of FCNNs may be used and are considered to be within the scope of the invention.Attorney Docket: 8350.2022-254WO Claims 1. A system comprising: one or more physics-based models describing thermodynamic laws and transport laws for a chemical mixture; and a machine learning model taking as input a chemical composition and producing a set of physical coefficients for use as inputs to the one or more physics-based models; wherein the system outputs one or more mixture properties for the chemical mixture.

2. The system of claim 1 wherein the chemical composition input comprises: one or more chemical graphs, each graph describing a component of the chemical mixture; a composition of the chemical mixture; and a vector describing environment conditions.

3. The system of claim 2 wherein the machine learning model is a graph neural network.

4. The system of claim 3 wherein the graph neural network comprises:Attorney Docket: 8350.2022-254WO a graph convolution transformation to extract a feature vector from the graphs of each chemical components of the chemical mixture; and a mixture-level machine learning model taking as input the graph feature vectors, the compositions of the chemical mixtures and the environmental conditions and outputting the physical coefficients of the physics-based models.

5. The system of claim 5 wherein the mixture-level machine learning model comprises: a first fully-connected neural network to generate per-substance embeddings from the input; and a second fully-connected neural network to generate pairwise-interaction embeddings from the input; wherein the per-substance embeddings and the pairwise-interaction embeddings are combined to create the physical coefficients of the physics-based models.

6. The system of claim 5 further comprising: a third fully-connected neural network taking as input a combination of the per- substance embeddings and the pairwise-interaction embeddings and outputting the physical coefficients of the physics-based models.Attorney Docket: 8350.2022-254WO 7. The system of claim 5 wherein the per substance embeddings comprise a feature vector for each substance in the chemical mixture.

8. The system of claim 5 wherein the pairwise-interaction embeddings comprise feature vectors for each pair of substances in the chemical mixture.

9. The system of claim 6 wherein the combination of the per-substance embeddings and the pairwise-interaction embeddings is a mixture feature vector; 10. The system of claim 9 wherein the mixture feature vector is a pooling of the per substance embeddings and the pairwise-interaction embeddings.

11. The system of claim 10 wherein a mapping from the per substance embeddings and the pairwise-interaction embeddings to the mixture feature vector is permutation-invariant and is created by concatenating weighted sums of the per substance embeddings and the pairwise-interaction embeddings.

12. The system of claim 1 wherein the thermodynamic characteristics of the chemical mixture are modelled using a Vogel-Fulcher-Tammann model.Attorney Docket: 8350.2022-254WO 13. The system of claim 1 wherein the transport characteristics of the chemical mixture are modelled using Redlich-Kister polynomials.

14. A method comprising: receiving a chemical composition describing a chemical mixture; inputting the chemical composition to a machine learning model which produces a set of physical coefficients; inputting the set of physical coefficients to one or more physics-based models; and outputting one or more mixture properties of the chemical mixture wherein the one or more physics-based models describe thermodynamic laws and transport laws for a chemical mixture.

15. The method of claim 14 wherein the chemical composition comprises: one or more chemical graphs, each graph describing a component of the chemical mixture; a composition of the chemical mixture; andAttorney Docket: 8350.2022-254WO a vector describing environment conditions.

16. The method of claim 15 wherein the machine learning model is a graph neural network, the method further comprising: performing a graph convolution transformation to extract a graph feature vector from the graphs of each chemical component of the chemical mixture; and inputting the graph feature vectors from each chemical component, the compositions of the chemical mixtures and the environmental conditions to a mixture- level machine learning model; and outputting the set of physical coefficients of the physics-based models.

17. The method of claim 16 wherein the mixture-level machine learning model comprises: a first fully-connected neural network to generate per-substance embeddings from the input; and a second fully-connected neural network to generate pairwise-interaction embeddings from the input; wherein the per-substance embeddings and the pairwise-interaction embeddings are combined to create the physical coefficients of the physics-based models.

Images