Generating predictions relating to chemical processes using machine learning models

Abstract

In some embodiments, a computer-implemented method of generating a prediction related to a chemical workflow that includes a process performed on at least one chemical is provided. A computing system encodes a representation of the at least one chemical using a chemical encoder to create at least one latent chemical representation. The computing system creates a latent experiment representation based on the at least one latent chemical representation. The computing system decodes the latent experiment representation using an experiment decoder to predict one or more properties for an output of the chemical workflow, to generate an updated representation of the at least one chemical, or to generate an updated representation of the process.

Description

GENERATING PREDICTIONS RELATING TO CHEMICAL PROCESSES USINGMACHINE LEARNING MODELSCROSS REFERENCE(S) TO RELATED APPLICATION(S)

[0001] This application claims the benefit of Provisional Application No. 63 / 590341, filed October 13, 2023, the entire disclosure of which is hereby incorporated by reference herein for all purposes.BACKGROUND

[0002] Machine learning is a rapidly advancing field. Recently, new technologies for processing text have been created that allow for the prediction of novel text outputs based on input prompts. These technologies are often based on artificial neural networks, and use architectures such as transformers. At a high level, transformers are commonly trained on a corpus of text, and encode the corpus into a latent representation. A decoder is then commonly used to generate novel text from the latent representation.

[0003] While the ability to generate novel output based on an artificial understanding of a training corpus has significant utility, these tools and techniques have been underutilized in areas outside of text processing. What is desired are techniques for using machine learning models such as these to understand and generate content in domains other than language processing.SUMMARY

[0004] This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This summary is not intended to identify key features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.

[0005] In some embodiments, a computer-implemented method of generating a prediction related to a chemical workflow that includes a process performed on at least one chemical is provided. A computing system encodes a representation of the at least one chemical using a chemical encoder to create at least one latent chemical representation. The computing system creates a latent experiment representation based on the at least one latent chemical representation. The computing system decodes the latent experiment representation using an experiment decoder to predict one or more properties for an output of the chemicalworkflow, to generate an updated representation of the at least one chemical, or to generate an updated representation of the process.

[0006] In some embodiments, a non-transitory computer-readable medium having computer-executable instructions stored thereon is provided. The instructions, in response to execution by one or more processors of a computing system, cause the computing system to perform a method as described above.

[0007] In some embodiments, a computing system configured to perform a method as described above is provided.BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The foregoing aspects and many of the attendant advantages of this invention will become more readily appreciated as the same become better understood by reference to the following detailed description, when taken in conjunction with the accompanying drawings, wherein:

[0009] FIG. 1 is a schematic illustration of a non-limiting example embodiment of a system for using machine learning to predict properties of chemical processes according to various aspects of the present disclosure.

[0010] FIG. 2 is a block diagram that illustrates aspects of a non-limiting example embodiment of a material property prediction computing system according to various aspects of the present disclosure.

[0011] FIG. 3 is a schematic illustration of a non-limiting example embodiment of an encoder / decoder that processes workflow information, according to various aspects of the present disclosure.

[0012] FIG. 4A - FIG. 4C are schematic illustrations of techniques used for predicting properties of an output of a chemical process, according to various aspects of the present disclosure.

[0013] FIG. 5 is an illustration of a non-limiting example embodiment of a structured chemical representation according to various aspects of the present disclosure.

[0014] FIG. 6A and FIG. 6B are an illustration of a non-limiting example embodiment of a structured process representation according to various aspects of the present disclosure.

[0015] FIG. 7A and FIG. 7B are a flowchart that illustrates a non-limiting example embodiment of a method of predicting properties of an output of a chemical process according to various aspects of the present disclosure.

[0016] FIG. 8 is a schematic illustration of a non-limiting example embodiment of an encoder / decoder in which context information is optionally encoded and injected into the encoding / decoding processing at various points, according to various aspects of the present disclosure.

[0017] FIG. 9A - FIG. 9C illustrate three non-limiting example techniques that may be used for latent space fusion, according to various aspects of the present disclosure.

[0018] FIG. 10 illustrates a non-limiting example embodiment of an agent network according to various aspects of the present disclosure.DETAILED DESCRIPTION

[0019] Through the storage of massive amounts of chemical experiment data generated by research and development efforts, embodiments of the present disclosure are able to train machine learning models to generate latent representations of chemicals and processes. These latent representations encode fundamental aspects of the chemicals and processes such that predictions can be generated for chemicals and the chemical properties of outputs of novel experiments. That is, embodiments of the present disclosure can predict the chemicals and properties of the chemicals generated by experiments that have not been conducted before. This can reduce or eliminate the need to physically conduct experiments on every potential combination of chemicals, combination of reaction process parameters, or chemical products, for addressing a given chemical product or chemical process goal.

[0020] In some embodiments, the present disclosure provides one or more machine learning models designed to consume structured and / or unstructured data which describes a full end-to-end formulary and synthetic chemistry workflow (e.g. combinations of raw materials, the parameters describing the stages / steps of how to combine / process the raw materials into intermediates, any stages / steps used to turn intermediates into a final material, and / or any final processes for converting the final material into a form on which characterization techniques can be applied to understand the material performance), and to predict properties of the final material or chemical composition. In some embodiments, similar models may be designed to generate novel workflows (or to adjust existing workflows) to accomplish a stated goal, simply by providing a structured or unstructured description of the desired outcome.

[0021] The present disclosure provides machine learning architectures which create latent representations that encode a chemical workflow in order to generate predictions of chemical product material properties and / or make recommendations and discoveries of novel workflows and / or chemical products based on those predictions. In someembodiments, the general technique involves converting a description of a chemical workflow into a series of intermediate representations (latent representations) that encode pertinent information about each component or process variable in the workflow in a form that is usable for predicting the final material properties. The latent representation itself is learned from stored experimental data.

[0022] Depending on the type of parameter being encoded, different techniques for deriving the latent representation may be used. As a non-limiting example, when considering a chemical compound that is present in the workflow, if the molecular structure is known, then a machine learning model may be used that encodes that chemical structure, or aspects of that chemical structure, into a latent representation having a vector of some dimension D, where D is a hyperparameter of the model determined at training time. This model may be designed to encode any possible chemical compound, or any chemical compound comprising requisite chemical moieties, that might be used during the synthesis or chemical process into the same latent space so that all chemical compounds, regardless of their initial molecular structure, can be analyzed by subsequent models in the same space of chemical representations. In the case of chemical process parameters, a similar technique may be employed which converts different processes and parameters and / or groups of processes and parameters into latent representations which can be used in subsequent machine learning models to reason about the interaction of those processes and parameters in a learned latent space.

[0023] FIG. 1 is a schematic illustration of a non-limiting example embodiment of a system for using machine learning to predict properties of chemical processes according to various aspects of the present disclosure. In the system 100, a material property prediction computing system, or MPP computing system 102, is configured to receive information regarding experiments from one or more laboratories, including a first laboratory 104, second laboratory 106, and third laboratory 108. The illustration of three laboratories is a non-limiting example only, and in some embodiments, more or fewer laboratories may provide information to the MPP computing system 102.

[0024] Each of the laboratories 104, 106, 108, or a combination thereof, may be associated with one or more computing systems configured to record aspects of experiments executed by the laboratory’ (e.g., chemicals used, process steps applied, parameters for the process steps, measurements of properties of the chemical compound products / results of the experiments), and to report these aspects to the MPP computing system 102 using structured, semi-structured, or unstructured representations. In some embodiments, instead of having separate computing systems at each of the laboratories 104, 106, 108, one or moreof the laboratories 104, 106, 108 may record the aspects of the experiments directly to the MPP computing system 102, via a user interface provided by the MPP computing system 102. By collecting information regarding experiments from a plurality of laboratories, the MPP computing system 102 can generate data that can be used to train its machine learning models, and thereby create more accurate predictions of chemical product properties and chemical products from novel experiments.

[0025] In some embodiments, the first laboratory' 104, second laboratory 106, third laboratory 108, or combinations thereof, may be controlled by a single entity, and so the MPP computing system 102 may pool information from any or all of the laboratories together to form a single set of training data. This can enable the MPP computing system 102 to provide significant benefits to the single entity. Individual laboratories controlled by the single entity can benefit from predictions generated by the MPP computing system 102 using the collective information generated by any or all of the laboratories controlled by the single entity. In large organizations this can be a particularly significant benefit, as multiple laboratories across the organization may not be otherwise equipped to share data or experimental results in this way, and each of the multiple laboratories across the organization may not separately generate enough information to allow the machine learning models to be effectively trained.

[0026] In some embodiments, at least one of the first laboratory' 104, second laboratory 106, and third laboratory 108 may be controlled by different entities. In some such embodiments, the MPP computing system 102 may compartmentalize information received from laboratories controlled by different entities and train separate machine learning models for each entity to prevent proprietary information from being identified between entities. In other such embodiments, the MPP computing system 102 may train at least some of its machine learning models on all of the received information regardless of source due to a low likelihood of proprietary information transfer, or due to a preference by the entities to balance the risk of proprietary' information transfer in order to have access to a larger amount of training data.

[0027] In some embodiments, the MPP computing system 102 may also communicate with other components within the system 100 in order to further advance its functionality'. For example, the MPP computing system 102 may transmit instructions to, and receive experimental results from, one or more automated laboratories 112. These instructions may include identities and amounts of one or more chemicals to be used, one or more chemical reaction / processing steps, and / or the parameters associated with each chemical process. Theexperimental results may include measurements of one or more properties of the chemical product / result of the chemical process.

[0028] In some embodiments, the MPP computing system 102 may further communicate with a reference data store 114. The reference data store 114 may include further information about chemicals that are otherwise identified by a chemical identifier, such as a Chemical Abstracts Service (CAS) number or other unique identifiers. The reference data store 114 may include information such as capabilities, configurable settings, and so on, about one or more machines used to execute process steps that may be looked up by an identifier, such as a model number or serial number.

[0029] In some embodiments, the system 100 may also include one or more end user computing devices 110. The end user computing devices 110 may be used to access a user interface provided by the MPP computing system 102 for inputting information, viewing results, or any other purpose.

[0030] FIG. 2 is a block diagram that illustrates aspects of a non-limiting example embodiment of a material property prediction computing system (MPP computing system) according to various aspects of the present disclosure. The illustrated MPP computing system 102 may be implemented by any computing device or collection of computing devices, including but not limited to a desktop computing device, a laptop computing device, a mobile computing device, a server computing device, one or more computing devices of a cloud computing system, and / or combinations thereof. As described above, the MPP computing system 102 is configured to receive information from one or more laboratories regarding executed chemical processes, to use the information to train one or more machine learning models for predicting properties of the outputs of chemical processes, and to use the machine learning models for predictions of novel chemical processes.

[0031] As shown, the MPP computing system 102 includes one or more processors 202, one or more communication interfaces 204, a model data store 218, a chemical data store 208, a process data store 212, an experiment data store 214, a result data store 216, and a computer-readable medium 206.

[0032] As used herein, "data store" refers to any suitable device configured to store data for access by a computing device. One example of a data store is a highly reliable, highspeed relational database management system (DBMS) executing on one or more computing devices and accessible over a high-speed network. Another example of a data store is a keyvalue store. However, any other suitable storage technique and / or device capable of quicklyand reliably providing the stored data in response to queries may be used, and the computing device may be accessible locally instead of over a network, or may be provided as a cloud-based service. A data store may also include data stored in an organized manner on a computer-readable storage medium, such as a hard disk drive, a flash memory, RAM. ROM, or any other type of computer-readable storage medium. One of ordinary skill in the art will recognize that separate data stores described herein may be combined into a single data store, and / or a single data store described herein may be separated into multiple data stores, without departing from the scope of the present disclosure.

[0033] In some embodiments, the model data store 218 is configured to store machine learning models that have been trained for the prediction of properties of chemical process outputs. In some embodiments, the chemical data store 208 is configured to store structured chemical representations that represent chemicals to be used in one or more chemical processes. In some embodiments, the process data store 212 is configured to store structured process representations that represent steps and parameters of chemical processes. In some embodiments, the experiment data store 214 is configured to store representations of experiments, which include a combination of representations of chemicals and chemical processes to be performed on the chemicals. In some embodiments, the result data store 216 is configured to store measured properties of experimental chemical products for use in training the machine learning models discussed herein.

[0034] As used herein, "computer- readable medium" refers to a removable or nonremovable device that implements any technology capable of storing information in a volatile or non-volatile manner to be read by a processor of a computing device, including, but not limited to, the following: a hard drive; a flash memory; a solid state drive; randomaccess memory’ (RAM); read-only memory (ROM); a CD-ROM. a DVD, or other disk storage; a magnetic cassette; a magnetic tape; a magnetic disk storage; or combinations thereof.

[0035] In some embodiments, the processors 202 may include any suitable type of general- purpose computer processor. In some embodiments, the processors 202 may include one or more special-purpose computer processors or Al accelerators optimized for specific computing tasks, including but not limited to graphical processing units (GPUs), vision processing units (VPUs), and tensor processing units (TPUs).

[0036] In some embodiments, the communication interfaces 204 include one or more hardware and or software interfaces suitable for providing communication links between components of the system 100. The communication interfaces 204 may support one or morewired communication technologies (including but not limited to Ethernet. FireWire, and USB), one or more wireless communication technologies (including but not limited to WiFi, WiMAX, Bluetooth, 2G, 3G, 4G, 5G, and LTE), and / or combinations thereof.

[0037] As shown, the computer-readable medium 206 has stored thereon logic that, in response to execution by the one or more processors 202, cause the MPP computing system 102 to provide an encoder execution engine 210, a property prediction engine 220, and a model training engine 222. As used herein, "engine" refers to logic embodied in hardware or software instructions, which can be written in one or more programming languages, including but not limited to C, C++, C#, COBOL, JAVA™, PHP, Perl, HTML, CSS, JavaScript, VBScript, ASPX, Go, and Python. An engine may be compiled into executable programs or written in interpreted programming languages. Software engines may be callable from other engines or from themselves. Generally, the engines described herein refer to logical modules that can be merged with other engines, or can be divided into subengines. The engines can be implemented by logic stored in any type of computer-readable medium or computer storage device and be stored on and executed by one or more general purpose computers, thus creating a special purpose computer configured to provide the engine or the functionality thereof. The engines can be implemented by logic programmed into an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or another hardware device.

[0038] In some embodiments, the encoder execution engine 210 is configured to retrieve encoders from the model data store 218 and to use them to encode structured representations of the chemicals and the chemical processes into corresponding latent spaces, and to convert combined experiment representations from the latent spaces into latent experiment representations. In some embodiments, the property prediction engine 220 is configured to retrieve a decoder from the model data store 218, and use the decoder to generate predicted properties based on latent experiment representations. In some embodiments, the model training engine 222 is configured to use differences between measured properties and predicted properties for a given experiment to improve the performance of the encoders and decoder.

[0039] Further description of the configuration of each of these components is provided below.

[0040] FIG. 3 is a schematic illustration of a non-limiting example embodiment of an encoder / decoder that processes workflow information, according to various aspects of the present disclosure. The workflow to be processed is described by input workflowinformation 302, which includes chemical data 304 and process data 306. The chemical data 304 may contain information including but not limited to one or more of molecular structures, CAS numbers, bulk material properties, formula components (e.g., weights, ratios, etc.), and / or combinations thereof, which describe constituents and properties related to physical matter being incorporated or manipulated as part of a chemical workflow to transform and create a new chemical or other form of matter. The process data 306, which may be optional, may describe procedures, methods, and / or techniques to be used to process the constituents described by the chemical data 304; and / or settings and / or conditions of the environment and equipment used for the processing of the constituents described by the chemical data 304.

[0041] The chemical data 304 and the process data 306 may be provided in any suitable format. In some embodiments, at least some of the chemical data 304 and / or the process data 306 may be provided in a structured format, including but not limited to a tabular format, a JSON format, or a BSON format. In some embodiments, at least some of the chemical data 304 and / or the process data 306 may be provided as unstructured data. The unstructured data may be provided in any suitable form, including but not limited to a natural language description of chemical constituents and / or process steps (e.g., lab notes, patent publications, white papers, etc ); images, audio, or video captured of the chemical constituents and / or process steps; or other forms.

[0042] The input workflow information 302 is provided to an encoder 308, which transforms the input workflow information 302 into an intermediate latent representation 310. The intermediate latent representation 310 is consumed by a decoder 312, which generates a prediction relating to the input workflow information 302. The encoder 308 may also share intermediate state information from various stages of the encoder 308 with the decoder 312, with an optional transform 320 being applied to the intermediate state information before being provided to the decoder 312.

[0043] One example of a prediction that may be generated by the decoder 312 is a workflow prediction 314, which may represent a prediction of a chemical output generated when applying the steps described by the process data 306 to the components described by the chemical data 304). Another example of a prediction that may be generated by the decoder 312 is a workflow latent representation 316, which may represent features of the workflow that were extracted by the encoder / decoder 300. Yet another example of a prediction that may be generated by the decoder 312 is generated workflow information 318. The generated workflow information 318 may include chemical data and, optionally, process data (similar to the input workflow information 302), but with generated differencesbetween the generated workflow information 318 and the input workflow information 302 that are predicted to achieve a desirable result (e.g., a desired change in a resultant chemical property, a desired change in a process step, etc.). Any suitable type of encoder 308 and decoder 312, which are typically complementary machine learning models having appropriate architectures, may be used.

[0044] In some embodiments, the latent representations (i.e., the intermediate latent representation 310 and the workflow latent representation 316) may be a human-readable encoding of the input workflow information 302, or may be a high- or low-dimensional array of values that encode the input workflow information 302 (not necessarily human- readable) in a way analogous to a fingerprint or biometric of the input workflow information 302. In some embodiments, the latent representations may be an explicit output of an encoder 308 that may be stored in isolation as a distinct entity separate from the encoder 308 and the decoder 312. In some embodiments, the latent representations may be implicit as information within a combined encoder 308 / decoder 312 architecture, as opposed to being a distinct entity7. That is to say, the intermediate latent representation 310 and / or the workflow latent representation 316 may be extracted from any part of the architecture of the encoder 308 and / or the decoder 312.

[0045] In some embodiments, the chemical data 304 and the process data 306 may be encoded by a single encoder 308 into a combined intermediate latent representation 310. In some embodiments, the chemical data 304 and the process data 306 may be encoded separately, with the separate latent representations being combined into a new latent representation which represents an encoding of the entire workflow. In some embodiments, the encoding of the chemical data 304 and / or the process data 306 may be further divided into multiple subcomponents using different sub-encoders trained to handle particular types of data (e.g., chemical structures, audio, video, material property measurements / characterizations, etc.) or data from particular applications (e.g., paints / coatings, polymers, adhesives, etc.), with the latent representations generated by the sub-encoders being combined at a later point in the processing.

[0046] In some embodiments, transformer architectures may be used to implement the encoder 308 and / or the decoder 312. Many types of transformers may be used, depending on the modality of the input workflow information 302. Some transformer architectures may handle multiple modalities simultaneously, while others may be usable to encode a specific modality (e g., text, audio, or video) into a latent representation that may be injected as contextual data. In this way, the encoder / decoder 300 may combine modalities of input workflow information 302 or other data that can then be injected as context data (asdescribed in further detail below). This enables the encoder / decoder 300 to combine modalities into a form amenable to latent representation fusion, as also described in further detail below. Some ty pes of transformer architectures may be amenable to be used as an encoder 308, other types of transformer architectures may be amenable to be used as a decoder 312. and other types of transformer architectures may be amenable to be used as both an encoder 308 and a decoder 312.

[0047] A non-limiting and non-exhaustive list of suitable transformer architectures, along with indications of whether they are appropriate for use as an encoder 308 or decoder 312, is provided in Table 1. In cases where the transformer is suitable as an encoder 308 but not as a decoder 312, the resulting latent representation generated by the encoder 308 may be decoded by combining with a suitable transformer decoder 312, or any other suitable ty pe of machine learning model (e.g., a deep neural model (multi-layer perceptrons (MLPs), recurrent neural networks (RNNs), etc.) or statistical model (e.g., random forest, support vector machine (SVM), linear / logistical regression, etc.). Essentially, a transformer used as an encoder 308 may generate a rich latent representation that can then be used with multiple different types of models used as a decoder 312 to convert the latent representation into a prediction or generation of some sort.

[0048] FIG. 4A - FIG. 4C are schematic illustrations of techniques used for predicting properties of an output of a chemical process, according to various aspects of the present disclosure. The techniques illustrated in FIG. 4A - FIG. 4C are a non-limiting example of the processing of an encoder / decoder 300 where the chemical data 304 and the process data 306 are encoded separately, with the latent representations being merged later in the analysis.

[0049] FIG. 4A illustrates a non-limiting example embodiment of a process for encoding a chemical according to various aspects of the present disclosure. As used herein, a chemical is an input to a chemical process, and may include information relating to one or more aspects or moieties therein. For example, a chemical may include a name, an identifier, a description, a concentration, or other information about one or more aspects or moieties therein. As an example, a relevant chemical aspect or moiety can include an electrophile such as a carbonyl carbon or other positively charged or partially positively charged atom; or a nucleophile such as an amine, oxygen atom, or other negatively charged or partially negatively charged atom.

[0050] First, a structured chemical representation 402 is received. The structured chemical representation 402 provides the information relating to aspects or moieties of each chemical molecule or ingredient of a process in a format that aids in processing, and is typically provided in a specially formatted human-readable text format. As one non-limiting example, the information may be provided in a JavaScript Object Notation (JSON) format as a set of key-value pairs.

[0051] In other embodiments, other intermediate formats, including but not limited to XML, CSV, or YAML, may be used which functionally describe the data in an equivalent way, as the chemical encoder 406 is capable of tokenizing and encoding any data type by treating it as plain text. In some embodiments, the chemical information (and / or the process information described below) may be represented in a graph structure, either explicitly or implicitly. For example, chemical information may be provided as an array of JSON records wherein each record in the array includes a key named “step” that implies a time ordering to the data, and this data may be represented as a graph. Similarly, a SMILES string representation implies an equivalent graph representation of a molecule, wherein the nodes of the graph are atoms with certain properties (e.g., mass, charge, valence, etc ), and the edges of the graph are bonds with properties characteristic to a bond (e.g., degeneracy, orbital configuration, etc.).

[0052] In some embodiments, unstructured text may also be provided as a chemical representation, either alongside the structured chemical representation 402, as a value in a key-value pair of the structured chemical representation 402, or to replace the structured chemical representation 402 entirely. For example, a prose description of the chemical, its relevant aspects or moieties, its properties, its uses, or a reason why it is included in the process may be provided.

[0053] FIG. 5 is a non-limiting example embodiment of a structured chemical representation according to various aspects of the present disclosure. The illustrated structured chemical representation is in a JSON format, but as noted above, any other structured format may be used for the structured chemical representation, and some information may also be provided in an unstructured format.

[0054] The illustrated structured chemical representation includes information for two materials, “Raw Material A” and “Raw Material B”, each of which is represented by a set of key-value pairs. As illustrated, the key-value pairs include a name, a Chemical Abstracts Service (CAS) Registry Number, a line notation string identifying the material in simplified molecular-input line-entry’ system (SMILES) format, and a concentration. These key-valuepairs are shown as non-limiting examples only, and in other embodiments, more, fewer, or different key-value pairs may be present. Further, more or fewer materials may be included within the structured chemical representation. Some additional non-limiting examples of information that may be provided for one or more materials in the structured chemical representation include other chemical identifiers, other types of line notations (e.g., a Polymer SMILES (PSMILES) string, an International Chemical Identifier (InChi) string, a Wiswesser Line Notation (WLN) string, a Smiles arbitrary target specification (SMARTS) string, a SYBYL Line Notation (SLN) string, etc.), empirical properties (e.g.. boiling point, melting point, glass transition temperature, etc.), molecular descriptors (e.g., specific aspects of the molecule, specific moieties within the molecule, specific molecular functionalities, charge distributions, etc.), or any other relevant information for describing the chemical compounds.

[0055] Returning to FIG. 4A, after obtaining the structured chemical representation 402, the structured chemical representation 402 then undergoes one or more optional preprocessing 404 steps. The one or more preprocessing 404 steps may include various actions that prepare the content of the structured chemical representation 402 for further processing, including but not limited to flattening the structured chemical representation 402 by removing line breaks, converting the string to a tokenized format (e g., breaking the string into individual words, punctuation, and / or other tokens, etc.), or any other suitable technique for preprocessing. In some embodiments, preprocessing 404 may include retrieving additional information regarding one or more chemicals described in the structured chemical representation 402. For example, if a given chemical description in the structured chemical representation 402 includes a value for a CAS Registry Number, a reference data store 114 may be queried to retrieve additional information associated with the CAS Registry Number that is not otherwise provided in the structured chemical representation 402.

[0056] The structured chemical representation 402, after going through optional preprocessing 404, is then provided to a chemical encoder 406, which creates a latent chemical representation 408. The latent chemical representation 408 is an encoded version of the structured chemical representation 402 in a latent space, such that similar values within the latent space indicate chemicals with similar characteristics, aspects, or moieties, and different values within the latent space indicate chemicals with different characteristics, aspects, or moieties.

[0057] Any suitable architecture may be used for the chemical encoder 406, and the characteristics of the latent space (e.g., the mapping of features from the structured chemicalrepresentation 402 to the latent space) are learned during training of the chemical encoder 406. In some embodiments, encoder architectures that would typically be used for text processing may be used. It has been found that text-processing encoder / decoder architectures are highly effective in encoding chemical information into a latent space useful for predicting properties, if such information is provided in a human-readable format, and that further effectiveness can be obtained if that format is structured as discussed above. Since text-processing encoder / decoder architectures are capable of compensating for differences in the precise terms, context, spelling, and other aspects of the input text, imposition of a strict format requirement on the structured chemical representation 402 (i.e., a list of key names that must be used, a list of key-values pairs that must be present, etc.) is not necessary, thereby improving the flexibility and predictive power of embodiments of the present disclosure.

[0058] In some embodiments, encoder architectures that use self-attention may be used. For example, in some embodiments, a Bidirectional Encoder Representations from Transformers (BERT) encoder may be used for the chemical encoder 406. BERT encoders are described in Devlin et al.. “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding / ’ arXiv: 1810.04805v2, May 24, 2019, the entire disclosure of which is hereby incorporated by reference herein for all purposes. To provide the structured chemical representation 402 to the chemical encoder 406, the structured chemical representation 402 is first tokenized (which may occur during preprocessing 404). Each token is converted to a unique identifier within a vocabulary used as the basis for the chemical encoder 406, and words not present in the vocabulary are broken into sub-words that are mapped to tokens. A one-dimensional vector is created, representing an embedding for each token. The embeddings are then used to encode the input into the latent space to create the latent chemical representation 408.

[0059] FIG. 4B illustrates a non-limiting example embodiment of a process for encoding a chemical process according to various aspects of the present disclosure. As used herein, a chemical process is a set of operations to be performed on a chemical, or combination of two or more chemicals. For example, a chemical process may include performing one or more steps such as: adding chemicals in a specified order; solvent use; heating; cooling; stirring; bubbling a gas; evacuating air or other gas; placing the chemicals under a specific atmosphere such as an inert atmosphere (e.g., nitrogen, argon) or a reactive atmosphere (e.g., oxygen, hydrogen, other reactive chemical gas); placing the chemical under vacuum (zero or near zero pressure), increased or reduced pressure; freeze-pump-thaw, introducing light of a specified wavelength or range of wavelengths; removing light; removal of acomponent (e.g., water, a gas, solvent, a byproduct); isolation of a chemical; purification of a chemical; drying the chemical; or any other chemical process for carrying out the desired chemical reaction process and which is known to one of skill in the art.

[0060] The chemical process can use various equipment for earn ing out the chemical process. For example, the chemical process can use non-reactive glassware or other containers; stir plates; heating elements; temperature regulators; cooling baths (e.g., ice, salt, liquid nitrogen, dry ice with inorganic or organic solvents (e.g., ammonia, sulfur hexafluoride, fluoride sulfuryl chloride, acetone, isopropanol), water with a salt (e.g., calcium chloride, sodium chloride)); pressure regulators; gas tanks; inert atmospheric technique equipment; light sources (e.g., luminescent, UV, visible); light blocking covers; vacuum pumps; valves; desiccants; specialized glassware (e.g., water removal, condensation); tubing; connectors; evaporators; condensers; ovens; computers or machines (e.g. for controlled introduction of chemicals, gases, solvents, etc.; for removal of chemicals, gases, solvents, etc.; for purifying chemicals; for characterizing chemicals, etc.); chromatography equipment; and / or any other equipment for carrying out the chemical process as known to, and employed by, one of skill in the art.

[0061] Any of the one or more chemical processes may be configured with certain parameters for carry ing out the chemical process. For example, parameters may include one or more of: a specified frequency for stirring; a specified temperature for heating and / or cooling; a specified gas pressure or absence of pressure; a specified wavelength or range of wavelengths of light; a specified period of time (for, e.g., the heating, cooling, stirring, introduction of light, evacuation, dry ing, addition of chemicals or solvents, etc.); a specified rate (for e.g., stirring, introduction of chemicals, removal of chemicals, solvent addition, solvent removal, chromatography purification of chemicals, etc.); or any other parameter for effectively carrying out the chemical process, as known to one of skill in the art.

[0062] The chemical process results in a chemical product which can be an intended product, a final product, or an intermediate product to which one or more additional chemical processes is carried out, or a chemical product which is an output that can be tested for one or more properties.

[0063] To encode a chemical process, first, a structured process representation 410 is received. As with the structured chemical representation 402, the structured process representation 410 provides the information relating to the configuration of the chemical process, and may provide the information in a format that aids in processing, such as a specially formatted human-readable text format. As with the structured chemicalrepresentation 402, the structured process representation 410 may be provided in a format such as JSON, XML, CSV, YAML, or any other format that can be tokenized and encoded via the process encoder 414 treating it as plain text, and may provide the information as a set of key-value pairs. In some embodiments, unstructured text may also be provided with the structured process representation 410, either alongside the structured process representation 410, as a value in a key-value pair of the structured process representation 410, or instead of the structured process representation 410. For example, a prose description of the chemical process, its steps, its benefits and / or drawbacks, its outputs, or any other information relevant to the chemical process may be provided.

[0064] FIG. 6A and FIG. 6B are a non-limiting example embodiment of a structured process representation according to various aspects of the present disclosure, with FIG. 6B illustrating a continuation of the structured process representation that starts in FIG. 6A. The illustrated structured process representation is in a JSON format, but as noted above, any other structured format may be used for the structured process representation, and some information may also be provided in an unstructured format.

[0065] The illustrated structured process representation includes two types of information: (1) batch or general process to be carried out (illustrated in FIG. 6A), and (2) the specific parameters associated with a process (illustrated in FIG. 6B). Collectively, the batch information and the parameter information detail steps for chemical processes to be carried out by a chemist, an automated workbench, or any other suitable means for conducting the chemical processes disclosed herein.

[0066] The batch information and parameter information may also specify one or more of an order in which steps should be performed and / or an order in which chemicals should be added, specific operating conditions for each step, and / or properties to be measured under different conditions of interest in order to characterize the output of the chemical process. In some embodiments, the batch information and the parameter information may also include condition values for one or more steps that indicate conditions under which a given step may or may not be performed, based on a parameter, a result of another step, or for any other suitable condition.

[0067] The illustrated structured process representation of FIG. 6A and FIG. 6B shows information for a mixing step (‘'Mixing Components’’) and a second curing step ("Curing Stage"), each of which is represented by a set of key-value pairs. The set of key -value pairs for the FIG. 6A mixing step, named “Mixing Components,” includes pairs that identify the step order ("1"), the equipment to use to perform the action ("Mixer - E001"), a frequencyfor the mixing action ("200"), a unit for the mixing setting ("Hz"), the chemical in which the mixing is to be applied ("Raw Material B"), the identity of the chemical (identified by the “SMILES”), and the amount of the chemical to be used (have a concentration of “0.3”).

[0068] The illustrated parameter representation in FIG. 6B includes information for two parameters. The first parameter, named “PG509,” provides information related to a mixer, the specific equipment to be used ("Mixer 1"), the specific temperature at which the mixing is to occur ("35C"), the mixing speed ("200RPM"), the volume (" ImL"), etc.

[0069] The second parameter, named “PG4234,” provides information related to a “Curing Stage,” which includes pairs that identify the step order ("2"), the equipment to use to perform the action ("Cure Oven - El 11 "), and the specific settings for the step, including introduction of light and identity of the type of light ("UV"). the wavelength of light to be used ("280"), the units for the wavelength ("nm"), the duration of curing as “Cure Time” ("60"), and the units for the duration of curing ("s").

[0070] Though not illustrated in FIG. 6A or FIG. 6B, in some embodiments, the structured process representation may include information about properties to be measured, and can include one or more steps for measuring the property. For example, a property may include any property of interest such as a boiling point, a melting point, ion exchange capacity, solubility, stability under certain conditions (e.g., exposure to air or sunlight), one or more acoustical properties, one or more chemical properties, one or more electrical properties, one or more magnetic properties, one or more mechanical properties, one or more optical properties, one or more thermal properties, any other property of specific interest to an end user, or combinations thereof.

[0071] Returning to FIG. 4B, after obtaining the structured process representation 410, in some embodiments, the structured process representation 410 then undergoes one or more optional preprocessing 412 steps. As with the optional preprocessing 404 steps in FIG. 4A, the one or more preprocessing 412 steps may include various actions that prepare the content of the structured process representation 41 for further processing, including but not limited to flattening the structured process representation 410 by removing line breaks, converting the string to a tokenized format (e.g., breaking the string into individual words, punctuation, and / or other tokens, etc ), or any other suitable technique for preprocessing. In some embodiments, preprocessing 404 may include retrieving additional information regarding one or more machines described in the structured process representation 410. For example, if a given machine referenced in the structured process representation 410 is identified by a model or serial number, a reference data store 114 may be queries to retrieveadditional information associated with the machine that is not otherwise provided in the structured process representation 410.

[0072] The structured process representation 410, after going through the optional preprocessing 412, is then provided to a process encoder 414, which creates a latent process representation 416. The latent process representation 416 is an encoded version of the structured process representation 410 in a latent space, such that similar values within the latent space indicate chemical processes and parameters with similar characteristics, and different values within the latent space indicate chemical processes and parameters with different characteristics.

[0073] Any suitable architecture may be used for the process encoder 414, and the characteristics of the latent space (e.g., the mapping of features from the structured process representation 410 to the latent space) are learned during training of the process encoder 414. As with the chemical encoder 406, encoder architectures that would typically be used for text processing may be used for the process encoder 414, providing the same benefits discussed with respect to the chemical encoder 406. In some embodiments, encoder architectures that use self-attention, including but not limited to BERT encoders, may be used for the process encoder 414, with the structured process representation 410 being provided to the process encoder 414 using similar techniques to those discussed above with regard to providing the structured chemical representation 402 to the chemical encoder 406, including tokenization, conversion to unique identifiers, etc.

[0074] FIG. 4C illustrates a non-limiting example embodiment of a process for combining an encoded chemical and an encoded chemical process, encoding the combined representation to create an encoded experiment, and predicting properties based on the encoded experiment according to various aspects of the present disclosure.

[0075] First, a latent chemical representation 408 (see FIG. 4A) and a latent process representation 416 (see FIG. 4B) are combined to create a combined experiment representation 418. The combined experiment representation 418 is meant to represent a process of applying the chemical process and parameters represented by the latent process representation 416 to the chemicals represented by the latent chemical representation 408. The latent chemical representation 408 and latent process representation 416 may be combined in any suitable manner. In some embodiments, the latent chemical representation 408 may simply be concatenated with the latent process representation 416 in either order.

[0076] The combined experiment representation 418 is provided to an experiment encoder 420, which creates a latent experiment representation 422. Any suitable architecture may beused for the experiment encoder 420, including text-processing architectures that use selfattention, including but not limited to BERT encoders. Again, the characteristics of the latent space (e.g., the mapping of features from the combined experiment representation 418 to the latent space) are learned during training of the experiment encoder 420, and the latent space is such that similar values within the latent space indicate experiments with similar outcomes, and different values within the latent space indicate experiments with different outcomes. One will note that by using architectures that use self-attention, the chemical encoder 406, process encoder 414, and experiment encoder 420 encode both local and global contextual information into the representations in the respective latent spaces. As such, the same chemical or parameter reused in two separate experiments may have different representations in the latent spaces, given that the global context in which they are situated in the separate experiments.

[0077] Finally, the latent experiment representation 422 is provided to an experiment decoder 424, which generates predicted properties 426 based on the latent experiment representation 422. Any suitable architecture may be used for the experiment decoder 424. In some embodiments, an experiment decoder 424 architecture complimentary to the architectures of the chemical encoder 406. process encoder 414, and experiment encoder 420, such as a decoder related to a BERT encoder, may be used. In some embodiments, the architecture may use a stochastic process, including but not limited to a Gaussian process, as the experiment decoder 424. In such architectures, any suitable type of kernel may be used for the Gaussian process, including but not limited to a radial basis kernel. In some embodiments, an artificial neural network may be used as the experiment decoder 424. The experiment decoder 424 treats changes in latent experiment representation 422 as being representative of changes in the predicted properties 426, and determines the mapping between the latent experiment representation 422 and the predicted properties 426 during training.

[0078] FIG. 7A and FIG. 7B are a flowchart that illustrates a non-limiting example embodiment of a method of predicting properties of an output of a chemical process according to various aspects of the present disclosure. In the method 700, the MPP computing system 102 uses machine learning models to generate encoded representations of the chemical and the chemical process, and predicts the properties based on the encoded representations. As illustrated, the method 700 also shows a technique for training the machine learning models, either to initially train the models or to re-train models that had previously been trained in order to improve their accuracy. By training encoders to encode the chemical and the chemical process into representations in a latent space, accuratepredictions can be generated for novel chemicals and / or chemical processes (that is, chemicals and / or chemical processes that have not previously been considered by the MPP computing system 102 and / or were not used to train the encoders or the decoder). The method 700 describes training and use of a split model such as that illustrated in FIG. 4A - FIG. 4C, though one will recognize that similar techniques may be used with the encoder / decoder 300 of FIG. 3, with structured or unstructured input data, and / or any other model architecture or information described herein.

[0079] From a start block, the method 700 proceeds to block 702, where an MPP computing system 102 receives structured chemical representation 402. In some embodiments, the MPP computing system 102 may receive the structured chemical representation 402 by providing a user interface that allows an end user to enter or upload the structured chemical representation 402. In some embodiments, the MPP computing system 102 may receive the structured chemical representation 402 by retrieving it from a chemical data store 208 associated with the MPP computing system 102 in which the structured chemical representation 402 was previously stored, such as during a previous execution of the method 700, or as part of building a library of structured chemical representations 402 expected to be used in experiments. In some embodiments, the MPP computing system 102 may present a user interface that displays information regarding ingredients of chemicals in previously stored structured chemical representations 402, and may allow the user to select one or more previously stored chemicals to be included in a new structured chemical representation 402.

[0080] At block 704, an encoder execution engine 210 of the MPP computing system 102 uses a chemical encoder 406 to generate a latent chemical representation 408 based on the structured chemical representation 402. In some embodiments, the MPP computing system 102 may retrieve the chemical encoder 406 from a model data store 218 associated with the MPP computing system 102, and use the retrieved chemical encoder 406 to generate the latent chemical representation 408 based on the structured chemical representation 402. In some embodiments, the chemical encoder 406 may have previously been partially or fully trained using the method 700.

[0081] At block 706, the MPP computing system 102 receives a structured process representation 410. In some embodiments, the MPP computing system 102 may receive the structured process representation 410 by providing a user interface that allows an end user to enter or upload the structured process representation 410. In some embodiments, the MPP computing system 102 may receive the structured process representation 410 by retrieving it from a process data store 212 associated with the MPP computing system 102in which the structured process representation 410 was previously stored, such as during a previous execution of the method 700, or as part of building a library of structured process representations 410 expected to be used in experiments.

[0082] In some embodiments, the structured process representation 410 may be built by combining elements from previously created structured process representations 410. For example, a user interface may present information for previously stored process steps and / or parameters, and may allow an end user to select one or more previously stored steps and / or one or more previously stored parameters in order to help create a structured process representation 410 for a new chemical process.

[0083] At block 708, the encoder execution engine 210 uses a process encoder 414 to generate a latent process representation 416 based on the structured process representation 410. In some embodiments, the MPP computing system 102 may retrieve the process encoder 414 from a model data store 218 associated with the MPP computing system 102, and use the retrieved process encoder 414 to generate the latent process representation 416 based on the structured process representation 410. In some embodiments, the process encoder 414 may have previously been partially or fully trained using the method 700.

[0084] At block 710, the encoder execution engine 210 combines the latent chemical representation 408 and the latent process representation 416 to create a combined experiment representation 418. As described above, the latent chemical representation 408 and latent process representation 416 may be combined using any suitable technique, including but not limited to simply concatenating the two representations in an order expected by the experiment encoder 420.

[0085] The method 700 then proceeds to a continuation terminal ("terminal A"). From terminal A (FIG. 7B), the method 700 proceeds to block 712, where the encoder execution engine 210 uses an experiment encoder 420 to generate a latent experiment representation 422 based on the combined experiment representation 418. In some embodiments, the MPP computing system 102 may retrieve the experiment encoder 420 from a model data store 218 associated with the MPP computing system 102, and use the retrieved experiment encoder 420 to generate the latent experiment representation 422 based on the experiment encoder 420. In some embodiments, the experiment encoder 420 may have previously been partially or fully trained using the method 700.

[0086] At block 714. a property prediction engine 220 of the MPP computing system 102 uses an experiment decoder 424 to generate one or more predicted properties 426 for an output of the chemical process based on the latent experiment representation 422. In someembodiments, the property prediction engine 220 may be pre-configured with one or more types of properties to predict for all latent experiment representations 422 provided to it. In some embodiments, the latent experiment representation 422 may suggest the properties to be predicted for the associated experiment. Once the one or more predicted properties 426 are generated, the property prediction engine 220 may store the predicted properties 426 in a result data store 216 associated with the MPP computing system 102, and / or may generate a user interface to present the predicted properties 426 to an end user.

[0087] The method 700 then proceeds to a decision block 716, where a determination is made regarding whether the method 700 is in a training mode or a regular prediction mode. In some embodiments, this determination may be made based on a user configuration (i.e., whether a user has placed the method 700 in the regular prediction mode or the training mode). In some embodiments, this determination may be made based on whether a predetermined number of initial training iterations have yet been performed or whether performance of the models has converged to an acceptable level, and the training mode will be determined if the iterations have not yet been performed or the performance is not yet acceptable. In some embodiments, this determination may be made based on how long it has been since one or more of the chemical encoder 406, process encoder 414, experiment encoder 420, or experiment decoder 424 were previously retrained.

[0088] If it is determined that the method 700 is in a regular prediction mode, then the result of decision block 716 is NO. and the method 700 proceeds to an end block and terminates. Otherwise, if it is determined that the method 700 is in a training mode, then the result of decision block 716 is YES, and the method 700 proceeds to block 718. At block 718, a model training engine 222 of the MPP computing system 102 receives one or more measured properties from a laboratory that executed the experiment. The measured properties are treated as ground truth properties for the experiment. In some embodiments, the measured properties may be transmitted by the laboratory for storage in a result data store 216 associated with the MPP computing system 102, along with an identification of the experiment that led to the measured properties. In some embodiments, the laboratory may be given instructions to execute the experiment as defined in the structured chemical representation 402 and structured process representation 410, and a unique identifier for the instructions may be used to link the measured properties to the associated predicted properties 426. In some embodiments, the identification of the experiment may be a copy of the latent experiment representation 422 or any part of the data used to create the latent experiment representation 422 (e.g., the structured chemical representation 402, the structured process representation 410, the combined experiment representation 418, and / orthe latent space representations thereof). The model training engine 222 may then obtain the measured properties from the result data store 216 by querying for measured properties that match the experiment being predicted.

[0089] At block 720. the model training engine 222 compares the one or more predicted properties 426 to the one or more measured properties to determine a value of a loss function. In some embodiments, a value for each predicted property 426 is compared to the corresponding measured property', and a difference between the two is taken as the loss value to be fed into a suitable loss function, such as Mean Squared Error.

[0090] At block 722, the model training engine 222 determines a gradient of the loss function, and uses the gradient to update at least one of the experiment decoder 424, the experiment encoder 420, the process encoder 414, or the chemical encoder 406. By updating the experiment decoder 424, experiment encoder 420, process encoder 414, and / or chemical encoder 406 using the gradient of the loss function, the performance of the method 700 is iteratively improved. In some embodiments, the gradients may be used to update all of these models during each iteration. In some embodiments, some models may be held constant while other models are updated for one or more iterations.

[0091] The method 700 then advances to a decision block 724, where a determination is made regarding whether the method 700 is done training the models, or whether further training iterations should be performed. In some embodiments, the determination may be based on whether a predetermined number of iterations have been performed. In some embodiments, the determination may be based on whether performance of the models has converged to an acceptable level. If it is determined that training is not yet done, then the result of decision block 724 is NO. and the method 700 advances to a continuation terminal ("terminal B"), in order to return to block 702 to conduct one or more subsequent training iterations. Otherwise, if training is done, then the result of decision block 724 is YES, and the method 700 proceeds to block 726.

[0092] At block 726, the model training engine 222 stores the trained experiment decoder 424, experiment encoder 420, process encoder 414, and / or chemical encoder 406 in a model data store 218 of the MPP computing system 102 to be used for subsequent executions of the method 700. The method 700 then proceeds to an end block and terminates.

[0093] The illustration of the method 700 provided in FIG. 7A and FIG. 7B shows the creation of a single training example (the pairing of one input structured chemical representation 402 and structured process representation 410 with one set of measured properties) for the ease of discussion. In some embodiments, training of the models mayinvolve collecting a large number of training examples with different structured chemical representations 402, different structured process representations 410, and different measured properties prior to placing the method 700 in training mode. In some embodiments of the system 100, by combining experimental results from a plurality of laboratories such as all laboratories across a large research institution that all use the MPP computing system 102 to store records of experiments and results, an adequate number of training examples for training the models to a high level of performance may be gathered fairly quickly.

[0094] By providing the ability to predict the outcome of chemical processes, the MPP computing system 102 greatly enhances the ability of chemists and other researchers to explore previously untested combinations of chemicals in the search for new chemicals with desirable properties. Predictions provided by the MPP computing system 102 can allow researchers to avoid actual physical execution of experiments that are unlikely to result in desired outcomes, and to focus on physical execution of experiments that are more likely to have successful outputs. This reduces waste in materials, time, and other laboratory resources.

[0095] The combination of the MPP computing system 102 with an automated laboratory 112 may provide even further benefits. For example, if there are areas within the latent spaces of the latent chemical representation 408, latent process representation 416, or latent experiment representation 422 that are not well covered by the training data already present in the result data store 216, the MPP computing system 102 may automatically create experiments that would cover these missing areas, transmit the experiments to the automated laboratory 112, and add the automatically generated results to the result data store 216 to improve the coverage of the training data.

[0096] Further, once the chemical encoder 406, process encoder 414, experiment encoder 420, and experiment decoder 424 are fully trained, the steps of the non-training mode of the method 700 may be run in reverse in order to perform inverse design of chemicals and / or chemical processes. For example, a starting set of chemicals and a starting chemical process may be provided, along with one or more desired properties. The method 700 may be executed to generate one or more predicted properties, which are then compared to the one or more desired properties to determine a loss. This loss may then be backpropagated through the experiment decoder 424 and the experiment encoder 420 to determine a gradient to be used to update the latent chemical representation and / or the latent process representation. These updated latent representations may then be used to update the input chemicals and / or the input chemical process in an attempt to bring the predicted properties closer to the desired properties, and the updated information can be simulated again. Byiterating over this prediction / comparison / backpropagation / update loop, the MPP computing system 102 can comprehensively explore the search space of potential chemical and chemical process combinations without actually performing the physical experiments, thereby dramatically reducing the research effort and resources involved in obtaining the desired properties.

[0097] The above description contemplates a single chemical encoder 406 associated with the structured chemical representation 402, and a single process encoder 414 associated with the structured process representation 410. In some embodiments, multiple encoders or decoders may be present. For example, encoders and / or decoders to provide additional levels of detail may be added. For example, since the structured process representation 410 includes both batch and parameter sections, in some embodiments, separate batch encoders and parameter encoders may be provided. In some embodiments, value-specific encoders may be provided for well-structured key-value pairs that are not expected to vary in formats, such as information provided in a chemical line notation. For example, if the chemical encoder 406 detects that a SMILES string is present in the structured chemical representation 402, a SMILES-specific encoder may be used to encode that string prior to encoding the remainder of the structured chemical representation 402.

[0098] As another example, in some embodiments, multiple different encoders and / or decoders may be trained in order to have encoders and / or decoders that are considered expert in specific domains in order to improve domain-specific results. For example, a first chemical encoder 406, first process encoder 414, first experiment encoder 420, and / or first experiment decoder 424 may be trained using data from a first domain (e.g., experiments related to adhesives); a second chemical encoder 406, second process encoder 414, second experiment encoder 420, and / or second experiment decoder 424 may be trained using data from a second domain (e.g., experiments related to inks), and so on for a plurality of domains (e.g., coatings, ceramics, food, polymers, etc.). In some simple embodiments, a user may select a domain, and appropriate encoders and / or decoders trained with data from the selected domain may be used. In more complex embodiments, the plurality of encoders and / or decoders may be combined in an agent network, where an additional machine learning model (e.g., a transformer model) is trained to determine appropriate domain expert encoder and / or decoder models to be used for a given experiment based directly on the structured chemical representation 402 and / or structured process representation 410. Embodiments using agent networks can be particularly powerful when used in an inverse design or “what experiment should be performed next" context, as the agent network canleverage context learned from multiple domains in suggesting additional experiments while still providing accurate domain-specific predictions.

[0099] The encoder / decoder 300 illustrated in FIG. 3 and the process illustrated in FIG. 4A - FIG. 4C describe relatively simple pairs of encoders and decoders, in some embodiments, more complicated architectures may be used. For example, a given encoder or decoder may be made up of a stack of encoders and / or decoders, such that a given encoder or decoder includes additional embedded latent representations which represent intermediate stages of encoding information, and which may encode multi-scale features of the information. As a non-limiting example, a given encoder may be configured to encode an image of a laboratory apparatus involved in a chemical workflow. Intermediate layers or sub-encoders of such an encoder may be configured to detect or analyze specific types of visual data. These layers may extract distinct features that correlate with relevant chemical, process, or environmental factors within the workflow, including but not limited to apparatus configuration, reactant states, and / or fluid dynamics. These features, detected by specific layers within the encoder, can provide insights that other encoders or decoders might not capture, offering a deeper understanding of the chemical process that could influence further stages of analysis or decision making.

[0100] In some embodiments, a transformer architecture may be used to implement at least one of the encoders. Encoders implemented using a transformer architecture may be particularly effective in processing and correlating various forms of input data due to the attention mechanism. Transformers may be configured to handle diverse data types, including but not limited to structured / tabular data representing sensor readings, time-series data, or chemical measurements, alongside unstructured data including but not limited to images, audio, video, or natural language text such as laboratory notes. For example, a transformer may ingest structured data sets (e.g., temperature readings, pressure measurements, reagent concentrations, etc.) in tabular data or other formats, and may leam temporal patterns or correlations that are relevant to the workflow's stages or outcomes. As another example, a transformer may ingest video or audio input (e.g., a live stream of the execution of a workflow, or a sound recording of an apparatus functioning, etc.), and can capture temporal dependencies to identify process phases like stirring, boiling, or other physical changes in the system executing the workflow. As still another example, a transformer can encode textual descriptions (e.g., laboratory notes or procedural instructions) into meaningful representations, capturing high-level context, procedural steps, and operational parameters that describe the ongoing workflow in a more abstract manner. As the encoder processes these different data streams, the attention mechanism within thetransformer allows it to focus on the most relevant parts of each input modality. This means that the transformer can simultaneously attend to critical points in time (e.g., a temperature spike in sensor data, etc.), key visual elements (e.g., color changes in a reactant, etc.), and / or important procedural cues (e.g., an instruction to begin heating, etc.). The multiscale, multimodal features extraction provided by the transformer architecture empowers the encoder to create a comprehensive representation of the workflow, which facilitates better predictions, adjustments, and / or insights into the chemical process.

[0101] More complex encoder / decoders may optionally provide for additional manipulation of their encoding / decoding processing via the injection of additional contextrelevant information at different stages of the encoding / decoding process. FIG. 8 is a schematic illustration of a non-limiting example embodiment of an encoder / decoder in which context information is optionally encoded and injected into the encoding / decoding processing at various points, according to various aspects of the present disclosure. The encoder / decoder 800 may represent an entire encoder / decoder 300, or may represent a subencoder / decoder used as a component of a larger encoder / decoder 300. By injecting additional information at various points within the encoder / decoder 800, richer latent representations may be generated, which may better encode the complexities of chemical workflows by taking context into consideration at various points.

[0102] In the encoder / decoder 800, input data 802 is provided to a first encoder / decoder 804, which generates an intermediate latent representation 806. The input data 802 may be input workflow information 302 as illustrated in FIG. 3, or may be raw data or signal input from a chemical workflow. The raw data or signal input may include one or more of sensor readings, video, images, time-series data, and / or other information representing an ongoing chemical process. By generating an intermediate latent representation 806 based on the input data 802, the encoder / decoder 800 can not only compress meaningful features from the input data 802 but can also facilitate reconstruction or transformation of a workflow representation for downstream tasks. Once the intermediate latent representation 806 is generated, the encoder / decoder 800 may optionally perform latent space fusion 808 to combine the intermediate latent representation 806 with other information, and the combined latent representation is provided to a second encoder / decoder 810 which generates one or more of a workflow prediction 820 (similar to workflow prediction 314 in FIG. 3), a latent representation 822 (similar to workflow latent representation 316 in FIG. 3), or a generated chemical workflow 824 (similar to generated workflow information 318 in FIG. 3).

[0103] At various points in the encoder / decoder 800, information from context data 818 may be extracted and injected into the encoding / decoding processes, or directly into the latent representations. Context data 818 may include information representing the context within which a workflow is being conducted, including but not limited to one or more of procedural information, chemical descriptor information, prior experimental results, laboratory notes, environmental factors (e.g., ambient temperature, humidity, reagent properties, etc.), or other information regarding the context. This context data 818 can represent factors that influence experimental outcomes, allowing the encoder / decoder 800 to adjust predictions or compare experiments more accurately. The richer latent representations provided by injecting context improves generalization across different environments or tasks by allowing flexible integration of both intrinsic and extrinsic data. Fusion of context and general knowledge allows the encoder / decoder 800 to be adaptable, and it helps enable support of different tasks and domains. In some embodiments, attention mechanisms may be included as part of the context encoding, ensuring that the encoder / decoder 800 emphasizes relevant context based on given task goals.

[0104] The context data 818 may be passed to one or more optional encoders, and latent context representations output from those encoders may be provided to various points in the encoder / decoder 800. As shown, a first context encoder 812 may encode the context data 818 for injection into the first encoder / decoder 804, a second context encoder 814 may encode the context data 818 for fusion with the intermediate latent representation 806, and a third context encoder 816 may encode the context data 818 for injection into the second encoder / decoder 810. Each of the first context encoder 812, second context encoder 814, and third context encoder 816 may be trained separately, and may encode different aspects of the context data 818 that are appropriate for injection into the various points of the encoder / decoder 800.

[0105] Though single encoder / decoder and context encoder blocks are illustrated at each position in encoder / decoder 800 for the sake of clarity, in some embodiments, multiple encoder / decoders or context encoders may be provided at each point. The use of multiple context encoders and / or multiple encoder / decoders at each level of the overall encoder / decoder 800 can help handle multimodal input data 802 and / or multimodal context data 818, thus allowing different forms of context (e.g., structured data, laboratory notes, audio / video, etc.) to be processed.

[0106] For the latent space fusion 808, any suitable technique may be used to join the intermediate latent representation 806 and a latent representation (or other output) generated by the second context encoder 814. FIG. 9A - FIG. 9C illustrate three non-limiting exampletechniques that may be used for latent space fusion, according to various aspects of the present disclosure. These three examples show how contextual information from molecular descriptors may be fused with the intermediate latent representation 806 of the structure of the input data to produce a fused latent representation that is contextualized to that molecule residing in the given environment. This type of latent spaced fusion improves the richness of latent representations over an encoder-only representation as measured by accuracy metrics of predictions made based on just molecular descriptors, just latent representations learned by a simple encoder, and fused latent representations involving an encoder and the injection of context relevant information by descriptors. In other embodiments, different types of information may be fused with the intermediate latent representation 806.

[0107] In FIG. 9A, input data 902 (for example, input data 802 as described in FIG. 8) is provided to encoder 906 to generate an intermediate latent representation 926, illustrated in FIG. 8 as intermediate latent representation 806. Meanwhile, the input data 902 is also provided to a molecular descriptor calculator 908 along with context data 904 (for example, context data 818 as described in FIG. 8). In some embodiments, the molecular descriptor calculator 908 acts as a context encoder that accepts the input data 902 and context data 904 as input (e.g., a description of a chemical, including but not limited to a SMILES string representing a molecule, or other structured or unstructured descriptions of the chemical), but rather than computing a learned fingerprint or other intermediate latent representation 926 (as does the encoder 906), the molecular descriptor calculator 908 computes and outputs one or more descriptors 930 regarding the chemical, which may represent specifically computed quantities for the molecule. The descriptors 930 are then fused with the intermediate latent representation 926 to create a complete latent representation 928. Because the complete latent representation 928 has had context-relevant information injected therein, the complete latent representation 928 becomes more useful in making predictions and generations related to the outcome of a given workflow.

[0108] As a non-limiting example, if the overall encoder / decoder 800 would find a latent representation useful that could be used to predict reactivity of the molecule / chemical in the presence of an electromagnetic field, the descriptors 930 may include characteristics related to reactivity. For example, the descriptors 930 may include a number of nitrogen-carbon double bonds in the molecule, an empirical property' which may be used to query a look-up table for the electromagnetic susceptibility of the molecule at the frequency of a given electromagnetic field to be applied, or other information related to the reactivity. These descriptors 930 may then be used as context information to be fused with the intermediate latent representation 926, thus allowing the complete latent representation 928 to be usedfor more accurate predictions and generations in contexts including the electromagnetic field context that was injected as part of the context data 904.

[0109] FIG. 9A illustrates a first non-limiting example technique for fusing the descriptors 930 with the intermediate latent representation 926. As shown, the intermediate latent representation 926 and the intermediate latent representation 926 simply undergo concatenation 910. In other words, if the intermediate latent representation 926 is represented by an array [ao, a / , . . . , aw / ] of size N, and the descriptors 930 are represented by an array [do, di, . . . . dw- / ] of size M, then the concatenation 910 results in an array [ao. a / , . . . , aw / , do, d / , . . . , dw- / ] of size N + M.

[0110] After concatenation 910 the concatenated latent representation undergoes normalization 912. Any suitable normalization technique may be used. In some embodiments, the U normalization may be used. In a general sense, the U norm may be defined as:

[0111] Once the IJ:norm is determined, the elements of the concatenated latent representation may be scaled by the U" norm value so that the complete latent representation 928 has a Lpnorm of 1. In some embodiments, / may be set to 2, which represents the Euclidean norm. This converts all complete latent representations 928 to unit length vectors, meaning that they represent points on the surface of a multi-dimensional sphere. The use of normalization 912 provides numerical stability, as the encoder / decoder 800 need not handle infinitely large or infinitely small numbers, and it simplifies distance computations between complete latent representations 928. These distance computations are a technique used to train the various encoders and decoders of the encoder / decoder 300 to encode similar things with similar latent representations (e g., by including minimization of angles between the latent representations that are known to represent similar things as a training goal), and similarly to maximize the distance between latent representations that are known to represent dissimilar things.

[0112] The data flow illustrated in FIG. 9A is a non-limiting example embodiment of how context data may be fused with an intermediate latent representation 926. FIG. 9B illustrates another non-limiting example embodiment of such fusion. Similar to FIG. 9A, in FIG. 9B the input data 902 is provided to an encoder 906 to generate an intermediate latent representation 926, and is also provided to a molecular descriptor calculator 908 along with context data 904 to generate descriptors 930. Instead of directly fusing the descriptors 930with the intermediate latent representation 926 as illustrated in FIG. 9A, in FIG. 9B the descriptors 930 undergo embedding generation 914, which converts the descriptors 930 into a latent space. In some embodiments, the embedding generation 914 may be performed by a multilayer perceptron or other appropriate model.

[0113] Normalization 918 is applied to the intermediate latent representation 926, and normalization 916 is applied to the output of the embedding generation 914. The resulting normalized vectors undergo a weighted combination 920 (e.g.,a<i + (1 —wherein Zd is the normalized intermediate latent representation 926, and Zmis the normalized output of the embedding generation 914). The output of the weighted combination 920 undergoes an additional normalization 922 to generate the complete latent representation 928. Each of the normalizations 918, 916, and 922 may be similar to the concatenation 910 and normalization 912 described in FIG. 9A.

[0114] FIG. 9C illustrates yet another non-limiting example embodiment of how fusion of context and the intermediate latent representation 926 may be accomplished. FIG. 9C is similar to FIG. 9B through the normalization 918 applied to the intermediate latent representation 926 and the normalization 916 applied to the output of the embedding generation 914. In FIG. 9C, however, instead of applying a weighted combination 920 to the normalized vectors, the normalized vectors simply undergo concatenation 924 to create the complete latent representation 928.

[0115] While the above discussion primarily describes the use of a single model to generate workflow predictions, latent representations, or generated workflow information, in some embodiments, multiple models may be combined together in a network of agents that collaborate together. FIG. 10 illustrates a non-limiting example embodiment of an agent network according to various aspects of the present disclosure. In the agent network 1000, two or more encoder / decoders 300 may be provided that have been trained on data sets related to specific domains. For example, a first encoder / decoder 300 may be trained on data relevant to paints, a second encoder / decoder 300 may be trained on data relevant to coatings, a third encoder / decoder 300 may be trained on data relevant to polymers, a fourth encoder / decoder 300 may be trained on data relevant to adhesives, and so on. As another example, one or more encoder / decoders 300 may be provided that are trained or architected in a way that addresses specific workflow complexities (e.g., injecting specific types of context information into specific latent representations or encoders / decoders in the model).

[0116] As shown, the agent network 1000 includes a chemical supervisor agent 1002 and two or more domain specialized agents (domain specialized agent one 1004 to domainspecialized agent n 1006). Each of the chemical supervisor agent 1002 and the domain specialized agents 1004 - 1006 may be implemented as an encoder / decoder 300 or using any other model architecture described herein. The chemical supervisor agent 1002 represents an entire workflow, and coordinates the execution of the domain specialized agents 1004 - 1006. The chemical supervisor agent 1002 may be configured to integrate information from the domain specialized agents 1004 - 1006, to manage communication with the domain specialized agents 1004 - 1006, and to ensure that tasks delegated to individual domain specialized agents 1004 - 1006 align with goals of the overall workflow.

[0117] In some embodiments, the chemical supervisor agent 1002 may receive updates from the domain specialized agents 1004 - 1006, adjust parameters, and / or redistribute tasks between the domain specialized agents 1004 - 1006 as desired. As a non-limiting example, in the context of a complex workflow the chemical supervisor agent 1002 may use a result of a given domain specialized agent 1004 - 1006 to help determine a better way to predict an outcome of the workflow, or to generate a further refined workflow to better align with desired goals / properties of the result of the workflow.

[0118] Each of the domain specialized agents 1004 - 1006 may be specialized in handling a specific domain or type of chemical workflow (e.g., paints / coatings, polymers, adhesives, etc.). Each domain specialized agent 1004 - 1006 may use its encoder / decoder architecture to process data specific to its domain, generating a relevant latent representation and decoding the latent representation into actionable information usable by the chemical supervisor agent 1002 (e g., a predicted result, another latent representation, a generated workflow, etc.).

[0119] When a workflow is represented by an agent network 1000. the workflow may be divided into subtasks, with each subtask being assigned to an appropriate domain specialized agent 1004 - 1006. For example, if a chemical process involves coatings and adhesives, the relevant data may be sent to a first domain specialized agent trained to process information relating to coatings, and a second domain specialized agent trained to process information relating to adhesives. Each of these domain specialized agents may generate latent representations that encode features specific to their domains, either as an output and / or as an intermediate result. The chemical supervisor agent 1002 may be configured to access these latent representations, and may be configured to inject context or additional knowledge (as discussed above) to improve performance of the domain specialized agents 1004 - 1006. Each domain specialized agent 1004 - 1006 may continue refining its understanding of the workflow through an iterative process of encoding and decoding. As new data comes in (e.g.. additional context data captured during execution orsimulation of the chemical workflow), the domain specialized agents 1004 - 1006 may adapt by updating their latent representations and sending relevant feedback to the chemical supervisor agent 1002, which in turn coordinates updates across the agent network 1000.

[0120] Various actions that can be performed by an encoder / decoder 300, such as predicting outcomes of experiments and generating updates to chemical workflows, may be extended by the use of an agent network 1000 instead of a single encoder / decoder 300. For example, for predicting experimental outcomes, an agent network 1000 may use the domain specialized agents 1004 - 1006 to contribute additional latent representation features to be decoded for the outcome prediction. By fusing latent representations from different domains, the agent network 1000 can predict experimental outcomes for complex workflows involving multiple types of materials. As another example, for generating updates to chemical workflows, once an outcome is predicted, the agent network 1000 can generate updated workflow parameters, formulation components, etc., to help optimize performance. For example, if an experimental outcome is predicted to fail based on polymer sensitivity to current reaction conditions described in the workflow and detected by a domain specialized agent trained on polymers, the agent network 1000 might generate workflow parameters to adjust a temperature, pressure, or other parameter to make the process more likely to succeed.

[0121] By distributing these tasks among domain specialized agents 1004 - 1006, the agent network 1000 can scale easily across more complex workflows without overburdening any single agent, and can increase the likelihood that agents will be able to be successfully trained on sub-domains while allowing their domain-specific knowledge to be combined for an overall workflow. In some embodiments, a single domain specialized agent may itself be composed of a multi-agent network, thus enabling even a domain specialized agent to be specialized in multiple domains. For example, within a domain specialized agent trained to process polymer information, there may be a sub-specialized agent trained with respect to polymer chemical structure, another sub-specialized agent trained with respect to polymer synthesis processes / mechanisms, another sub-specialized agent trained with respect to polymer use in formulations, etc. Together, these sub-specialized agents would allow the polymer agent overall to analyze various aspects of the chemical workflow as applied to polymers in a multi-scale way.

[0122] Embodiments of the present disclosure have been tested on the prediction of various physical properties of chemicals, with promising results. An embodiment that uses a Uni-MOL transformer model was tested and compared to other model architectures, as well as the T.E.S.T. tool provided by the EPA that uses a quantitative structural activityrelationships (QSAR) model and is the current gold standard for predicting boiling point, melting point, toxicology, and other characteristics. Using a trainable transformer model would be preferable to a QSAR model, as QSAR models are difficult to update, and are generally overfit to the training data.

[0123] Uni-MOL is a transformer architecture that takes as input an atom representation and a Pair representation. Pretraining was performed using two strategies: masking with random coordinates, and re-assignment plus noise range. The internals of Uni-MOL were also manipulated as described above to inject additional information into the latent space. Performance of Uni-MOL compared to T.E.S.T. (i.e., QSAR), and graph neural networks (GNNs) with context injection is illustrated in Tables 2A-2B, 3A-3D, and 4A-4H. A GNN is a graph neural network machine learning model where data is represented in the form of a graph made up of nodes and edges. The nodes represent entities (e.g., atoms, in the case of a chemical molecule, or states / stages / steps of a workflow in the case of a process), and the edges represent connections between the entities (e.g., bonds in the case of a chemical molecule, or dependencies in time / state in the case of a process). A GNN is a type of artificial neural network model which takes a graph as its form of input and performs operations on the graph (e.g.. graph convolutions) to produce latent representations of the graph that can be used to compare different graphs, or can be decoded to make predictions about the data / structure of the graph.

[0124] In the performed testing, it was desired to show that the inclusion of context injection improves models in general, whether using a GNN as an encoder or using a transformer. In nearly all cases, however, the results show7that the transformer outperforms the other technologies.

[0125] While illustrative embodiments have been illustrated and described, it will be appreciated that various changes can be made therein without departing from the spirit and scope of the invention.EXAMPLES

[0126] The following numbered paragraphs describe non-limiting example embodiments of the subject matter disclosed herein.

[0127] Example 1. A computer-implemented method of generating a prediction related to a chemical workflow that includes a process performed on at least one chemical, the method comprising: encoding, by a computing system, a representation of the at least one chemical using a chemical encoder to create at least one latent chemical representation; creating a latent experiment representation based on the at least one latent chemical representation; and decoding, by the computing system, the latent experiment representation using an experiment decoder to predict one or more properties for an output of the chemical workflow, to generate an updated representation of the at least one chemical, or to generate an updated representation of the process.

[0128] Example 2. The computer-implemented method of Example 1, wherein predicting one or more properties for the output of the chemical workflow includes predicting one or more acoustical properties, chemical properties, electrical properties, magnetic properties, mechanical properties, optical properties, or thermal properties.

[0129] Example s. The computer-implemented method of any one of Examples 1-2, further comprising: encoding, by the computing system, a representation of the process using a process encoder to create a latent process representation; and wherein creating the latent experiment representation based on the at least one latent chemical representation includes: encoding, by the computing system, the at least one latent chemical representation and the latent process representation using an experiment encoder to create the latent experiment representation.

[0130] Example 4. The computer-implemented method of Example 3, further comprising: encoding, by the computing system, context data related to the chemical workflow to create a latent context representation.

[0131] Example 5. The computer-implemented method of Example 4, further comprising: fusing, by the computing system, the latent context representation with at least one of the latent chemical representation, the latent process representation, or the latent experiment representation.

[0132] Example 6. The computer-implemented method of any one of Examples 4-5, further comprising: using, by the computing system, the latent context representation while creating at least one of the latent chemical representation, the latent process representation, or the latent experiment representation.

[0133] Example 7. The computer-implemented method of any one of Examples 3-6, wherein at least one of the chemical encoder, the process encoder, and the experiment encoder is a transformer model.

[0134] Example 8. The computer-implemented method of Example 7, wherein the transformer model is a Bidirectional Encoder Representation from Transformers (BERT) model.

[0135] Example 9. The computer-implemented method of Example 7, wherein the transformer model is a Uni-MOL model.

[0136] Example 10. The computer-implemented method of any one of Examples 7-9, wherein at least one of the chemical encoder, the process encoder, and the experiment encoder includes a plurality of domain-specific encoders arranged as an agent network.

[0137] Example 11. The computer-implemented method of any one of Examples 3-10. wherein the experiment decoder is a Gaussian process.

[0138] Example 12. The computer-implemented method of Example 11, wherein the Gaussian process includes a radial basis kernel.

[0139] Example 13. The computer-implemented method of any one of Examples 3-12, wherein the representation of the process includes a structured representation of the process.

[0140] Example 14. The computer-implemented method of Example 13, wherein the structured representation of the process is a structured text string having one or more keyvalue pairs, wherein the one or more key-value pairs include one or more key-value pairs representing one or more process steps and one or more key-value pairs representing one or more parameters.

[0141] Example 15. The computer-implemented method of any one of Examples 13-14, wherein the one or more key -value pairs representing one or more process steps include representations of at least one of: a step name; a step ty pe; a step order; an equipment identifier of a machine to perform the process step; or a setting value for a setting of the machine to perform the process step.

[0142] Example 16. The computer-implemented method of any one of Examples 13-15, wherein the one or more key-value pairs representing one or more parameters include representations of at least one of: a parameter name; a step order; an equipment identifier ofa machine associated with the parameter; an attribute of the machine associated with the parameter; a setting value for the machine associated with the parameter; or a step condition value.

[0143] Example 17. The computer-implemented method of any one of Examples 3-16. further comprising training at least one of the experiment decoder, the experiment encoder, the process encoder, or the chemical encoder, by: comparing, by the computing system, the one or more predicted properties to one or more ground truth properties measured from an executed instance of the chemical workflow to determine a value of a loss function; determining, by the computing system, a gradient of the loss function; and using, by the computing system, the gradient to update the at least one of the experiment decoder, the experiment encoder, the process encoder, or the chemical encoder.

[0144] Example 18. The computer-implemented method of any one of Examples 1-17, wherein the representation of the at least one chemical includes a structured representation of the at least one chemical.

[0145] Example 19. The computer-implemented method of any one of Examples 1-18, wherein the structured representation of the at least one chemical is a structured text string having one or more key-value pairs, wherein the one or more key-value pairs represent one or more of: a chemical name; a chemical identifier; a chemical line notation; a concentration; a property; a unit; a chemical property; or unstructured text related to the at least one chemical.

[0146] Example 20. The computer-implemented method of Example 19, wherein the chemical line notation includes a Simplified Molecular Input Line Entry System (SMILES) string, a Polymer SMILES (PSMILES) string, an International Chemical Identifier (InChi) string, a Wiswesser Line Notation (WLN) string, a Smiles arbitrary target specification (SMARTS) string, or a SYBYL Line Notation (SLN) string.

[0147] Example 21. The computer-implemented method of any one of Examples 19-20, wherein the chemical identifier includes a CAS identifier.

[0148] Example 22. A non-transitory computer-readable medium having computerexecutable instructions stored thereon that, in response to execution by one or more processors of a computing system, cause the computing system to perform a method as recited in any one of Examples 1-21.

[0149] Example 23. A computing system configured to perform a method as recited in any one of Examples 1-21.

Claims

CLAIMSThe embodiments of the invention in which an exclusive property or privilege is claimed are defined as follows:

1. A computer-implemented method of generating a prediction related to a chemical workflow that includes a process performed on at least one chemical, the method comprising: encoding, by a computing system, a representation of the at least one chemical using a chemical encoder to create at least one latent chemical representation; creating a latent experiment representation based on the at least one latent chemical representation: and decoding, by the computing system, the latent experiment representation using an experiment decoder to predict one or more properties for an output of the chemical workflow, to generate an updated representation of the at least one chemical, or to generate an updated representation of the process.

2. The computer-implemented method of claim 1. wherein predicting one or more properties for the output of the chemical workflow includes predicting one or more acoustical properties, chemical properties, electrical properties, magnetic properties, mechanical properties, optical properties, or thermal properties.

3. The computer-implemented method of claim 1, further comprising: encoding, by the computing system, a representation of the process using a process encoder to create a latent process representation; and wherein creating the latent experiment representation based on the at least one latent chemical representation includes: encoding, by the computing system, the at least one latent chemical representation and the latent process representation using an experiment encoder to create the latent experiment representation.

4. The computer-implemented method of claim 3, further comprising: encoding, by the computing system, context data related to the chemical workflow to create a latent context representation.

5. The computer-implemented method of claim 4, further comprising:fusing, by the computing system, the latent context representation with at least one of the latent chemical representation, the latent process representation, or the latent experiment representation.

6. The computer-implemented method of claim 4. further comprising: using, by the computing system, the latent context representation while creating at least one of the latent chemical representation, the latent process representation, or the latent experiment representation.

7. The computer-implemented method of claim 3, wherein at least one of the chemical encoder, the process encoder, and the experiment encoder is a transformer model.

8. The computer-implemented method of claim 7. wherein the transformer model is a Bidirectional Encoder Representation from Transformers (BERT) model.

9. The computer-implemented method of claim 7, wherein the transformer model is a Uni- MOL model.

10. The computer-implemented method of claim 7, wherein at least one of the chemical encoder, the process encoder, and the experiment encoder includes a plurality of domainspecific encoders arranged as an agent network.

11. The computer-implemented method of claim 3, wherein the experiment decoder is a Gaussian process.

12. The computer-implemented method of claim 11, wherein the Gaussian process includes a radial basis kernel.

13. The computer-implemented method of claim 3, wherein the representation of the process includes a structured representation of the process.

14. The computer-implemented method of claim 13, wherein the structured representation of the process is a structured text string having one or more key-value pairs, wherein the one or more key-value pairs include one or more key-value pairs representing one or more process steps and one or more key -value pairs representing one or more parameters.

15. The computer-implemented method of claim 13, wherein the one or more key-value pairs representing one or more process steps include representations of at least one of: a step name;a step type; a step order; an equipment identifier of a machine to perform the process step; or a setting value for a setting of the machine to perform the process step.

16. The computer-implemented method of claim 13, wherein the one or more key-value pairs representing one or more parameters include representations of at least one of: a parameter name; a step order; an equipment identifier of a machine associated with the parameter; an attribute of the machine associated with the parameter; a setting value for the machine associated with the parameter; or a step condition value.

17. The computer-implemented method of claim 3, further comprising training at least one of the experiment decoder, the experiment encoder, the process encoder, or the chemical encoder, by; comparing, by the computing system, the one or more predicted properties to one or more ground truth properties measured from an executed instance of the chemical workflow to determine a value of a loss function; determining, by the computing system, a gradient of the loss function; and using, by the computing system, the gradient to update the at least one of the experiment decoder, the experiment encoder, the process encoder, or the chemical encoder.

18. The computer-implemented method of claim 1, wherein the representation of the at least one chemical includes a structured representation of the at least one chemical.

19. The computer-implemented method of claim 1, wherein the structured representation of the at least one chemical is a structured text string having one or more key-value pairs, wherein the one or more key-value pairs represent one or more of: a chemical name; a chemical identifier; a chemical line notation; a concentration; a property; a unit; a chemical property; orunstructured text related to the at least one chemical.

20. The computer-implemented method of claim 19, wherein the chemical line notation includes a Simplified Molecular Input Line Entry System (SMILES) string, a Polymer SMILES (PSMILES) string, an International Chemical Identifier (InChi) string, a Wiswesser Line Notation (WLN) string, a Smiles arbitrary target specification (SMARTS) string, or a SYBYL Line Notation (SLN) string.

21. The computer-implemented method of claim 19, wherein the chemical identifier includes a CAS identifier.

22. A non-transitory computer-readable medium having computer-executable instructions stored thereon that, in response to execution by one or more processors of a computing system, cause the computing system to perform a method as recited in any one of claim 1 to claim 21.

23. A computing system configured to perform a method as recited in any one of claim 1 to claim 21.

Images